Qwen 3.8 on my own machine

A few days ago I pointed Qwen 3.8 at my own machine and set out to find out how it actually feels in daily use, not how it scores on a benchmark. After a couple of days, the striking thing wasn't that it's good. It's that it stopped reading as a local model at all.

The setup

I'm running it on a desktop with an A6000, which is by no means a new card, and I'm using the Q4_K_V quant from Ollama with a plain chat interface on top. There's nothing in that stack where I spent time, no sampling experiments or context tricks, and I think that's worth saying up front. A local model you have to prepare and coax before it's useful has already lost half its appeal, so the fact that I can treat this one like any other model without a second of thought is a substantial part of why I keep choosing it.

The speed did the convincing

The quality is good, but honestly it was the pace that changed my mind. On a card this old it still runs snappy, and the loop of typing, reading, and typing again never stalls in that dead air that used to kill local models for me. A capable model that's also slow is a model you close after a single session, and this one stays open. It sits in the background while I work, and I reach for it the way I'd reach for a search bar, which is about as far as a tool gets from the category of something you have to remember to fire up.

What I actually use it for

Most of it is the everyday. Drafts and rewrites, tidying a messy thought into something I can actually send, explaining a stretch of code. None of that requires a frontier model, and I've found that a local model that does this well is quietly more useful to my day than the ones that are impressive in a demo and only shine under stress. This handles the daily stuff to the point where I often forget which side of the line the answer came from.

The job I rely on it for most is writing my image and video generation prompts. I give it a loose idea, and it comes back with structured prompts that already carry real art direction in them, the lighting, the mood, the material, what to cut. It's the single use case I'd be most annoyed to lose, and it lands reliably.

Where it's not there yet

I'll be straight about the edges of this. I haven't stress-tested the deep reasoning end, so I'm not going to claim I've measured it against the frontier, and for the hardest problems I'd still route them to the big hosted models. For what I throw at it, the intelligence reads as fine.

What I do find consistently limiting is the context window, which is smaller than I'd like. On short tasks it never registers, but the moment a job pulls in a lot of material it starts to bite, and that's the one thing I genuinely want to see improve before I'd build anything serious on top of it.

The part that got me going

What excited me most is that it doesn't stop at a chat box. Right now I'm coding through a remote agent on my MacBook, and that agent reaches the model on my server, so I work on the laptop while the inference happens back on the machine, and the whole thing feels fast and clean. Pointing the agent at that local endpoint took minutes, not a project.

And that's where the bigger shift is, even beyond the security side, which is genuinely nice but rather the easy win. The real change is that this has moved from a compromise to a default for me. I use it because it's there and it works, not because it's the principled choice, and the absence of a token meter to watch or a per-call cost creeping up changes how much I'm willing to play with something. When running an idea costs me nothing, I end up running a great many more of them.

The catch

The catch is the part nobody shows you in a demo: the hardware. This wants an A6000 or something close to it, real VRAM and a real amount of money, and that entry ticket is a hard no for a lot of people.

There's genuine movement toward models that run on smaller amounts of VRAM, and I think the direction is right. It's just not usable yet, either too slow or too much of a quality drop to be worth the trouble, and while I'm watching it I wouldn't tell anyone to build on it.

So where does that leave me? For me local AI has crossed from something interesting to keep around to something I'd genuinely miss if it weren't there. That's a real change. The honest footnote is that it's a change only available to people who already have the hardware in their life, and for everyone else the gap is still the whole story.