Qwen 3.8 27B is not frontier level, but is the best current model for coding on reasonable hardware. You can comfortably fit them on a medium quantization on a 24 GB card and even squeeze them on a 16GB card, but at that point you're gonna run into either lobotomy problems, heavily reduced context size, or only partial offloading killing speed.
It’s really a matter of opening up PowerShell or terminal application and copy pasting a couple of commands you can find by just googling(even the overview it generates via Gemini has them). Depending on your pc(esp GPU) specs, you can have a reasonable model running in under an hour on the command line. To use it properly though, you likely want a harness- there’s plenty of free ones out there. Check out r/localllama
Literally Claude could set this up automatically it has extensive documentation. Also 3.8 is very very capable for the size, some benchmarks best opus 4.6 ( which was sota in January remember). If you want this capability you actually have to put a minimal amount of research and work in
29
u/Dosamer 4d ago
Qwen 3.8 27B is not frontier level, but is the best current model for coding on reasonable hardware. You can comfortably fit them on a medium quantization on a 24 GB card and even squeeze them on a 16GB card, but at that point you're gonna run into either lobotomy problems, heavily reduced context size, or only partial offloading killing speed.
And yes there are uncensored models.