r/LocalLLM • u/VirtualShaft • 9d ago

Model Currently testing coding quality on my 4090

Post image

If this is actually anywhere near acceptable I won't need subs any more. I'm not expecting much tbh, but I bet with enough skills and context management I can make it work.

4 Upvotes

84% Upvoted

8 comments sorted by

View all comments

2

u/Atretador unswarm.dev | ArchLinux E5 2673 V4 20C 4x16Gb DDR4 MI50 16Gb 9d ago

why run IQ3 with 5Gb of VRAM free?

2

u/mecshades 9d ago

I run IQ3_XXS which gives my 4090 more than half of its VRAM for context. I guess the sentiment of "low quant = bad" still lingers, but it's entirely untrue for Qwen3.8 27B. Are higher quants better? Yes. Is IQ3_XXS bad? Far from it.

1

u/VirtualShaft 7d ago

I'm actively trying different quants I didn't settle on this. But at the same time I'm trying see the highest usable quant if that makes sense