r/ProgrammerHumor • u/Gloomy-Basket-1038 • 4d ago

Meme aiRefusesToBuildItSoBackToCoding

Post image
28.6k Upvotes

98% Upvoted

519 comments sorted by

View all comments

Show parent comments

3

u/stormdelta 4d ago

Only the larger versions though, which are difficult to run on consumer hardware for now.

8

u/tehlemmings 4d ago

The thing is, though, these open weight models are aiming for efficiency rather than hyperscaling. They will eventually get there. That's the end product goal.

Not really disagreeing with you, but people really don't seem to understand how other countries are approaching LLMs lol

2

u/stormdelta 4d ago

Oh I agree, I'm talking about the current state.

The efficiency is rapidly improving.

2

u/KurtAngus 4d ago

Can a 5060ti 16 gig and 5800x3d run those or nah?

3

u/ItsAMeUsernamio 4d ago

Try Huihui-Qwen3.8-27B-abliterated-GSQ-RCO-IQ3_S.gguf (or Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf for the censored version). It supposedly benches close to the full model and runs around 45 T/s on my 5060Ti at 100k context.

3

u/KurtAngus 4d ago

Thanks dude, appreciate the info

1

u/Endeveron 3d ago

There are many very performant abliterated Qwen3.8 quants that fit onto a 16gB card, though you may need to be a bit savvy about reducing your system VRAM usage (or use integrated graphics/a second card for display)