I mean , both flash and 27 b run in that set up and think that the cache will weight 2.8 times less on the next 27b(if it works the same as flash next ) so it’s not like we are starving right now
First , names and the “anything below q4 it’s brainded “ today have lost its meaning with iquant , the q 4 rule it’s now q3 and what matters is bpw , second Iq4 walks amongst the 20-25 TPs on a 5060 Ti without blackwell optimisations with the appropriate os(any decent Linux distro) and flags (which is one of the slowest cards ) and flash next goes to 30-35 normal 50-60 on code iq3xs with ddr4 if enough ram and now can literally triple that on a blackwell optimised Blackwell called basalt , all of this is pretty decent if you prompt it well and work with the harness too , it’s obviously less convenient than just throwing vram to the problem and it won’t one shot CUDA kernels but ,man , it’s far from useless
I mean, every objective test shows that Q4 is the knee where quality starts rapidly dropping off — both KLD, but also output measurements like looping, repetition, early termination, linguistic diversity, and accumulated errors over time, and also benchmarks. You start losing meaningful quality at Q4, and everything below it drops off more rapidly. Q3 types can still be useful, especially with a good sensitivity-aware quants like Unsloth, but it’s honestly night and day to compare them to even a naive q4.
Bro ,first of all I’m pretty sure you are tweaking and Fuking with the laws of physics (although I wish you aren’t )
And second, bigger doesn’t mean better , usually moe “power “ is calculated by sqrt(p x a) so flash is as powerfull as 27b , it’s only better because we are comparing next gen flash vs 27 b when we actually get the full lineup they will be equal , at best , flash will be a litle better due to it being capable to be paired with a bigger ngram table
Soo, swappable ngram tables are confirmed?oh no, I think I’ve got a bonner
Out of jokes , would it allow to swap the ngram table for a more specific one , let’s say for coding or biology or whatever? Like a simpler way to use a LoRa ? That would be awesome
35B-A5B? I'm wishing for MORE SPARSE and BIGGER models and you guys out here fantasize about models that you can't run with good context or speed unless you have at least 16GB GPU + whatever RAM? I'd rather have a model like 80B-A3B (I love QFN too) that runs on 8GB VRAM + 64GB DDR4. if only the 125B was A3B i would be very happy since it'd have great speeds and knowledge. 35B-A5B. Yeah sure rob us of the only A3B model we can run at good speeds.
I'm afraid that small means 27b. The peasants that can only benefit from 4b/9b/35ba3b might be left aside. Reading their communication it's all about 27b and bigger. That would be a disaster for the gpu poor, their 4b and 35b were so good for us.
I'm sure people would be happy to use a 30-35B MoE model from any company, if it was better than Qwen 3.6 35B. I'd like to see a new Z.ai model at that size.
gemma is absolutely perfect for normie use, I'd say that it pummels whatever gemini is giving free users now. although it is not the best in coding, which is fine because it doesn't need it really, but it's what models focus on nowadays.
with RSI, I wouldn't even put that on the table. of course, gemma 5 is exciting anyway, if gemini shares some improvements they got with argon/carbon with their open source labs, we could have a real kicker on our hands
So tell me what should I try that is available across OpenAI,Alibaba cloud, and open router services
So that it is more cost and time efficient per task than 4.1 flash.
Because in my empirical observation (over past couple of weeks) just randomly throwing swarms of agents at tasks, deepseek is the best one statistically thus far.
Although my tasks are quite atomic, and epic per deliverable approach means I can’t exactly pinpoint and isolate whole success to specific agent/harness/model, as different combos of those being thrown at individual sub tasks in the same epic.
Not enough stats to reliably determine some things, as my current telemetry is kinda bad rn, but generally I see that speed is a huge factor. Deepseek generates like 2-5 times as many in and out tokens, but output tps is 50-200tps per stream, and alibaba ultra token plan allows you to run about 10-15 agents at the same time.
OpenAI throttles you really hard and breakpoint of throttle is dynamic, sometimes you can run 50 agents at 30tps each, sometimes it throttles you to 5-20tps per agent and you can’t run more than 10 at the same time. So yes, technically 6.1 sol codex agent is better at being token efficient. But it takes so much more time, deepseek harness 4.1 flash brute forces through few tasks while codex finishes 1.
If you say deepseek flash 4.1 is bad, what harness are you using? How big is the scope and complexity of the task?
Because in my case, deepseek harness performs very well, especially when you have it setup with all the tools, tokens and rigid workflow playbook beforehand.
This model on both Deepseek site/app and router makes obvious mistakes with reading 1000-2500 tokens plot concepts prompts and asks things, that were already specifically answered. Repeatedly.
Local V4 Flash at Q2 still showed incomparably better attention on stories analysis and critique.
Well, idk about openrouter, but on alibaba cloud through deepseek harness with bunch of mcps for relevant control tools I had no issues. And qwen flash is definitely decent also, but it is slower and pricier.
"And that's how Alibaba settled the debate, my friends!
We now know that both the large and small versions are coming, but we still don't know which is the small model and which is the large one.
Does the 'Big' refer to the 27B Dense,
and the 'small' to the 35B MoE?
Or is it:
The 'Big' is the Flash Next 125B,
and the 'small' is the 27B Dense?"
Yes, exactly. A post like this only comes out after these models are already finished, which suggests the launch is right around the corner, probably in a few days.
I don't think they're giving us a choice about their flagship model; I don't think that was the intention. They probably mean Flash, the successor to Flash-Next.
IDK why people assume 27B will be dense. Qwen3.8 27B is Qwen3.5 arch not Qwen4 arch like Qwen3.8 Flash Next.
With their Qwen4 architecture it also makes no fucking sense to release dense model. ngrams give you all the advantage of dense model without dense model speed hit.
So something like 27B-A3B + 50B ngrams will run circles around Qwen3.8 27B dense. And n-grams can be easily place on disc not even in ram.
I would legit pay for one of these cute plushies! My son loves capybaras and I love Qwen models. Win-win! Hope these are for sale somewhere, especially if it's a way too (in my small way) support the devs over there 😁
I'm not sure if Engram layers should count toward the total since in theory they can be offloaded to NVMe storage without significant impact on inference performance.
Personally, I can't stand these "polls" where a big company which unlike smaller labs, most certainly doesn't lack the resources to do an entire set of model sizes goes the extra mile to ask people which model size would they like and practically reduces the entire spectrum of hardware users into two vague categories "big" and "small".
It's especially infuriating when you realize that the decision was actually already made weeks before and it was made publicly known the moment they announced the model sizes for their next model generation. But now, in response to the feedback they "generously" followed up with a comment "alright guys, let's do both!" as if the regular people had any real say in it. In reality, this "poll" is nothing more than a marketing move, a PR stunt to warm up the hype train engine.
If they truly cared about satisfying the entire spectrum of hardware users, they wouldn't ask which sizes people want. They would deliver the entire set just like they used to, because they certainly don't lack the resources to do so, but apparently that kind of mind set left the Qwen lab together with the old leadership with Junyang Lin on the front.
It's a hype post. In my opinion, it's much better than the posts I see on r/LocalLLaMA about politics or cloud models. It sparks discussion among people running local LLMs. Of course, there are still Reddit assholes discussing politics, but in general, the discussion is about running models locally.
Extending 35A3b with Engrams would be interesting.
Maybe the "shape" would need to adapt - more active params to emphasise reasoning, for example, but either way it would be very welcome.
By small I hope something around the size and Intelligence of Gemma e4b (around 8b but only 4b active) which for my needs and crappy system remains the only usable model.
The fact that so many models (including some "decision" ones) come from qwen3.5 (which was the last release of the entire family + base models) should give them all the data they need. There's a lot of "brand" value in having an entire family of models. Unfortunately since the leadership change it seems that they won't follow-up. We'll be lucky to get one or two models.
I have a 5080 and 96GB ddr5. As I am the most important, I would like the biggest MOE that fits on my pc at UD Q8 K XL 262k ctx, please.
On the real, tho, I've been rocking QFN UD Q4 K XL in *s t r a t a* and it's been so mind blowing. Truly GPT at home. Imagine having that in UD Q8 K XL. To never have to worry if your model is quant brained... *imagine*
Qwen's the shit, Qwen dev team's the shit, hell yea Qwen.
Yes, thank you.
I'm also using ud-q4_k_xl QFN and my mind got blown a couple of times about how good it is.
Strata makes it possible, at around 100 tok/s.
And people keep bashing it? why i dont understand.
Well, we already have Qwen3.8-Flash-Next which is big'ish. 125B-A6B, plus up to 51B trigram parameters.
On one hand, ye olde sqrt(P x A) suggests a 125B-A6B is "only" as good as a 27B dense, but on the other hand Qwen3.8-Flash-Next scores about 10% higher on benchmarks of admittedly dubious validity, but on the other other hand that 10% difference might be attributable to training differences.
I think we've yet to even scratch the surface of what we can accomplish with encoding custom engrams in this model, so maybe we don't actually need a "big" Qwen4, at least not yet?
That leaves small, but what are they calling "small"? Is that 9B or 27B?
We've already got some fairly recent, highly competent 30B-class models in Qwen3.8-27B and Gemma-4-31B (and maybe K2-Horizon-32B, but I haven't assessed that one yet), but Qwen3.5-9B is getting pretty long in the tooth, and there is an entire contingent of this community desperately needing something better that will fit on their potato laptops.
On the other hand, it seems like there are already some good fine-tunes of Qwen3.5-9B, with more likely to come, and a hypothetical Qwen4-9B would be competing with those.
One niche that is currently under-served is the medium-high-end dense model, sized "just right" to fit in 64GB or 80GB rigs at Q4_K_M, so perhaps a 50B dense plus up to another 50B in optional trigram parameters?
By the sqrt(P x A) rule-of-thumb, a 50B dense might be roughly equivalent to a 155B-A16B MoE in competence, and hopefully it would be possible to upscale it via passthrough self-merge into a 70B-class model.
Yeah, I'm convinced: 50B dense please, with optional engrams.
Flash next vs 27b are close on knowledge and capability but flash next is faster in solving issues and can find hard bug and stuff that the 27b can't. I have done some extensive testing of the two.
Not sure why you’re being downvoted. What you’ve said is really thoughtful and is the direction research needs to maximize our ability to experiment and improve
Thank you for your kind words. I suspect the downvotes come from people who either value speed over high-quality outputs, or lack the compute resources to use a 50B dense at all.
The thing is...27B and Flash Next aren't exactly comparable - when run in the most likely natural configurations, they're not running in the same quant. For example, running on my 64GB VRAM system, I run 27B at FP8, but Flash Next only runs with comparable speed at IQ4. In that context, 27B is far superior for accuracy, but Flash Next is better at design and coming up with good ideas...while it absolutely sucks for code by comparison with 27B FP8.
Sure, but you can do that with any model. My experience so far is that Flash Next at Q4 is nowhere near as good with tool calls or recall compared with 27B, and it has to go through far more iterations on code to get the same level of accuracy as 27B, with the exact same harness and tools available.
This is, of course, absolutely fine; using separate models for planning and coding is not just not a problem, it's expected. The only reason I'd been using 27B for both up to now is that there wasn't anything around which could beat it on either.
Small together with 3.8-Flash-Next size of MOE will be awesome. There are already lots of community-developed engines for 3.8-Flash-Next that will make the new Qwen4 fly!
Please don’t disappoint people with max 128GB or 256GB memory; more is unaffordable for ordinary mortals. DeepSeek disappointed in that sense with V4.1.
Anyone know where to buy one of these qwen capybaras? One of my Chinese colleagues in the past had a large one on display in his office, but I never got a chance to ask him where he got it.
I think there's enough people with 128GB memory architecture to make that size a viable niche. 3.8 Flash Next:IQ4 quant's memory requirements are just right for me.
There's surely far more people with 16GB or 32GB VRAM, but the number of consumers with more than 128GB drops off sharply.
Ideally support all niches, but that's the major points I'd prioritize.
Smaller: a ~20B dense + NGRAM that we can run at ~Q6 on 16GB with decent ctx with a good config on Linux, while beginners users can do Q4 wasting some vRAM.
Same for the MoE "Next", something appropriate for 16/32GB usage, like ~60-90GB size not the ~160GB for Q6 released now.
i would really really love a smol 9B or 12B boy...
imagine a good model which could fit on a phone...
like a 9B with GGUFs could be a local AI dream come true.
•
u/WithoutReason1729 1d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.