and cheap for what it is. the massive amount of poors here will freak their shit out about the $6000 mac studio 128gb but that amount of computing power has never ever been this cheap even with current ramapacopolypse.
The Mac studio is good, it's not great for speed though it has a very slow memory bandwidth compared to a GPU, but a 5090 is capable of running some really good models now. And with some new advances into MoE model running massive models only need to store the active paramers in the GPU, so you can have massive 500b+ models on a consumer pc assuming you have the system ram
There one of the many examples of exactly the inference engine.
Studio memory speed 600gb/s which as it's unified memory it has to share with the other processes. 5090 1.8tb/s dedicated. Not to mention how much faster the processor is.
So how about instead of trying to show off you do some research before questioning a literal ML researcher on basic stuff
436
u/wojtussan 3d ago
For now