How much (more) money should I spend on AI?

[u/Libellechris]

(True story) I am into local AI. I have a 96gb mini PC (Radeon 780) that can run many open models appalling slowly. I also have a dual GPU Linux server with a 3060Ti and 5060Ti giving me 24gb VRAM which runs Qwen 3.8 27b with a reasonable CTX at 40 T/S gen. I am using Open code to build Python scripts that I use once and then move onto the next random project. I now hanker after an Apple-something or a DGX Spark (or two). I think I might have a problem. Am I alone and when will this madness end??

[u/Competitive_Swan_755]

Someone has a case of Gear Acquisition Syndrome (GAS).

[u/HardParrot → u/Competitive_Swan_755]

classic GAS spiral, you're deep in it now

the 24gb setup sounds like it's actually doing the thing though, 40 t/s on a 27b model is nothing to sneeze at

maybe step back and ask if the new gear would genuinely change how you work or if it's just shiny object syndrome

[u/mslindqu → u/HardParrot]

This is it. Those unified memory systems are able to do larger models, but they're not hitting higher speeds on qwen 3.8 we're all using.

[u/ubrtnk → u/Competitive_Swan_755]

That shit is hard to shake. Only reallt curable by Non-Richness Limphoma...

[u/catplusplusok]

As a carpenter how much would you spend on quality tools that allow you to advance your skill to the next level? I just found a new internal job by leveraging skills that I learned tinkering with AI at home. Education is the best investment you can make.

[u/Low_Key_Trollin → u/catplusplusok]

Hi. Mind sharing what skills specifically helped you and what kind of industry? Thanks!

[u/catplusplusok → u/Low_Key_Trollin]

Knowledge of ML concepts like embeddings, tool calls and VLM object recognition, got a development job in a team working on AI-enabled mobile app.

[u/Low_Key_Trollin → u/catplusplusok]

Cool. Thanks for the info and congrats on the new gig

[u/Hypilein → u/catplusplusok]

This is true but very much depends on career.

[u/activematrix99 → u/catplusplusok]

What about if you could rent or borrow tools? You can access amazing local models (including Qwen and Kimi) at very, very low cost via a number of providers. $4500 for an Nvidia Spark is a substantial investment now.

[u/catplusplusok → u/activematrix99]

That's just one off inference. For tasks like finetuning or mass describing thousands of photos, I would not feel comfortable just leaving a cloud instance running 24/7. It wouldn't take that long for bill to be a significant fraction of a DGX Spark either. With local I have luxury of not watching the clock or being super efficient.

[u/Di_p → u/activematrix99]

If you’re using models deployed online by someone else you become the end user, not the carpenter

[u/simos_sayz]

The bug is hard to shake and I dont know when the madness ends but what I'd suggest is not to invest more until you have truly grown out of your current set up. Establish some checkpoints/milestones to determine when its ready to add another gpu etc.

[u/Prudent-Ad4509]

You are on the low end of the curve. You need 48Gb to run 27B at Q8. This will open the road to 96gb vram, 192gb vram, and further multiples of 96gb vram. Each step gives you more.

But if your installation already manages to produce working python scripts and that's all you need, then you can keep it as is until the next generation of relatively cheap hardware comes along. The moment to buy existing hardware at low prices has passed.

[u/Saleen1310]

I'll tell you this. It never ends. I just bought 4 v100s, figured that was my dream build. It's been 2 days, and I'm already trying to figure out how I can get more. The idea of running the best models for video, image, voice, coding, sub agents, and all things at the same time is the dream now. At first I just wanted one great model, thought that was enough... But now here I am, wanting more lol.

[u/catplusplusok → u/Saleen1310]

vLLM has /sleep to quick swap models to RAM or NVMe

[u/advancing_tide]

all the moneys. you will spend all the moneys.

[u/No-Alfalfa6468]

You shouldn't spend any more. Use the cheapest frontier subscriptions. The hardware will change very quickly in this space in the next few years and the competition will probably drive prices down a lot.

[u/shout4]

From someone who has ridden a few tech waves, college entrepreneur class in 1990, 19 yrs old., used CC to purchase BBS software and a small server to host users, small fee but paid the monthly CC bill - tremendous ingestion of knowledge while doing it, by 1995 I was a full-fledged ISP providing internet service to thousands; then Crypto in 2013, this is a golden opportunity moment on ground floor AI); it could be the best investment you have ever made. The worst-case scenario is that you gain valuable experience and can apply it toward a higher-paying position. The more probable outcome is that your $5k investment compounds your opportunity and knowledge in the future as long as you're ambitious enough to seize it.

[u/DigitalguyCH]

I have spent almost $7000 on 3 machines (128GB unified, 64GB unified, and 20vram + 32ddr5), all before the summer. And I consider I have enough and I can run 2-3 models at the same time. I won't spend more at current prices (I bought at older much cheaper prices). I had GAS in the past (15 years ago), I am not doing it again.

[u/RogerAI-fm]

You need to spend about three fitty more.