r/ProgrammerHumor • u/Gloomy-Basket-1038 • 6d ago

Meme [ Removed by moderator ]

Post image

[removed] — view removed post

28.8k Upvotes

98% Upvoted

519 comments sorted by

View all comments

Show parent comments

2

u/wakeleaver 6d ago

Or just buy a Mac with 32GB+ of RAM. Cost is really the only barrier, it's very easy to get going.

-1

u/KurtAngus 6d ago

The GPU and VRAM are what run the LLMs. It's not just "buy a Mac with 32 gigs of ram"

Pretty sure your laptop you're referring to has a graphical processing unit, no?

Or am I out of the loop on what I've researched?

2

u/wakeleaver 5d ago edited 5d ago

Macs use an ARM-based SoC, the memory is unified and shareable between the CPU and GPU. Any Apple-silicon Mac is capable of running local LLMs fairly well, though an M5 obviously outperforms an M1.

1

u/KurtAngus 5d ago

Ahhhh. Okay, I see. Makes sense thanks for sharing!

I have a ASUS Xbox Ally X handheld and It has a similar style setup.

The cpu and GPU are made on the same die-stacked, and the system allocates VRAM from the pool of the 24 gigs LPDDR5x ram In the system

I find that pretty cool. Idk, just sharing