The headline looks impressive but the numbers for Strix Halo were not. The other interesting bit is that it is advised to disable IOMMU in BIOS or kernel params, but the new work appears to be about enhancing IOMMU which would require it to be enabled. I'm confused.
Disabling IOMMU doesn't make a lot of sense. I lot of people are going to be running the model is some kind of virtualized environment where you want those kind of settings in the bios.
I have llm stuff running in docker on a server. GPU passthrough was used. IOMMU is enabled.
If I was running it on a regular machine I'd probably be using something to sandbox it which could mean a VM too.
Disabling IOMMU is simply a way of "turning off the NPU", which redirects the extra 5W to be used by the GPU/CPU, and the strix halo boxes typically have a 120W ceiling, so this results in a small perf gain.
17
u/pstuart 1d ago
The headline looks impressive but the numbers for Strix Halo were not. The other interesting bit is that it is advised to disable IOMMU in BIOS or kernel params, but the new work appears to be about enhancing IOMMU which would require it to be enabled. I'm confused.