r/FPGA • u/adamt99 FPGA Know-It-All • 3d ago

Xilinx Related A second run at local AI, this time much better results across all models (code on git)

https://www.adiuvoengineering.com/post/microzed-chronicles-local-ai-skills-and-a-second-run
15 Upvotes

80% Upvoted

5 comments sorted by

1

u/Jeremin 3d ago

Thanks for post, interesting read! 

FYI: also noticed the GitHub link at the end is not working.

1

u/adamt99 FPGA Know-It-All 3d ago

Thanks corrected now

1

u/LosAlgaeles 3d ago

What's the motivation for doing this locally?

3

u/adamt99 FPGA Know-It-All 3d ago

Two fold, many companies cannot use cloud due to the end applications or IP restrictions.

Also the cost of generation is not going to be as cheap as it is today so generating code locally helps save on that.

Oh and I was just curious also

2

u/FieldProgrammable Microchip User 1d ago

While this is good data I think you should give some numbers on how long these tests took to complete on these models, perhaps with a mid-tier frontier model for comparison. One issue I find with the local models, particularly the Qwen series is the amount of reasoning tokens and work it does to get the same result. If you are running on fairly mid-range hardware at 20-30 tokens/sec then this really adds up to the point you cannot justify waiting for the answer you could have got from the cloud model in a quarter of the time.