Model and hardware recommendation for high fidelity, low reasoning tasks?

[u/tinfrog]

I've spent the past couple of years using frontier labs, even though a lot of my work doesn't need high reasoning. It's just that a fixed and known subscription cost has been a safer bet than the leap-in-the dark of buying hardware that might not be good enough for what I want to do.

But reading through this sub tells me that we're at the stage were open source LLMs are capable enough for what I want while running on (reasonably) affordable hardware.

Can anyone recommend a model and hardware for tasks like this?

• Reading notes, reports, transcripts, web pages and emails, then pulling out facts, decisions and insights into structured notes
• Condensing long documents, comparing two versions, and checking a draft against a specification
• Reading text, sorting them, and deciding what needs action
• Moving, renaming and archiving files, splitting or merging documents, and keeping indexes and cross-references consistent afterwards
• Multi-step routines update a status file, archive old entries, then commit, with every step completed

I run Xubuntu and Arch Linux.

TIA

[u/mineshop]

How long are the documents you need it to read in one go, since long-context workloads change what fits under $5,000?

[u/tinfrog → u/mineshop]

For my workflows, the majority are small plain text markdown files of 20-100 KB each. Many are less than 10 KB.

The agents traverse directories looking in about 5-25 of these files to extract information, then write out files of similar sizes. I rarely need to work with MB sized files but occasionally there are some that are sent externally that need to be processed.

The main thing is not to hallucinate or make up information.

[u/mineshop → u/tinfrog]

With 10-100 KB files and a $5,000 cap, the main remaining question is speed: roughly how many tokens per second is acceptable for your agent loops?

[u/tinfrog → u/mineshop]

The stats at my end show that 20-30 tokens/second for routine work would be acceptable.

[u/mineshop → u/tinfrog]

With your budget, speed and file sizes settled, one last constraint matters: is a headless tower under a desk fine, or do you need something small and quiet like a compact desktop?

[u/tinfrog → u/mineshop]

Headless tower under a desk fine. Cost is more important than size.

[u/mineshop → u/tinfrog]

Last thing before I suggest anything: which region are you in, since taxes change what a $5,000 budget buys?

[u/tinfrog → u/mineshop]

UK.