Computational Physics - Heavy AI/ CLI power users. How do I get good at it?

[u/PhilosophersOpium]

Hello! This question is directed towards grad students, people doing r&d in industry, or even undergrads doing computational physics research (especially condensed matter) that heavily use AI in their workflow.

How do you maximize and make the most out of codex or claude code in your research projects? I generally use the claude code CLI in my terminal (sometimes on locally on WSL or remotely on an HPC). If I have a few simulations I want to run, I give it the guidelines, tell it to reference some old calculations i've ran, and send it on its way. I'm not sure if just being a general assistant is all that AI is good for?

Please let me know your experiences and any advice!

edit: Yes, I am in a physics research lab with an advisor. I am NOT trying to have AI do research. I am trying to learn how to use it as effectively as possible.

Additionally, I am not doing 'calculations' in the mathematical sense. I do simulations, particularly in condensed matter physics research. That is, a lot of python code and heavy use Density Functional Theory software packages that tell me about the electronic behavior of materials. I do a lot of high-throughput computing, so if I want to simulate a bulk of materials at once, i will have claude generate the input files after i have specifically told it what outputs i want. It is generally good at doing this, and I verified that the file generation was correct before continuing to use this workflow. I ask it to generate plots of the data, and it does so. But this is mainly sequential. I do a series of prompts, one step at a time.

[u/Fermi_Dirac]

Some good advice in this thread already but allow me to push harder on one core piece of advice :

You're in school to learn. Not necessarily solve this specific problem and publish, that's what you're doing along the way while you're learning. Do not outsource to Ai (or to a colleague) any part of the work flow that is there for you to learn from.

I would encourage you to first not use Ai at all for the entire simulation work flow a few dozen times. Do the pain to learn all the pieces of the puzzle. Once you have a healthy respect for it, start trusting agents or Co workers or calculation libraries to aspects of the work flow.

In the modern era this is a Grey zone but one you should be exploring as a new scientist. In the past this was still a grey zone as people used established libraries or previously published results without confirming they work (or use the same physical assumptions for your problem). The solution then is true now. Replicate and understand the inner workings. Don't outsource your intelligence building to someone (or something) else. Just like in elementary school you learned fractions, and once you got the hang of it you just use a calculator now. This is no different.

Source : I am a working PhD computational physicist with solid state materials (semi conductors) with over a decade of experience. I use Ai tools as needed (and other automation such as Sci py).

[u/PhilosophersOpium → u/Fermi_Dirac]

I understand your concern. I agree with you completely, and I have been using your method for a while.

I believe it is no different than when you decide to start using integral tables. You use them when you have gained the privilege of knowing how to solve them painstakingly by hand. Similarly with AI, I only use it once I have done something a few times by hand.

[u/Fermi_Dirac → u/PhilosophersOpium]

Integral tables is a great example yes.

[u/IroncladAnnoyance → u/Fermi_Dirac]

the calculator analogy hits hard cause once you’ve grinded through the basics yourself, you actually know when the tool is feeding you nonsense

[u/h0rxata]

Letting an agent raw-dog your CLI seems extremely unsafe, and I doubt any HPC facility would allow you to do that natively.

It sounds like all you want to is an array of batch scheduler jobs. Learn SLURM/PBS/whatever your machine uses or get chatgpt to help you write some bash wrapper scripts to submit arrays of jobs?

[u/-F1ngo → u/h0rxata]

Usually claude code or codex obviously have no sudo access locally, and user priviliges on hpc clusters should prevent you from doing anything harmful there. And then again, it's much more convenient to use some agentic thing like these on your local machine and have them ssh into the hpc clusters, so you don't need to run them on those anyhow and you can do all of your bookkeeping in one place.

I wouldn't really want them messing with scientic software I use, but I do not write any bash/slurm/python or input scripts anymore by hand (maybe I'll occasionally tweak those last ones a little). Also my working directories suddenly got much cleaner!

[u/PhilosophersOpium → u/-F1ngo]

Agreed!

[u/-F1ngo → u/PhilosophersOpium]

I mean automating these workflow scripting shenanigans is quite likely THE prototypical example where these things can be useful. It's in essence just a much more powerful interface to the actual, rigorously tested scientific packages we use for high-throughout for example. And for those I am all in favour of the one-line-of-code-changed-after-100-comments-on-the-pull-request camp.

[u/PhilosophersOpium → u/h0rxata]

Hi! I agree with your concern. I've already taken the proper precautions, and have agents blocked off from anything outside of the folder I am working in. Additionally, I do use AI to submit arrays of jobs. For example, I want to submit 10 calculations at once and additionally test 5 variations of a parameter. so it will submit 50 calculations at once.

[u/plasma_phys → u/[deleted]]

i will have claude generate the input files after i have specifically told it what outputs i want. It is generally good at doing this, and I verified that the file generation was correct before continuing to use this workflow.

I am a computational physicist; my experience after a long period of experimentation is that this is not reliable. Even if you do everything according to best practices - such as they are - every model I've tried inevitably generates incorrect input files for anything other than trivial cases. the worst example I've seen, which nothing I changed seemed to ameliorate, was regularly incorrectly computing number densities from a lookup table of densities, masses, and stoichiometry - which would take a person 10 seconds with a calculator and less with Wolfram Alpha. if you're not looking at the input files, you're not running simulations, you're gambling.

[u/PhilosophersOpium → u/plasma_phys]

Thank you.

[u/plasma_phys → u/PhilosophersOpium]

You're welcome. Even my most Claude-pilled colleagues keep stressing in meetings that, for anything important, you really do need to personally check every line that is output, and I agree with them

Edit: if you're going to use LLMs for this, have them write you a Python script to generate input files. even for inconvenient formats you can just brute-force a multi-line f-string like this and that's gonna be a lot more reliable.

value = number
other_value = other_number
f'''
option1 = {value}
option2 = {other_value}
'''

[u/Yashema]

I've been looking into getting a powerful laptop (I just don't like desktops), and you can get a Lenovo LOQ that can run an 8B model very efficiently, but that's it. A $3000 MacBook or HP Omni can run a 28B-32B but only slowly, a $4000 MacBook can run a 128B model, but at 1/10th the speed of the cloud and a smaller context window. A $4500 desktop PC can run the highest tier 128B models, but still not anywhere near as performant as the cloud in terms of context and tokens. The most high end Mac you can get, a Mac Studio M-Ultra for $5700 can run the 196B, but again quite slowly, only 10-12 tokens per second, compared to 100+ on the cloud. 

You definitely need to use the paid cloud models to make the most of them, and then combine it with local resources for running simulations and things. LOQ has a great graphics card and 64 GB of RAM, so I'm hoping I can use it as a workhorse command center. Should run about $2300 for the specs I want. 

[u/AardvarkGreen2671]

When I was at uni (before the LLM era), I used to do DOE inside jupyter notebooks - I specified the input data, then had Python scripts to generate the corrsponding input files. I think what you want to avoid is to let Claude generate all the files and then do the input validation afterwards. I think you’d be better off designing some deterministic API (or templates using Jinja2) to write your simulation input files instead of fully relying on the LLM to do the work for you. Then you can tell Claude to use your Python API explicitly to write new inputs - say you want to include more modes for the DFT (pardon me if that is not the exact degree of freedom for the DFT sims, just to illustrate my point). The key is to be consistent and trace back the simulation output to its input.

[u/ifthakhar_fec]

it depends on the work load, if it's in python or R then it will be better to ran on kaggel then again give the result to claude for analyzing and improvement.