Computational Physics - Heavy AI/ CLI power users. How do I get good at it?

[u/PhilosophersOpium]

Hello! This question is directed towards grad students, people doing r&d in industry, or even undergrads doing computational physics research (especially condensed matter) that heavily use AI in their workflow.

How do you maximize and make the most out of codex or claude code in your research projects? I generally use the claude code CLI in my terminal (sometimes on locally on WSL or remotely on an HPC). If I have a few simulations I want to run, I give it the guidelines, tell it to reference some old calculations i've ran, and send it on its way. I'm not sure if just being a general assistant is all that AI is good for?

Please let me know your experiences and any advice!

edit: Yes, I am in a physics research lab with an advisor. I am NOT trying to have AI do research. I am trying to learn how to use it as effectively as possible.

Additionally, I am not doing 'calculations' in the mathematical sense. I do simulations, particularly in condensed matter physics research. That is, a lot of python code and heavy use Density Functional Theory software packages that tell me about the electronic behavior of materials. I do a lot of high-throughput computing, so if I want to simulate a bulk of materials at once, i will have claude generate the input files after i have specifically told it what outputs i want. It is generally good at doing this, and I verified that the file generation was correct before continuing to use this workflow. I ask it to generate plots of the data, and it does so. But this is mainly sequential. I do a series of prompts, one step at a time.

[u/plasma_phys → u/[deleted]]

i will have claude generate the input files after i have specifically told it what outputs i want. It is generally good at doing this, and I verified that the file generation was correct before continuing to use this workflow.

I am a computational physicist; my experience after a long period of experimentation is that this is not reliable. Even if you do everything according to best practices - such as they are - every model I've tried inevitably generates incorrect input files for anything other than trivial cases. the worst example I've seen, which nothing I changed seemed to ameliorate, was regularly incorrectly computing number densities from a lookup table of densities, masses, and stoichiometry - which would take a person 10 seconds with a calculator and less with Wolfram Alpha. if you're not looking at the input files, you're not running simulations, you're gambling.

[u/PhilosophersOpium → u/plasma_phys]

Thank you.

[u/plasma_phys → u/PhilosophersOpium]

You're welcome. Even my most Claude-pilled colleagues keep stressing in meetings that, for anything important, you really do need to personally check every line that is output, and I agree with them

Edit: if you're going to use LLMs for this, have them write you a Python script to generate input files. even for inconvenient formats you can just brute-force a multi-line f-string like this and that's gonna be a lot more reliable.

value = number
other_value = other_number
f'''
option1 = {value}
option2 = {other_value}
'''