My first two months with AI agents: some confident assertions

About two months ago, I started using modern AI tools in my research in a serious way. It’s been a tremendously exciting and also unsettling experience. To give you an idea how fast this happened, my in-laws arrived from Korea in early September to visit us for three weeks, right about the same time all this started for me. By the time they left, my way of working had completely changed. Not only that, but my sense of what is easy, what is hard, and what is possible in my research was dramatically altered compared to the beginning of September.

You can’t go through a change like this without it affecting you on a personal level. Some of you who know me also know that I got into trail running over the last few years. For about a month, every single morning felt like the morning of an important race, with all the race-day nerves. That was fun, but it was also exhausting and unsustainable. I might have two races in a year where I feel that way, and that’s already a lot! Fortunately things have calmed down a bit.

I’ve found myself having a lot of thoughts that might be useful to share with our community. So with some trepidation — I can’t help but feel a bit presumptuous — I’m going to try writing these down here in this blog. When I say “our community,” I am thinking about theoretical physicists. These days, I think of myself as a theoretical quantum physicist, a condensed matter theorist, and a mathematical physicist, not necessarily in that order. When I’m writing here, I have in mind a general theoretical physics audience. Of course, these issues aren’t at all unique to our field, so you might find this interesting or useful even if you’re working in a very different area. Or perhaps no one at all will read this, and I’m writing here for myself, to clarify my own thinking. That would be fine with me!

So what did it mean for me to start using AI in a serious way? Like basically everyone, I was already asking chatbots (mostly Gemini) questions related in some way to research. In September, what changed is that I started working on projects with AI agents. If you have no clue what that means, that’s fine, I didn’t either — I was definitely behind the curve here compared to many others — and I’ll say something about practicalities at the end of this post. Roughly, you provide a folder with some materials, and then work iteratively with agents on some task or set of tasks over an extended time period. The agents can keep a record of work that happened, so ongoing work builds on what’s already been done. The difference between an agent and a chatbot is that a chatbot is something you can talk with, while an agent can do things for you, at varying levels of autonomy that you control.

The experience has been like working with a talented collaborator, who knows a lot more math and physics than I do, but who does need guidance. Who does not get tired or overwhelmed or too busy, and who works orders of magnitude faster than a human can. Who does not write as well as a good human writer, but who writes surprisingly well, especially when I give smaller iterative writing tasks with more of my own input. Who largely does not make technical mistakes and who I haven’t seen make up a fake reference — these problems seem to be a thing of the past — and who is perfectly capable of handling much of the technical side of a research project, freeing me to be creative, think about the big picture, while also digesting, verifying, simplifying and organizing the more technical aspects. Not always, but sometimes, this collaborator and I can get done in an afternoon what would have previously taken weeks. It’s pretty fun!

I am going to make some confident assertions in the rest of this post. If you’ve heard me give a talk or if you’ve read some of my papers, you know I’m careful to qualify statements and give caveats. (Sometimes maybe too careful.) I don’t think caveats are needed or helpful below. At the same time, I’m sure I’m wrong about some things.¹ If you think I’m wrong about something that matters, or if you just have a different perspective, I’d like to hear from you. And also you should write your own blog post or article … that’s not sarcasm, I mean it sincerely. Our field needs and will need a lot more public and private discussion of these issues.

AI is going to transform theoretical physics in a profound way. Doing theory research two years from now is going to look very different from the way it did a year ago. What it means to be a theorist is going to be different, in a way that could be disruptive to your identity, even if these changes are positive for you. Personally, I find these changes exciting. There is also certainly a lot to be concerned about. That said, I don’t feel worried — I’m generally optimistic by nature — but that doesn’t mean I’m not thinking about possible negatives.

I do think there are still some of us who think, “oh, my subfield, or the problems I care about, aren’t going to be affected that much.” I’m not sure what I would have said a few months ago — I didn’t even pay much attention to AI at that time — but it easily could have been something along these lines. If that’s where you are now, then with maximum kindness and warmth I say: don’t fool yourself.

Our field is on the cusp of a major upheaval. This is a different point from the first one, but it follows from it. All kinds of disruption will come along with profound changes in our field, and I don’t know what form this will take. Mathematicians are ahead of us in terms of being affected by AI, incorporating AI tools into research, and thinking about what AI means for the present and future of their field. Many mathematicians describe their field as going through a crisis. Our field is similarly going to go through an upheaval; maybe I am just unaware, but I don’t feel like this has really started yet, or perhaps it is just starting. A lot is being written in the math community about these issues, and we should pay attention to it. If you’re not sure what to read, Terry Tao’s blog is a good place to start. I found this beautiful piece by Ben Antieau particularly inspiring.

Of course, our field is different. Some of the reactions from mathematicians don’t resonate with me personally, but many of them do. We should be prepared for extremely positive, extremely negative, and otherwise extreme reactions. We can look to math to anticipate the range.

While our field is different, again, don’t fool yourself. It would be a big mistake to think that the differences in fields mean theoretical physics won’t also be dramatically changed by AI. I say this as someone whose work has ranged across various research styles within theoretical physics, more recently venturing into mathematical physics. I don’t think we are going to find a phase transition on the mathematical-physics-to-theoretical-physics axis where AI stops being powerful.

Current AI capabilities are already enough to transform our field. Even if AI capabilities were frozen in their current state, we are in for massive changes. Of course, it would be foolish to assume that next year’s models will be the same as this year’s. Above, I talked about a rough division of labor between me and agents where I am thinking about the big picture, being creative, and figuring out what questions to ask, while agents are handling the technical side of things. That’s an oversimplification — I’ve sometimes found it useful to engage with agents about big-picture issues, and I do think agents can be creative — but there is some truth to it. Will that state of affairs continue? If I had to guess I’d say no.

If you’re reading this, you need to engage with AI in your own work. I don’t like telling people what they should or should not do, but I’ll make an exception for this. Why? AI tools are here and they will change our field whether we like it or not. Engaging with them gives you a chance to be a part of shaping the outcome of this change. On a more individual level, it is not going to be possible to maintain a competitive research program without using AI. This goes especially for students reading this post, and I’ll have more to say about this in future posts.

I promised some practical advice for people who want to try this but don’t know where to start. Install the ChatGPT or Claude desktop app. The key step is moving beyond chat to an agent that can work with files on your computer: in ChatGPT, that means Codex; in Claude, it means Cowork, or Claude Code, which is roughly Claude’s analogue of Codex (though not one-to-one). Create a project, attach a folder, and put in materials for something real you want to work on — for example, the LaTeX source of a paper in progress. If collaborators are involved, I recommend getting their permission before you try this. Then describe what you want to do and ask how the agent could work on it with you.

Disclosure on use of AI tools: I wrote an entire first draft of this post without any AI tools, then I revised with the help of some suggestions from Claude, both to improve the language and to help me avoid making a fool of myself in public. More substantively, Claude helped me come up with the title for the post, wrote the short description that appears in the blog listing, and rewrote the paragraph on practical advice to make it clearer. Otherwise the text is my own.