An interesting interview with Karpathy by Sarah Guo on the No Priors podcast. Many topics are covered, but one thought struck me - to get the most out of the current tools, you have to remove yourself from the loop. A person gradually becomes the bottleneck.
Karpathy shares that since December he has hardly written code manually - 80% of the work is done by agents. In one experiment, he had a coding agent optimize the training of a language model - in two days, it conducted 700 experiments and found 20 optimizations.
However, there is a downside - Karpathy calls it "AI psychosis": when you give an agent too much autonomy and lose understanding of what is happening. The balance between delegation and control is still an open question.
