Stumbled upon an interesting study: the folks at METR proposed a new metric for AI progress, similar to the famous Moore's Law (which states that the number of transistors on a chip doubles approximately every two years).
It turns out that over the past 6 years, the length of tasks that AI can perform independently (with a 50% success rate) doubles every 7 months. Despite the clarification about '50% success', it sounds impressive.
If the trend continues, within the next 5 years, AI will be able to independently handle a significant portion of tasks that currently take humans days or weeks.
The authors themselves note that AI is already almost perfect at tasks up to 4 minutes of human time, but tasks over 4 hours are still challenging (less than 10% success).
At this rate of progress, by the end of the decade, we can expect systems that will autonomously execute projects lasting a month or more. The implications of this are hard to overstate: from fully automated work to serious risks associated with such autonomy.
Details and methodology here: https://metr.org/research/measuring-ai-ability
Paper: https://arxiv.org/abs/2503.14499
Github: https://github.com/METR/eval-analysis-public
