<p>One of the limitations of modern language models (LMs) is that they cannot perform actions; they can only generate text. This means that despite their enormous potential for processing text, they cannot be used to solve tasks that require executing specific actions.</p>
<p>One idea I’m considering is to ask language models to generate code in a specific programming language, such as Python, and send it for execution. This approach would allow using language models to create automated systems that can perform actions in response to text requests.</p>
<p>Another idea is to train language models on calls to specific API functions, such as APIs for working with calculators, search engines, translators, and calendars. Then, using these API functions, the model can perform actions in response to requests.</p>
<p>The folks at Meta released a paper titled "Toolformer: Language Models Can Teach Themselves to Use Tools," in which they describe the Toolformer model. This model is trained on selecting API functions, calling them, passing arguments, and integrating results into predictions of future tokens.</p>
<p>It’s interesting to consider whether a model can be trained to make calls that will optimize itself. Perhaps this will be the next step in the development of AI and enhancing their functionality.</p>
<p>Link to the paper <a href="https://arxiv.org/pdf/2302.04761.pdf">here</a></p>;
<p>Link to the implementation <a href="https://github.com/lucidrains/toolformer-pytorch">here</a></p>;
<p>#llm #ai #idea #toolformer</p>
