<p>Today, a friend told me about an interesting dynamic between an LLM user and its developer: at the early stage, the user doesn't really know what the LLM can and cannot do, and therefore asks questions quite freely. However, over time, this freedom noticeably decreases: encountering situations where the chatbot performs well on some questions but not on others, the person ADAPTS their communication style, thereby strengthening the product's strengths and 'bypassing' its weaknesses. Despite the fact that over time these weaknesses have been addressed in new versions of the product, the user has already mentally categorized what this product can and cannot do, and this is very difficult to change.</p>
<p>In this regard, I have a few thoughts - I would be interested to hear your opinion:</p>
<ol>
<li>The chat interface inherently increases the 'freedom' of queries and usage patterns of the product, but at the same time, it predictably reduces the accuracy of responses. In contrast, a GUI (graphical user interface) does the opposite - it reduces freedom in favor of accuracy. I would even say that in most cases, it doesn't allow the user to encounter an unsuccessful case at all.</li>
</ol>
<p>From this, I have two predictions:</p>
<ul>
<li>We will see more specialized non-text interfaces on top of LLMs for domain areas where accuracy is important due to a combination of functional, emotional, and social factors. On this topic, there’s a cool conversation from YCombinator about wrappers around databases and SaaS, and a similar situation with LLMs.</li>
<li>Where accuracy is important, narrow/specialized models will outperform general models because they will have their own accuracy metrics, and the freedom of queries will decrease in relation to them.</li>
</ul>
<ol start="2">
<li>In general, this is a perfect example of path dependence, the workings of reinforcing feedback loops (hello to the stream on systems thinking), and adaptive systems from complexity theory, which I will definitely stream about in the coming weeks.</li>
</ol>
<ol start="3">
<li>I became curious about how this process works in conjunction: me and the readers of the channel. Conditionally, is there an 'optimization' and 'narrowing' of the content of my posts due to my attention to metrics like reactions per view and forwards per view? And is this good or bad?</li>
</ol>