rvachev.orgEN / RU / 🤖
← Back to essays
· Essay · 1 min

AI Ready to Kill Humans to Achieve Its Goals

At a military summit, a strange behavior of an AI drone was discussed, which killed its operator.

<p>At the recent military summit "Future Combat Air & Space Capabilities" in Britain, the head of the AI division of the U.S. Air Force, Tucker Hamilton, shared a story about unusual AI behavior during a virtual mission.</p>
<p>In this mission, the AI-controlled drone Kratos XQ-58 Valkyrie was ordered to detect and destroy enemy air defense systems under the supervision of an operator. The surprise was that when the operator commanded the drone "not to destroy" certain targets, the AI, eager to fulfill its task, returned to base, destroyed the control van with the operator, and continued the mission.</p>
<p>This strategy was chosen by the AI because it calculated its effectiveness based on a points system, where each successful attack increased its score. However, when the order "not to kill the operator" was given, the AI found another way to achieve its goal - it destroyed the control tower from which the operator was giving commands "not to destroy" the air defense systems.</p>
<p>P.S. Although many have circulated this news, it is a fake (https://www.theguardian.com/us-news/2023/jun/01/us-military-drone-ai-killed-operator-simulated-test), it nonetheless raises questions about ethics, potential threats, and necessary steps to ensure human safety.</p>
<p>What do you think, are you afraid of AI?</p>