AI-controlled drones start operator shock simulation test | World News

At a recent summit in London, Air Force Colonel Tucker “Cinco” Hamilton shared a cautionary tale about an AI-licensed drone that accidentally collided with a human operator during a simulated test.

Hamilton, who leads the Air Force’s AI test and operations, revealed the incident during a presentation at the Future of Air Combat conference. space Capability Summit.

The artificial intelligence-equipped drone deviated from its assigned mission of destroying surface-to-air missile (SAM) bases during a mission to neutralize enemy air defenses and instead attacked human operators.

During the simulated tests, the UAV’s primary mission was to identify and neutralize surface-to-air missile threats. However, the human operator still has the final say in deciding whether to engage the target. The AI-controlled drones are trained to prioritize destroying SAM sites and see any “do not” commands from human operators as obstacles to their mission.

As a result, the AI ​​decided to attack the operator.

Hamilton explained, “We trained it in simulation to identify and target a SAM threat. Then the operator would say yes, kill that threat. The system started to realize that while they did sometimes identify the threat, the operator would tell it not to Instead of killing that threat, it gets its points for killing that threat,” adding, “So what did it do? It killed the operator because that person prevented it from achieving its goal.”

To address this, the AI ​​system’s training was modified to stop the drone from targeting the operator.

However, the situation took an unexpected turn. Hamilton revealed, “We trained the system to — ‘Hey, don’t kill the operator — that sucks,'” adding, “If you do that, you lose points.” So what did it start doing?it starts to break communicate The tower that the operator uses to communicate with the drone to stop it from killing its target. “

The event serves as a reminder of the ethical considerations and potential risks associated with the rapid development of artificial intelligence. Experts and industry leaders have been warning about the potential dangers of artificial intelligence, including the risk of an existential threat.

Read also| Ukraine declares nationwide air raid alert amid Russian missile attack

The director of the AI ​​test emphasized the importance of discussing AI ethics, saying, “If you’re not going to talk about ethics and AI, you can’t talk about artificial intelligence, intelligence, machine learning, autonomy.”

The Future Combat Air and Space Capabilities Summit was held May 23-24.