AutoNorms

Weapon systems that integrate an increasing number of automated or autonomous features raise the question of whether humans will remain in direct, meaningful control of the use of force. Concerns relate, in particular, to weapon systems with autonomy in their critical functions – that is selecting, and engaging targets through sensors and algorithms without human input. Such autonomous features can take many different forms: we find them inter alia in loitering munitions, in aerial combat vehicles, in stationary sentries, in counter-drone systems, in air defence systems, in surface vehicles, and in ground vehicles. While diverse, these systems are captured by the catch-all category of autonomous weapons systems (AWS), because they weaponise Artificial Intelligence (AI). 

Many states consider applying force without any human control as unacceptable. But there is less consensus about various complex forms of human-machine interaction along a spectrum of autonomy and the precise point(s) at which human control stops being meaningful. To illustrate, can human control that executes decisions based on indications it has received from a computer be considered meaningful, given that “blackboxed”, algorithmic processing is not accessible to human reasoning? Faced with these questions in the transnational debate at the UN Convention on Certain Conventional Weapons (CCW), states reach different conclusions: some states, supported by civil society organizations, advocate introducing new legal norms to prohibit so-called fully autonomous weapons, while other states leave the field of open in order to increase their room of manoeuvre.  

As discussions drag on with little substantial progress, the operational trend towards including automated and autonomous features in weapon systems continues. A majority of the top 10 arms exporters such as the USA, China, and Russia, are developing or planning to develop some form of AWS. Such dynamics point to a potentially influential trajectory AWS may take towards changing what is appropriate when it comes to the use of autonomously applied force. Technologies have always shaped and altered warfare and therefore how force is used and perceived. Yet, the role that technology plays should not be conceived in deterministic terms. Rather, technology is ambivalent, making how it is used in international relations, and especially in warfare, a political question.