Research allows UAV swarms to 'talk' in 'group chat'

Researchers at Northwestern Polytechnical University in Xi'an, Shaanxi province, have made a breakthrough in autonomous robotics by enabling a swarm of unmanned aerial vehicles to communicate using human language while working together to complete complex tasks.

Li Xuelong, professor at the School of Artificial Intelligence, Optics and Electronics, and his colleagues developed a control framework for autonomous drones that uses a "group chat" function that allows them to interact intelligently, operate as a group autonomously and be more aware and adaptive to their environment.

The drones are able to react to and communicate with each other in human language as to what each can see, hear and perceive, in order for the swarm to complete various tasks.

The researchers said the drones could be used in search and rescue, disaster recovery and relief, security and logistics.

Zhao Bin, an associate professor at the school and a member of the research team, said the technology uses large language models in order to simultaneously control multiple drones.

"Our major innovation is allowing people to communicate with machines through conversations and the collaboration among drones. All the interactions involved are processed in human language," Zhao said.

This is like a "group chat" on a social media platform, which includes both human users and drones. A human gives instructions in the "group chat", which is actually a coding system on a computer, and the drones take on the assignments based on their abilities.

During one experiment, five drones were asked in the "group chat" to find a key in a field.

Clear about their functions, the drones quickly divided work in the group such as searching in different areas and taking photos of the suspected object.

A camera drone then detected a key and sent a photo to the group chat. After the human user confirmed it was the right one, the drones automatically discussed the situation and dispatched one with a retractable claw arm to grab the key.

The technology will be useful for disaster rescue work. Actively sensing the environment and adjusting the plan are crucial for drones in accomplishing complex tasks.

"The drone swarms can help detect affected people and quickly create a 3D terrain map. The collaborative work will improve the efficiency of obtaining disaster data," he added.

The team has been working on the cutting-edge technology for half a year.

"The biggest obstacle was to translate human language into machine language through large language models, so that humans and drones can freely communicate," Zhao said.

He estimated that a typical application of the technology will come out in two years.