When it comes to exploring complex and unknown environments such as forests, buildings, or caves, drones are hard to beat. They are fast, agile, and small, and they can carry sensors and payloads virtually everywhere. However, to date, expert human pilots are needed to release the full potential of drones.

We have trained an AI to fly a quadrotor through previously unseen environments such as forests, buildings, ruins, and trains, achieving speeds of up to 40 km/h and without crashing into trees, walls, or other obstacles. While humans require years to train, the AI, leveraging high-performance simulators, can reach comparable navigation abilities much faster, basically overnight. All this was achieved by relying only on the quadrotor’s onboard cameras and computation.

To achieve this goal, we train the AI navigation system exclusively in simulation from a privileged expert. The drone’s neural network learned to fly by watching a sort of “simulated expert” – an algorithm that flew a computer-generated drone through a simulated environment full of complex obstacles. At all times, the algorithm had complete information about the state of the quadrotor and readings from its sensors and could rely on enough time and computational power to always find the best trajectory. Such a “simulated expert” could not be used outside of simulation, but its data were used to teach the neural network how to predict the best trajectory based only on the data from the sensors. This is a considerable advantage over existing systems, which first use sensor data to create a map of the environment and then plan trajectories within the map – two steps that require time and make it impossible to fly at high speeds. Interestingly, these simulators do not need to be an exact replica of the real world. If using the right approach, even simplistic simulators are sufficient to train effective navigation algorithms.

This project represents an important milestone for robotic research. First, it could extend the use of quadrotors in many new applications or communities, where human pilots are too expensive to obtain. Second, the applications of this research are not limited to quadrotors. The same approach could be useful for improving the performance of autonomous cars, or could even open the door to a new way of training AI systems for operations in domains where collecting data is difficult or impossible, for example on other planets.

The project was published in Science Robotics, a top-tier journal for robotics research. In addition, it appeared on popular news outlets around the world, including Forbes and IEEE Spectrum.