Rodent vision inspires better autonomous mining vehicles

Michael Milford from the Australian Centre for Robotic Vision at Queensland University of Technology explains how to navigate dusty and dark tunnels using biologically-inspired camera systems

There’s a joke that academics create solutions and then go out searching for problems, when they should be doing it the other way around. I think either extreme is perhaps not ideal. 

However, as soon as we’d had our first discussions with our industry partner Caterpillar, it was clear that we’d found the dream application of much of the research our group had pioneered over the past decade. 

The problem was to create a reliable, primarily camera-based positioning system for underground mining vehicles, to enable position-tracking and also to improve their autonomous capabilities. The challenges of the underground mining environment – long, featureless tunnels with visibility problems caused by factors such as bad lighting and dust – were exactly the types of problems we’d already been tackling with our bio-inspired robotic navigation research.

One of the primary difficulties faced in creating navigation systems for use underground is the perception problem – the environment is visually and geometrically bland. This means different parts of the environment appear highly similar to both camera- and laser-based technology solutions. One of our contributions to the project has been integrating information from visual sensors over time to get more reliable estimates of a vehicle’s position.

One of the other main challenges is how much visual conditions can change. As soon as you go underground you notice things like dust in the air, and the effects of oncoming headlights from other vehicles. We’ve attacked this challenge by using relatively low-resolution images that give more robustness to these effects, and by developing some level of intelligence in the system, whereby it can evaluate the quality of the images it’s receiving from its cameras and decide how much to trust them.

Our research group has conducted a wide range of biologically-inspired navigation and perception research over the years, starting with rodents, but also basing systems on other animals such as ants and primates.

We have shown that for certain tasks, such as navigating along path-like environments (like mining tunnels), the low-resolution visual images of the world that animals such as rodents and some ant species see are sufficient, and in some cases even superior to the use of high-resolution cameras. 

We’re using a similar approach in the mines, but coupling some of the biologically-inspired aspects of the system with more conventional robotic or computer vision techniques such as filtering over multiple frames from the camera. We’re also investigating deep-learning-based approaches, which at a very loose level are biologically inspired by networks in the brain.

The mining industry has been one of the most innovative in terms of early adoption of autonomous technology. There have been autonomous underground vehicles in some mines for many years, so some of the risks have been ironed out or proven to be acceptable pretty thoroughly in the real world.

Obviously the most important risk to consider is endangering people. This is primarily a factor when vehicles are operating in mixed human-vehicle environments, where an error in navigation could cause problems. But it can also be a challenge if the vehicle technology breaks down, forcing a human to potentially go into harm’s way to ‘rescue’ it. Either is unacceptable, and mining companies will only deploy the technology when they’re convinced it will meet required standards of reliability.

One of my primary interests is in testing and in translating into applications fundamental research in navigation, perception and intelligence. While this particular project collaborating with the Queensland government and research body Mining3 is in underground mining, almost anything that needs to move autonomously, whether on a construction site, farm or road, or in the air or sea, must have navigation and perception technology that is robust enough to cope with changes in the environment and challenging conditions. This applies equally in civilian industry and defence contexts.

We are interested in moving into other areas of mining, including above ground, and into other domains such as self-driving cars.