Remember self-driving cars?
Except for a few dramatic failures (Tesla anybody), you don’t hear a lot about them these days. But in fact, there has been huge progress in developing the underlying artificial intelligence (AI) technology for them.
One person who has been involved in AI research and development from almost the beginning, is Bibhrajit Halder. Starting in the early 2000s, Halder worked with Defense Advanced Research Projects Agency (DARPA), then Caterpillar on its first autonomous mining trucks and eventually moved to Silicon Valley to work with Apple and others on self-driving vehicle projects.
HalderSafeAIToday, Halder is the founder and CEO of SafeAI, a company that retrofits heavy equipment such as dump trucks, pickup trucks and skid steers with autonomous technology. Additionally, the company has partnered with Obayashi (one of Japan’s top five construction companies) on an autonomous construction site and is working with Goodyear to integrate tire intelligence into autonomous vehicles.
We recently had the opportunity to talk with Halder and ask him just what happened to self-driving cars and what might that mean for the mining and construction industries.
As for self-driving cars, “That’s just hard,” says Halder. “The problem is massively challenging. Work is ongoing and there has been a huge amount of progress over the last ten years. But more has to be done. Companies like Waymo are getting close. And when they do get it, the floodgates will open. We just don’t know when that will be.”
On the other hand, heavy equipment, in particular mining, is way ahead in terms of adoption. “The mining industry has already moved more than five billion tons of material with autonomous trucks,” Halder says. Even more significant, he says, is the fact that most of those mining trucks are running on technology that is already outdated. And the new technology, developed to solve problems in automotive applications is just now beginning to make a difference in mining and heavy equipment autonomy.
For simplicity’s sake, Halder refers to the old and new technology as AI 1.0 and AI 2.0.
The new AI
The AI 1.0 that entered the mining truck market early in the last decade relied on GPS, LiDAR and CPUs on the machines. These did the job very well but were limited in how much they could do and how much information they could capture and process.
The big difference now with AI 2.0 is that engineers have Graphics Processing Units (GPUs) in addition to CPUs, says Halder. “GPUs offer a hundred times more computing power than before. That means we can do a lot more with our algorithms. GPU technology is more mature and able to handle many more use cases.”
GPUs were initially developed for the video game industry because in video games a lot of the pixels change very quickly, and CPUs are not adept at this kind of information processing. “GPUs are better at parallel processing and can do it at a significantly better scale, and that helps AI, because AI requires parallel processing,” Halder says.
Neural networks
Halder gives an example of a truck coming up on a berm as an example of how this works. With the older 1.0 systems if you wanted your machine’s perception system to identify a berm using LiDAR it would be difficult. To a LiDAR system a berm might as well be a building. All it sees is an object of a certain size and shape.
“Now we can take hundreds and hundreds of pictures of berms and feed it into this deep neural network, and it will recognize a berm and learn what it is,” says Halder. “And the more data you feed it, the better it works. This quality gets better and better over time, that’s a massive improvement.”
As the machine learns to distinguish between a berm, a building or a stockpile it can make better decisions on its own, Halder says. “Your engineers don’t have to change the algorithm every time the truck encounters a new situation. That’s the power of AI. The AI network will recognize anything that you feed it.”
The automotive industry is also using GPU’s but the sheer complexity of highways, vehicle sizes, road signs and pedestrian issues are still more than AI 2.0 systems can handle and guarantee 100 percent reliability and safety. But construction and mining sites, with their limited traffic and controlled environments are easily managed by GPUs and AI neural networks.
AI 2.0 is just beginning to make an impact in mining and a few big construction sites, but that’s going to change says Halder. “With every project we are gaining massive amounts of knowledge. And as we gain that knowledge, we will better know how to scale that down. It is a continuous learning process.”
Training and support
Another plus for the mining and construction industry is the fact that the companies involved in implementing AI 1.0 developed good processes for training workers and deploying the technology.
Construction supervisors already know how to plan jobs and orchestrate the work, says Halder. Only now, with autonomous machines and AI, instead of verbally telling operators what to do, they send the same instructions to the autonomous machines as a digital file.
“The transition from driving machines to managing machines remotely happens a lot quicker than you think,” says Halder. “Based on my experience, the operators and people on the sites picked up on it so quickly you would be amazed,” says Halder. “They actually love it. Their job has become much safer with them sitting in the office looking at a screen or out a window. And that’s the vision: nobody in the unsafe areas.”
Training on the mine sites is done with a staged approach, says Halder. “The tech provider will continuously train and support the mine operator on a daily, hands-on basis and then slowly pull back. The tech support, which is somebody we partner with, will stay on the site, sometimes for years,” he says.
The next big thing
“Autonomy is something every OEM is pursuing,” says Halder. “Everybody is trying to get there. Everybody knows the autonomous, connected site is the future. Everybody is either doing something internally with autonomy, or they are partnering with technology providers.”
But the winners in this race will not be the AI providers who just have the best technology.
“This isn’t a pure technology play,” says Halder. “If your technology isn’t good, you don’t even get a seat at the table. You will win on the people side of it. You have to understand the customer’s workflows. You have to make their life easier. Service, support, training and education, that’s what going to win. It requires patience and that’s hard work too. Anybody who isn’t in it for the long haul won’t win.”
We’re also on the cusp of a new generation of vehicle technology that will open even more possibilities, says Halder. “Autonomy will eventually come into a lot more than just self-driving cars,” he says. “Anything that moves or has a safety implication is a candidate for AI.”
