A self-driving car expert describes the biggest problem with semi-autonomous driving systems like Tesla's Autopilot and Cadillac's Super Cruise

Semi-autonomous driving systems like Tesla's Autopilot and Cadillac's Super Cruise are designed to reduce the stress of driving by automating tasks like steering, accelerating, and braking under certain circumstances, but the driver needs to be ready to take over at any moment.
Those systems rely on unrealistic expectations about how well humans are able to pay attention to the road and quickly react if the semi-autonomous system needs them to take control, Mary Cummings, a professor at Duke who studies the interaction between humans and autonomous driving systems, told Business Insider.
"We can't sustain attention, especially in boring environments like highway driving," she said. "Expecting the human to be able to just step in when we know they haven't been paying attention is a huge problem."

Semi-autonomous driving systems like Tesla's Autopilot and Cadillac's Super Cruise are designed to reduce the stress of driving by automating tasks like steering, accelerating, and braking under certain circumstances, but the driver needs to be ready to take over at any moment. That's a problem, Mary Cummings, a professor at Duke who studies the interaction between humans and autonomous driving systems, told Business Insider.

"Any reliance on the human to intervene in any kind of safety-critical capacity is a red herring. They're never going to be able to do it reliably enough to make these systems, I think, safe for the road," she said.

Systems like Autopilot, Super Cruise, Nissan's ProPilot Assist, and Volvo's Pilot Assist rely on unrealistic expectations about how well humans are able to pay attention to the road and quickly react if the semi-autonomous system needs them to take control, Cummings said.

"We can't sustain attention, especially in boring environments like highway driving," she said. "Expecting the human to be able to just step in when we know they haven't been paying attention is a huge problem."

A Tesla representative directed Business Insider toward a page on its website that describes its approach to vehicle safety and includes statistics about accident rates for Tesla vehicles with and without Autopilot engaged.

"We believe the unique combination of passive safety, active safety, and automated driver assistance is crucial for keeping not just Tesla drivers and passengers safe, but all drivers on the road," the page says.

During the fourth quarter of 2018, Tesla said there was one accident per 2.91 million miles driven by Tesla vehicles with Autopilot engaged, compared to one accident per 1.58 million miles driven by Tesla vehicles without Autopilot engaged and one car crash every 436,000 miles driven for all vehicles in the US. But Tesla's data does not account for factors, like the environments in which Autopilot is most often used, that could also account for some of the differences in accident rates.

Nissan and Volvo did not respond to a request for comment.

There are semi-autonomous features, like those that alert the driver if the car is drifting out of its lane or some advanced cruise control systems — which can brake or accelerate based on surrounding traffic — that can be useful, particularly at low speeds, Cummings said. The danger comes when vehicles take control of driving tasks at higher speeds.

"I'm a big fan of slow-speed autonomy," she said. "It's when we see cars at increasing speeds under unstructured environments with a lot more uncertainty that the car control systems really fall apart."

Join the conversation about this story »

NOW WATCH: Here's what it's like to drive trains on London's Tube — one of the most complicated subway systems in the world