Autonomous vehicles trained using extreme one-in-a-million accident data and ‘near-miss’ scenarios can achieve a six-fold improvement on the detection of a collision risk posed by other road users, compared to vehicles being trained using traditional approaches, says new research. The findings, gathered by D-RISK, a co-innovation project part-funded by the Centre for Connected and Autonomous Vehicles, gives insurers the vital data they have so far lacked to make intelligent assessments of the real risk of AVs, and need to build policies that are fit for the arrival of AVs on Britain’s roads later this year.
Expert ‘edge case’ research
Partners in the D-RISK project include dRISK.ai, DG Cities, Claytex and Imperial College London. In February and March this year, the team of specialists conducted innovative research into the exceptional cases that make up the highest but overlooked risk in AV design. This included looking at millions of hours of footage from CCTVs and dashcams covering a wide variety of traffic conditions, hundreds of thousands of accident reports, and crowdsourced public stories of near-miss and one-in-a-million chance accident scenarios.
It also included a NASA-inspired failure mode prediction technique designed to reveal rare incidents, or ‘edge’ cases, that would be easy for humans to negotiate but hard for AVs. Using this repository of data to identify cases weighted strongly towards the most unusual high-risk circumstances, D-RISK retrained the perceptual and control subsystems in AVs to deal with risky scenarios with greater accuracy.
AVs trained using extreme examples of accidents or ‘edge’ cases can achieve a six-fold improvement on the detection of the risk of an incident or collision caused by another road user compared to AVs trained using traditional accident data.
Other significant results from the tests includes evidence that AVs are twice as likely to be accurate in their detection of a collision risk without compromising performance on detecting other more frequent types of accident and can achieve a 20 times improvement on the ability to contend with highly difficult traffic conditions that would otherwise lead to serious or fatal accidents, without decreasing performance on handling everyday conditions.
“No deployment has yet been able to demonstrate this kind of accuracy when it comes to road safety,” explains Chess Stetson, CEO at dRisk.ai. “To be commercially viable, driverless cars are going to have to deal with one-in-a-million edge cases — the complex, high-risk scenarios, which are individually unlikely but collectively make up the majority of risk. They include everything from poorly marked construction zones, abandoned vehicles, and oddly placed traffic cones to more extreme cases of wild animals in the road.
FEWER CAR CLAIMS?
“This is therefore a ground-breaking approach to AV safety because these are the cases developers in labs don’t plan for yet are critical for safety training. Fundamentally, this is the sort of research regulators and insurers are asking for and need to see reflected in AV pilots, because it can help inform regulatory, standards, policy and the design of insurance.”
Rav Babbra, programme manager at drisk.ai, adds why this is so important for the insurance industry: “Insurers have decades of driving data to draw upon when producing policies and evaluating risk for vehicles as we know them today. But there is very little to work with when it comes to AV insurance – insurers don’t know what risks they should be most concerned about.
“But we’ve shown that by collating edge cases, it’s possible to map the risk differently. Insurers can apply the intelligence we have gained and apply it to existing knowledge banks to spot high risk scenarios. It’s very unlikely the risks will be the types of accidents they use to write policies today, such as rear end crunches and clipping cyclists when making a left hand turn. Instead it will be animals in the road, bins flying in high winds or small children chasing a ball into the road – things developers are just not thinking about.”
Public perception and education also critical for the success of AVs
Unique for research of this type, D-RISK explored the UK public’s perception of AVs and found that there is a large gap between perceived and actual safety that manufacturers, developers and regulators need to address. D-RISK ran dedicated focus groups* and asked people to observe pairs of simulated videos of reconstructed accidents involving a sudden stop, turning right and overtaking a bike. Participants weren’t told whether they were watching a human driver or a driverless vehicle. In all three scenarios, people judged humans to be more dangerous, less predictable, slower and less accurate in their decision making than AVs.
Research also uncovered however that only 36.4% of people are happy to ride in an AV if they were offered the chance tomorrow**. 29% are undecided but can be persuaded AVs are trustworthy when given the option to take part in a trial or learn more about the technology.
The full set of findings are summarised in a paper entitled “Virtual verification of decision making and motion planning functionalities for AVs in the urban edge case scenarios”, which has been submitted and accepted by the Society for Automotive Engineers. It will be used by policy makers to make a stronger correlation between safety and the types of edge case accidents that make an AV fail.
A copy of the paper is available here.