Autonomous Vehicles (AVs) are still futuristic – but there are plenty of people are thinking about them and what they would mean – particularly as they relate to safety.
And when they do, they invariably think about how vehicles are currently regulated as a starting point. We envisage perhaps more regulation, standards and rules – because AVs are more complex and complicated. But for every regulation, standard and rule, we take responsibility away from the manufacturer.
Why?
Because all the manufacturer needs to do is ensure that their AV meets each regulation, standard and rule for them to not be liable for subsequent accidents (this is a simplistic interpretation to be sure … but satisfactory for the sake of this article).
Is this desirable?
Is this possible?
There is a simplistic (and perhaps misguided) belief that AVs need to simply be subjected to a set of checks, tests and inspections before they can safely get on the road. This is ‘simplistic’ as a system that is more like a decision-making computer cannot really be subject to this sort of assurance. And perhaps ‘misguided’ because most people think that is how today’s non-autonomous cars are declared safe – when this is not entirely correct.
But aren’t today’s cars subject to sets of checks, tests and inspections for them to be declared ‘roadworthy’ or safe? Well yes they are, but these checks, tests and inspections are based on what auto manufacturers came up with before the regulations did. Vehicle regulations have forever played catch-up to each and every safety-related innovation. In fact, cars were being driven for almost a hundred years before the first regulatory standard was ever established.
Take a look at the timeline of the United States Federal Motor Vehicle Safety Standards (FMVSS). These standards form part of United States regulations that control the design of vehicles – primarily to make them safe. The first FMVSS was established in 1967 and dealt with seatbelts – a hugely effective safety device. Seatbelts were invented in the mid-19th Century, first patented in 1885, became optional on some cars from 1949, and became standard on others from 1958. Even something as simple as a seatbelt was never ‘predicted’ or ‘mandated in anvance’ by a regulatory agency. It took virtually a hundred years of development by manufacturers before the seatbelt became a mandatory part of a vehicle.
But perhaps you think that cars are a special case – first developed in a time when laws and regulations did not meet the level of ‘societal-technological enlightenment’ we assume we live in today. In case you think this … please read my article which looked at some extraordinary regulatory efforts from 1860 to 1900 to try and ‘get ahead’ of the manufacturers in the name of safety. The 19th century regulators and legislators I talked about probably thought they were themselves in a period of ‘societal-technological enlightenment’ as they watched the industrial revolution change the world around them. But their regulatory efforts appear somewhat farcical from today’s perspective. The rules that these regulators and legislators came up with (in some instances) required vehicles to be disassembled and hidden in bushes if they encountered livestock!
Of the many mistakes these 19th century lawmakers and regulators made, one of them was that they thought vehicles posed the biggest risk to those on the outside – not those on the inside. But perhaps the biggest mistake was the idea that these lawmakers and regulators could successfully understand and predict how these machines would function, how they would be used and the manner in which their underlying technology would be included in design. These assumptions resulted in spectacular regulatory failures.
But maybe you think that we are much more ‘illuminated’ than those lawmakers and regulators, and we can do a much better job now. And by this I mean come up with a more stringent checks, tests and inspections that meaningfully relate to how AVs will operate. Well … we (tried to) do this already for a number of other machines, and the results are not always great.
Those of you involved in governmental or military contracting for advanced physical systems are probably aware of the many things manufacturers are made to do (including testing) with the hope that if followed, the resultant system is safe and reliable. Acquisition contracts can contain an exhaustive (and exhausting) list of ‘safety’ activities. This approach routinely fails – think the F-35 Joint Strike Fighter, British Type 45 Destroyers that break down in warm water, and so on. The F-35 Joint Strike Fighter is a more than useful comparison as it involves $ 50 billion dedicated to research and development on a platform that is hugely dependent on autonomous or ‘autonomous-like’ control software. We are seeing firsthand how problematic it can be to ‘stipulate’ your way to safe operation for something like an AV.
And this doesn’t even consider manufacturers who are willing to pervert checks, tests and inspections for their own selfish aims. Think how Volkswagen designed vehicles to pass (as in ‘cheat’) emissions tests but continue to belch out nitrogen oxides at unacceptably high rates when being driven normally. Toyota tried to divert attention away from is faulty cars when they suddenly accelerated by themselves, killing many people. Just by having a set of checks, tests and inspections creates a battle that needs to be continually waged between the regulator and the regulated.
Well perhaps we can take a step back, and instead of saying ‘the system needs to have this gadget in it,’ we can simply say that ‘the system needs to be safe in this scenario.’ John Simpson from the Consumer Watchdog agrees, saying “What you want is performance standards. You don’t say: ‘This is how you make the car stop within so many feet of having the brakes applied.’ All you say is, ‘It has to be able to stop.”
We can turn the nuclear industry for a little history in this regard. Nuclear plants were built in accordance with a ‘design basis:’ the set of events or conditions that the plant needs to be able to encounter, and successfully deal with. And this approach has worked well – nuclear power is still relatively safe and reliable notwithstanding a few catastrophic counter-examples (we’ll come back to these.)
In 1975, a new approach to nuclear power plant safety was investigated. The United States Nuclear Regulatory Commission (NRC) sponsored the ‘Reactor Safety Study.’ It used a technique known as Probabilistic Risk Assessment (PRA) to quantify nuclear power plant safety – something that the ‘design basis’ approach could never do. PRA incorporates every piece of information ranging from expert opinion through to operational data. It also includes every aspect of system operation – including human error. The study did not initially receive a lot of attention. That changed four years later.
The 1979 ‘Three Mile Island Accident’ was a Loss of Coolant Accident (LOCA) which turned out to be the primary risk contributor identified by the Reactor Safety Study. The people who did not pay attention to it in 1975 suddenly became more enamored with PRA. The LOCA scenario was somewhat overlooked in the ‘design basis’ approach. And the accidents at Chernobyl and Fukushima involved substantial human error (and even negligence) which the ‘design basis’ approach struggles to deal with. We can design something based on every conceivable scenario and make it relatively safe – but we must not kid ourselves that ‘all conceivable scenarios’ equals ‘all scenarios.’
So what does AV regulation need to look like in the future? Perhaps less is more. As mentioned at the start of this article, the FMVSS limits manufacturer liability. That means if a vehicle meets the standards, the manufacturer is (generally) not liable for subsequent vehicle accident consequences. The driver becomes liable, which is why we have insurance. So if we create a set of checks, tests and inspections for AVs, we may simply be giving manufacturers a ‘get of jail free card’ when it comes to designing safe cars. If the regulators can’t keep up with and predict technological development (think the 19th century ‘red flags’ mentioned above), then there is no such thing as safe. In fact, having a set of standards for AVs may hamper the development of safe cars given there is so much we can never know about any emerging technology.
So if we have fewer standards, the manufacturers have to accept more liability. This will require current liability laws to be amended. But this may not be a bad thing. Why? Because it is all about assurance through motivation.
‘Assurance’ literally means a state of certainty. The terms ‘safety assurance,’ ‘reliability assurance’ and ‘quality assurance’ traditionally refer to a list of things one does for something to be considered ‘safe,’ ‘reliable’ or ‘high quality.’ But as hinted at above these things (or activities) can never guarantee a good outcome. For those of you been involved with reliability demonstration tests, you know there is always a risk (consumer’s risk) that something is ‘unreliable’ even if it passes that test designed to ‘demonstrate’ otherwise. Many acquisition contracts allow this risk to be as high as 20 per cent. So assurance and certainty are misnomers.
A better way to think of ‘assurance’ is as something that motivates system designers to create safe and reliable things. A product doesn’t magically become better if you test it or impose a standard. The reason we have tests is to motivate designers to create a system that will pass that test. So we need to think about how designers are motivated, and what we can do to help them. And this may mean making them do ‘less’ things.
If you make the designer do too many things, too much of their time is spent in meetings and videoconferences going through reports and tables to convince you that they have done what you have asked for (or finding ways to get around them). This can reduce reliability as the design team is focused on ‘compliance’ and not ‘performance.’ Many of you know this to be true from experience … and those that need a little more convincing would do well to read the National Academies of Science, Engineering and Medicine’s report ‘Beyond Compliance’ which looks at this very issue, and how it contributed to the Deepwater Horizon-Macondo blowout, explosion and spill in 2010.
If we blindly apply a regulatory certification, standards and compliance approach to AVs, we will see all the historical problems discussed above. One can only imagine how many ‘FMVSS-like’ standards would need to be in place for a brand new AV system whose safety revolves around algorithmic decision making processes. And we cannot escape the fact that Chernobyl, Fukushima, the Deepwater Horizon et cetera were all declared ‘safe’ by their respective regulators the day these accidents happened.
Perhaps we need to take the ‘scary’ path of actually imposing fewer standards on AVs, and just making manufacturer’s liable for everything. And that reality may already be here. Volvo Car Group President and CEO Håkan Samuelsson said that the company will accept full liability whenever one of its cars is in autonomous mode. That means the driver won’t need insurance (or at least insurance in the way we currently know it). It also means that Volvo will be motivated to continually improve the safety of the car. They are effectively providing their own insurance, and their ‘premium’ will go down the safer their vehicles get. It is worth noting that AV proponents such as Samuelsson see the biggest risk to an AV future will be regulation – not technology.
To be clear, there will always be a place for checks, tests and inspection underpinned by a set of standards. There is a lot of commentary to that end – and the fact I haven’t dedicated a lot of this article to talking about the good aspects of regulatory oversite should not be interpreted as a suggestion that there is no place for it. For example, a future standard may preclude an AV from travelling if any passenger does not have his or her seatbelt on.
But for what it is worth, I would be much happier driving an AV where the manufacturer is liable for its performance, versus another AV which has passed some checks, tests and inspections with liability then passed to me. I would be much happier again for an optimal mixture of both.
Liability and the way it is accepted by manufacturers could be the single most important thing that makes AVs safe. Scary for some right now. Maybe less scary when viewed in hindsight.
Ask a question or send along a comment.
Please login to view and use the contact form.
Leave a Reply