Toyota's Woes Raise Questions About Auto Electronics
Could software, or faulty logic design, be at the root of runaway acceleration problems plaguing Toyota automobiles?
Toyota steadfastly denies this is the case, but others are not sure.
In recent testimony to the U.S. House of Representatives, Toyota CEO President Akio Toyoda insisted that neither electronics nor software could be blamed for the rash of runaway Toyotas reported in the U.S.
Nonetheless, the U.S. National Highway Traffic Safety Administration (NHTSA) heard enough concern over electronics that it is opening an investigation of possible electronic and software defects. And Toyota itself is installing, at least on some models, a brake override system to bypass electronic control, one that shuts the engine off when both the accelerator and the brake pedal are held down at the same time.
That Congress, not to mention the American public, is reluctant to accept Toyota's assurances typifies the mistrust felt toward microprocessor-based systems in automobiles, even as people rely on them daily.
Could the growing complexity of software in our automobiles be leading to more software bugs in our automobiles, some leading to deadly behaviors? Or, are such suspicions based, perhaps unfairly, on experiences with the buggy software on home computers, software that is probably designed with far less rigor than automotive software?
In short, can the electronics in automobiles be trusted without a second thought? Depends on whom you ask.
The unintended acceleration problem has been seemingly plaguing Toyota for several years, though it was the news about the tragic deaths of a San Diego family last August that brought national attention to the matter.
A borrowed Toyota Lexus, driven by off-duty police officer Mark Saylor, started accelerating uncontrollably, reaching a speed above 100 miles per hour and plowing through an intersection before careening off into a river basin, killing all four passengers.
Toyota itself places the blame on accidents like these on sticky accelerators and improperly positioned floor mats. "Toyota is confident that no defect exists in the [electronic control unit]," an F.A.Q. on the Toyota site explains.
This response doesn't satisfy at least some members of Congress.
"You can't rule [electronics] out because you don't know. You can't conclude [the cause] one way or the other," said U.S. Senator Olympia Snowe, a Maine Republican, during a Senate Committee on Commerce, Science, and Transportation hearing held Wednesday to investigate the NHTSA's handling of the complaints about unintended acceleration.
Snowe noted that NHTSA couldn't rule the possibility out because it had no software engineers on staff to investigate the claim.
The component under scrutiny has been the electronic throttle control system (), which Toyota started installing in some of its cars beginning in 2002.
Before electronics were introduced in autos, the accelerator pedal was directly connected by a cable to the throttle, which regulates the amount of air, and hence gas, entering the engine.
Now, an electronic control module [ECM], consisting of two processors and nonvolatile memory that holds the logic of the unit, sits between the pedal and the throttle. The ETC monitors the location of the pedal through two position sensors connected to the accelerator.
Two additional sensors are also connected to the throttle. The throttle itself is controlled by a motor, which in turn is controlled by the ECM.
Toyota claims its unit operates under fail-safe conditions, using self-diagnostic logic. If output values from two pedal accelerators do not match, or if the two values from the throttle do not match, or either is showing values that are out of their normal ranges, then the ETC will revert to a fail-safe mode, which means an alert light will come on in a dashboard and the car will run at reduced speed.
David Gilbert, a professor of automotive technology at Southern Illinois University Carbondale, found that the ETC is not foolproof, despite Toyota's claims. In tests, which he later described before last week's Congressional hearings, he found that the ETC did not detect certain types of short-circuit malfunctions that could occur with the pedal sensors. If the ETC did not detect the complete possible range of errors, then it could not enter into a fail-safe mode, he argued.
"Some types of [ETC] circuit malfunctions were detectable by the ECM, and some were not," Gilbert told Congress. "The Toyota detection strategies were unable to identify malfunctions of the APP sensor signal inputs to the ECM."
With this in mind, it could be conceivable that malfunctions that were not anticipated by the ETC could lead to a runaway engine, Gilbert argued.
Speaking before Wednesday's Senate hearing, Toyota Vice President Takeshi Uchiyamada said that the company tried to replicate Gilbert's test and was unable to produce the same results. "In any case, it would be extremely unlikely or difficult to reproduce in the real world," he said, speaking through a translator.
But trying to ferret out everything that could happen in normal operations is a demanding task, one perhaps that the auto industry, and its parts suppliers, are just learning, some say.
Despite the best efforts of coders and reviewers, most professionally written software programs still have about one bug per thousand lines of code, said Andy Chou, chief scientist at Coverity. For code that has gone through extensive review -- such as code used in airplanes -- that number jumps to about one in 10,000
Coverity is an expert in the area. It analyzes software for defects and counts among its customers automobile-parts manufacturers such as Bosh, and car manufacturers such as Daimler and Renault.
Citing a widely used figure, Chou said that the average luxury automobile has more than 100 million lines of code, spread across all of its microcontrol units.
If this is indeed the case, then the auto has twice the amount of code found in a desktop operating system.
And, in many cases, this code could have been developed under the severe time pressure of getting new models out each calendar year.
"There is a different set of pressures for the automobile industry than there is in the avionics industry, where a plane may last for decades," Chou said.
As this collection of software grows more complex, it may become susceptible to more errors that are more difficult to find, or even reproduce.
For instance, one of the challenges investigators and mechanics have had is in reproducing the accelerator problem in test labs. This could be due to some sort of race condition, in which a system may behave differently from one moment to the next depending on the largely indeterminate timing of certain interrelated events, Chou said.
Also, as different control units of the automobile are networked together -- which manufacturers are increasing doing -- then the degree of complexity, and the ways errors can spring up, grow even larger still, Chou said.
"We're talking about very large, complex systems," Chou said. "When these systems are connected, they can start to have behaviors that are emergent. They are not necessarily behaviors that they would have on their own but you put them together, they may do things that are unexpected."
Not all agree with this assessment, though.
Reza Hoseinnezhad, a research fellow at the The University of Melbourne's Melbourne School of Engineering, has done extensive work on developing brake-by-wire prototypes, or electronic-based braking systems. He sees the worries over electronics design as largely overstated, given the quality-control measures the automotive industry uses for mission-critical components.
"The reported faults are very rare and if they are occurring within the software and logic design of the Toyota cars, they would have been predictable and debuggable before mass production," Hoseinnezhad said in an e-mail interview.
Which is not to say that Toyota, or other car manufacturers, haven't experienced software problems before. In February, the car maker recalled 400,000 of its Prius models to update the software for its antilock brakes. And in 2005, the car manufacturer recalled 160,000 of the Prius models due to faulty software that caused the engine to stop running.
But while automobiles do contain a lot of code, management is simplified quite a bit due to the modular design, Hoseinnezhad maintained.
"In modular style, the total length of the code is very long, but the complexity is comprehendible, the code is understandable and debugging is easy," he said. "Logical errors are usually expected to be detected and debugged before mass production of such systems. "
One thing is certain: This is not the last time suspicions will be raised about automobile electronics and the software that supports them. All cars rely on electronics, and there are few standards in place to ensure their safety.
This may be one reason why the U.S. White House has allocated funding for an additional 66 positions within NHTSA, and Congress is mulling over new laws to strengthen the review of potential automotive hazards. Toyota itself has set up an independent panel to review its ETC systems.
Whether or not electronic systems are already safe, industry and government are both learning that the public needs to trust in their safety as well.
"This loss of trust is more costly for us than anything else," Uchiyamada said.
