As our technological capabiities continue to expand, we rely on computers for increasingly important functions. While they were once used primarily for email and leisure, computers are now watching over many financial and even physical aspects of our lives. While this fact alone has troubling implications regarding our independence and self-sufficiency, I am currently speaking in concern of the general public’s expectations of these systems. We are holding unreasonably high standards for computers, with the Toyota Prius and PG&E SmartMeter cases being prime examples.

Developing perfectly reliable software is an incredibly difficult goal to reach. A large software project can be more complicated than a fine watch: Countless small parts interact with others, causing cascades of events that must be perfectly synchronized within tight tolerances. Furthermore, if one piece of the program misbehaves, the others must be able to keep running with a minimum of disturbance. There are usually far more ways in which a system can act than can be thoroughly tested. Testers need to consider every possible input in every possible order and ensure that each yields the expected output. This is like testing the security of a door lock by inserting every possible key and ensuring that only one opens the lock; even if it would eventually work, it’s foolish to attempt in most cases. However, even if an entire system behaves correctly, it must be able to handle errors beyond its control. “Defensive coding” is necessary to protect against these theoretical errors. What happens if your SmartMeter catches interference from radiation in the atmosphere while it’s retrieving your power usage report to send out? What happens of the memory chip holding your cruise control speed is defective? For software developers, two plus two just might equal five every now and then, and it can have devastating results if left unchecked in the wrong situations.

With all these problems, then, how do we place our money and even our lives in the hands of these machines and expect to be safe? Well, the average consumer just isn’t used to seeing these kinds of bugs. If a program on your computer happens to crash, you can just open it back up. In the worst case, you might lose a document or progress in a game. Even in the rare occasion where the entire system goes down, you can usually just reboot it. So even when you notice these bugs, they’re not very important because they have no large impact. So consumers carry the expectation that no “major bugs” should exist in software, and then the SmartMeters or the Priuses start malfunctioning.

The problem is that small and subtle bugs that are otherwise unimportant can cause large consequences when they effect a physical or financial system. These chaotic errors are misunderstood by most consumers. No amount of testing can ever eliminate these bugs, although they can be made very rare. We must either live with imperfection or lessen our reliance on computers. For example, Toyota could have made the brake system on their Prius mechanical rather than piping all driver input through a computer. PG&E might have incorporated a double check into the reporting functionality of their meters. This problem isn’t isolated to a couple of companies, and it won’t get any better if we just continue to expand the responsibilities of computers, because writing perfect software is much harder than we would like to believe.