The risk of heuristics
pp. 111-123
in: Robert Zwilling (ed), Natural sciences and human thought, Berlin, Springer, 1995Abstract
Heuristics are ways of reasoning that often lead to the solution of problems, although the reasoning is not logically valid. A simple example is this: coming home from a long day of work, you try to turn on the light in the hallway, but it remains dark. What do you do? If you are like most people, you will first move the switch a few times, just to check that you did the right thing. Then, when you believe that something is really wrong, you try a number of solutions. Probably, you will first check the light bulb, then the fuse, then the lamp etc. Most people have a large repertoire of solutions for such everyday problems; they access the items on the list in the order of frequency of occurrence. Since lightbulbs blow out more often than fuses, lightbulb comes before fuse (see [11]). The usage of frequency as the ordering principle seems perfectly logical, but in the context of the study of large industrial accidents it is called "frequency gambling" [8]. The gamble is that if the cause is not among the high-frequency causes that have occurred in the past, the problem may develop into a disaster before one reaches the end of the repertoire. This explains why actual accidents often follow weird scenarios: normal problems are correctly diagnosed before they turn into accidents, and unusual problems are not. Frequency gambling is helpful, because in most cases it rapidly leads to a solution, but it is not logical; if all the lights in the house are out, it is more likely that somehow the electricity supply has been cut off, rather than that all light bulbs have given up simultaneously. The logical solution makes use of such diagnostic information, which either presents itself or is actively sought. A structured diagnosis will always lead to a solution — more slowly than heuristic reasoning if the solution is a high-frequency phenomenon, but much more quickly when it has a low frequency. The advantage of the frequency heuristic is speed, with the risk being that the correct solution may be discovered too late. All heuristics suffer from this dilemma: they work quite well most of the time and lead to disaster in a few exceptional cases.