Skip to content
Tech News
← Back to articles

Illusions of understanding in the sciences

read original more articles
Why This Matters

This article highlights the complexities and ambiguities inherent in scientific understanding, emphasizing that our grasp of causality and explanation is often shaped by language, perception, and human conceptual limits. Recognizing these illusions of understanding is crucial for both scientists and consumers of scientific information to appreciate the provisional and interpretative nature of scientific theories. This awareness can foster more critical engagement with scientific claims and promote clearer communication in the tech industry and beyond.

Key Takeaways

Preface

Humans try to understand our universe in terms of causes that are often inferred from senses of agency and intervention: We push on a door and it opens. In most simple cases the result of an intervention is predictable. The first steps in forming such understandings in science are observations of regularities, relations that are often expressed as correlations and sometimes stated as ‘laws’. Starting with such observations, scientists form explanations through a process of inductionFootnote 1. They express their understanding of the causes of the data with theories. These are sometimes stated as verbal hypotheses of proposed mechanisms and sometimes as mathematical or computer simulation models that increase precision of explanation and allow quantitative prediction. As science progresses, the theories become more powerful and more valid, often doing so by explaining a narrow subdomain of phenomena, and more rarely, but importantly, generalizing and explaining a wider range of results.

Understandings are sometimes implicit and sometimes visual in character, but most often expressed with language, either in print or in explanations, and thereby inherit ambiguities of meaning. Such ambiguity of meaning applies as well to the terminology in this essay, terms such as “understanding”, “explanation”, “causality”, “deduction”, and “induction”. These are human conceptual constructs and will be understood and used differently by different readers. As such they resist precise definition and attempts to impose such definitions will limit the understandings and explanations reached by different readers. Our usage is therefore intended to be vague and admit of different connotations and meanings, and intended to match common parlance. Thus, roughly, understanding refers to an individual’s account of the causes of some phenomenon, explanation refers to the way an individual conveys their understanding to others, causality refers to the mechanisms thought to be most important in producing some phenomenon, deduction refers to the use of agreed upon rules of logic and mathematics to reach conclusions based upon agreed upon premises (though different deductions occur when there is disagreement about the rules and premises), and induction refers to forming a currently best causal account of some phenomena based on analyses of the data, past results and theories, and everything else known to the person carrying out the induction. That ambiguities of language abound in scientific practice is clear from fierce debates that often arise among researchers and theorists about the meaning and implications of experiments when those disagreements arise from different interpretations of the language used.

Understandings may produce particularly strong illusions when they are satisfying, even when scientists are aware the explanations are incomplete. Some scientists may be motivated by a plausible and satisfying explanation not to look deeper. Not looking deeper is natural if the incomplete theory is useful (C.S. Peirce, 1992/1898), and can be motivated by an awareness that all models are incomplete. George Box reprised a long scientific history in his 1976 publication with the phrase: All models are wrong, but some are useful (Box, 1976). Incomplete and partial causal accounts are indeed useful: they have led to the many advances in science that have benefited modern society. However, such incomplete accounts can also be deceptive, misleading, and/or wrong, and produce costs, such as slowing scientific progress, and even harm, as when poor science and a bad explanation led to erroneous conclusions that vaccines produced autism (Madsen et al., 2002).

It is easy to find multiple levels of explanations for almost any phenomena of interest to scientists, and those different levels might be useful for different purposes, or harmful for other purposes. Thus we may believe that humans are causing climate change by burning fossil fuel, a very shallow explanation of a very complex process, but useful if it leads to development of other forms of energy production. Conversely, taking the complexities into account could be and has been used to dismiss entirely the possibility of human agency. To take another example, a scientist may believe and publish studies claiming to show that forgetting occurs because humans inhibit and suppress unwanted memories. This is surely a shallow account (possibly a wrong one: Jonker et al., 2013; Deferme, Otgaar, Dodier, Körner, Mangiulli, Merckelbach, Sauerland, Panzavolta & Loftus, 2024; Raaijmakers & Jakab, 2013), but that belief may lead to useful empirical studies. Conversely, belief in this account could be harmful if incorrect, possibly leading to clinical treatments of adults presumed to have forgotten traumatic events in childhood when those events did not occur (Loftus, 1993; Loftus & Ketchum, 1994; Loftus, 1996).

Theories stated with mathematical equations or computer simulations add precision to theory, are often easily falsifiable, and are useful when they predict well. On the other hand, accurate prediction does not guarantee that even the producer of the model understands the way the model works. Examples of such failures to understand abound in the history of science. An early example involving a great scientist is described by Stigler (2006): In the 1690 s Isaac Newton was asked to assess which of three dice outcomes was the most probable. He correctly carried out the needed calculations and produced the correct answer. However, when offering a verbal explanation why the answer was correct, he missed a true explanation. His explanation, when applied to slight variants of the original problem, would have produced wrong answers. Thus accurate prediction can be misleading to scientists generally to the extent that it conveys the impression that the model explains well. This is especially true of mathematically specified theories that predict well (e.g., quantum field theory), and of some simulation models that often predict well (e.g. ChatGPT), but are enormously complex and difficult to understand by almost everyone.

Illusions of understanding can take several (overlapping) forms. Some that are commonly encountered are: (1) Illusions of explanatory depth (we think we personally understand things in more detail than we do). (2) Illusions of explanatory completeness (even if we don’t think we fully understand it ourselves, we think the best experts do). (3) Illusions resulting from understanding something other than the goal (e.g. we believe we understand the formation of memories because we understand the anatomy of the brain site, the hippocampus, that is needed for such learning). (4) Illusions due to simple statements giving a feeling of insight (such as when tautological statements seem insightful because they are framed in a reductionist manner). (5) Illusions (as described earlier) that one understands the cause of phenomena because there exists a model or procedure that predicts well. (6) Illusions of causal strength (attending to an observed relation makes one believe the causal connection is stronger than it is). (7) illusions that one can describe causes simply. (8) Illusions by the explainer that the recipient understands what the communicator intends. (9) Illusions by the recipient of an explanation that the communicator understands well and that the explanation is correct and complete.

Such illusions are inevitable in a universe that is infinitely complex, or almost so. Thus, all our theories (our causal explanations) are approximations. Giorgio Parisi, 2021 Nobel laureate in physics, said: “The truths of science are provisional, but not relative. Scientific truths are always ‘approximations to the truth’.” Many physicists are pleased that 150 years of experiments and theory development have produced quantum field theory, a theory that predicts observations of fundamental particles and their interactions incredibly well. Other physicists ask why this theory works, why the many constants have the values they do, why the processes interact the way they do, why some processes have symmetries and others do not. If reality is for all practical purposes infinitely complex, then all theories and understandings are incomplete. Thus not just novices but leading scientists are forced to use a variety of partial levels of understanding and explanation.

Even what seem to be simple questions asked about some observed events can lead to an unending and ever deeper series of questions. “Why do we see through glass and not other things?” “How does a mirror reflect light?” (Even physicists would struggle answering such questions at the deepest current level of understanding). This is often the case for queries from children, an example being a question asked of her mother by the first author’s five year old granddaughter: “I know how I was born and I know how you were born…but how was the first person born?” How would one start producing a deep answer and explanation? Would one start with the ‘big bang’? It is most likely that every question is fundamentally this complex: “What is water?” has a superficial answer much like a definition: “molecules with two hydrogen and one oxygen atom”. But asking what are molecules and what are atoms and what causes water molecules to form a solid, fluid, or gas leads to an unending series of other explanations.

History suggests there will always be better and deeper theories around the corner. Scientists working on the same problem often generate different types of explanations, and it is not always clear which level is ‘deeper’. Consider four scientists seeking an explanation of how monarch butterflies navigate thousands of miles to the same precise location while taking four generations to make a complete trip. Scientist 1 wants to understand functionally, explaining why it is necessary to have multiple generations because of a combination of lifespan limits and bioenergetics; Scientist 2 seeks to explain what navigational cues could provide such precision; Scientist 3 seeks to understand how location information is transmitted across generations; and Scientist 4 wants to understand the molecular basis of the navigation signals. Each of these offers an incomplete account, leaving gaps and the possibility of distortion. These four instances hardly exhaust the possibilities. In this case, and with many others, it is not obvious how to meld together the different types of explanations.

... continue reading