14.1 Essentials of modeling

The scientific revolution (see Wikipedia) — often linked to the Renaissance of the 15th and 16th century AD and the age of reason and enlightenment — is intimately related to the use of scientific models and theories. Although astronomers, mathematicians, and scholars from other disciplines have always been using models, the emergence of science based on the collection of data and systematic experimentation led to a proliferation of all kinds of models. One of the earliest and most influential protagonists of this revolution was the Italian polymath Leonardo da Vinci (1452–1519, see Wikipedia) whose journals, drawings, and sketches merge an obsession with detail with an intense curiosity for bold inventions. Figure 14.3 shows his drawing of the Vitruvian Man (see Wikipedia), which anchors the study of anatomical proportions in both art and science:

The Vitruvian Man by Leonardo da Vinci, around 1490. (Photograph by Luc Viatour/Lucnix.be.)

Figure 14.3: The Vitruvian Man by Leonardo da Vinci, around 1490. (Photograph by Luc Viatour/Lucnix.be.)

This chapter aims to clarify some key concepts (models vs. simulations vs. statistics, etc.) and provides some historical background and examples. For the rest of the book, we need to reflect on the goals of modeling (see Section 14.1.4) and the criteria by which models can and should be evaluated (see Section 14.1.5).

14.1.1 Terminology

What is a model and what is modeling?

We will adopt the following definitions:

  • A model is an abstract and formal structure that provides a simplified description of a phenomenon and aims to capture its essential elements.

  • By contrast, modeling is the activity and methodology of creating, running, and evaluating such models.

Noteworthy elements of these definitions include:

  • A mathematical model is an abstract structure that captures structure, patterns, and processes in the data (Luce, 1995).

    • “abstract” implies simplification
    • “structure” implies regularity

Thus, a miniature replica would be a poor “model,” as it would be as complex as the original.

  • A description or structure does not per se yield hypotheses, predictions, or data. To gain any of these, models must be interpreted, instantiated, run, and evaluated. This is the purpose of modeling.

  • As models are typically represented in some formalism (e.g., diagrams or mathematical formulas), they can be expressed and executed in computer code.

Models are often discussed in the same contexts as scientific theories or paradigms. The term paradigm literally denotes a model or pattern, that serves as an example or typical instance of something. The role of paradigms in the history of science was popularized by T. S. Kuhn (1962), who defines a scientific paradigm as “universally recognized scientific achievements that, for a time, provide model problems and solutions for a community of practitioners” (p. 10). Thus, the current paradigm defines the type and scope of problems that scientists are willing or able to address in their theories and models.

We will mostly use the term model to denote specific instantiations of hypotheses or theories. If science was organized in neat set/subset relations, a paradigm would contain many theories, and theories would contain many models. However, reality is more messy and involves blurred boundaries, overlapping categories, and outliers that do not fit into these categories.

14.1.2 Model types vs. levels

There are many types and typologies of scientific models (i.e., ignoring those used to advertise and fit products in design and fashion). Many characterizations of models emphasize particular mathematical constructs (e.g., linear vs. non-linear models), disciplines (e.g., statistical vs. machine learning models), or technologies (e.g., computer models, R models).

Basics: Medium/How the model is formulated?

  • verbal vs. formal models (abstract, parameterized)

Examples: Verbal theories that postulate three instances (e.g., super-ego, ego, and id) or two systems (fast vs. slow) can easily accommodate a wide variety of phenomena. Unless they are specified further, they are so vague and flexible that it remains unclear how much of their narrative appeal translate into concrete predictions.

A nice example for a verbal claim that seems innocuous, but may actually be vacuous is provided by Hintzman (1991) (p. 41): A familiar claim from a sociobiology textbook is the following:

While adultery rates for men and women may be equalizing,
men still have more partners than women do, and they are
more likely to have one-night stands

(Leahey & Harris, 1985, see Google books for context and full quote).

Hintzman (1991) criticizes verbal theories and issues the challenge of constructing an actual model. According to him, such statements may sound plausible, but risk making “a mathematically impossible claim” (p. 41). Building a model would force the authors to explicate what they mean by “more partners” — and there are interpretations that would not be “mathematically impossible” (see Exercise 14.3.4).

  • Distinction within formal models: mathematical vs. computer models

Corresponding remarks:

  • Model types: We can distinguish between different types of models: (1) formal models (abstract, using mathematical formalisms) and (2) simulations (concrete, instantiations). However, the boundaries are blurred, as any thing, construct, or symbol (e.g., a number) can be represented in a variety of ways.

Typology by Lewandowsky & Farrell (2011): Three increasingly deep types of models are models that

  1. describe the data;
  2. describe the process;
  3. explain the process.

An alternative to categorizing models into types consists in asking: At which level of analysis do models attempt to explain a phenomenon?

An analytically useful one stems from neuroscientist David Marr (1945–1980, see Wikipedia), who studied the physiology of the visual system. According to Marr (1982), we can study cognition at three complementary but distinct levels of analysis:

  1. The computational level: What task does the system address? What problems does it solve and why does it face and solve them?

  2. The algorithmic/representational level: How does the system do what it does? Specifically, what representations does it use and what processes does it employ to create and manipulate them?

  3. The implementational/physical level: How are the system and its processes physically realised? What neural circuits, structures, and processes implement the system?

Importantly, we can construct models at each level. Thus, different types of models will serve different goals and generate or explain different types of data. [See Lewandowsky & Farrell (2011), for an alternative typology.)

  • Historically, the idea of using a model or simulation for predicting outcomes is not new (see e.g., Craik, 1943; Gentner & Stevens, 1983; Johnson-Laird, 1983). But creating models has become much easier with the ubiquity of computers and tools that allow us to create and evaluate models.

14.1.3 Examples

Models are being used in many realms of life and all scientific disciplines. Some examples of models include:

  • astronomy: geocentric vs. heliocentric models of the universe or solar system
  • anthropology: mythology as narrative accounts of history
  • medicine: soul in heart vs. brain
  • finance: fundamental vs. technical analysis of stock markets
  • psychology: mind as a clock vs. as a computer
  • climate research: models of atmospheric layers, and global warming

It is instructive to consider the transition between scientific theories and models in more detail. One of the most prominent models in the history of science concerns the position and role of our planet in the solar system.

The Ptolemaic model

In the 2nd century AD, the Hellenistic mathematician, geographer and astronomer Claudius Ptolemaeus (see Wikipedia) standardized a geocentric model of our universe. In this model, Earth is viewed as a sphere in the center of the cosmos. Each planet is moved by a system of two spheres, known as deferent and epicycle. Figure 14.4 illustrates these basic elements of Ptolemaic astronomy. A planet is rotating on an epicycle which is itself rotating around a deferent inside a crystalline sphere. In Figure 14.4, the system’s center is marked by the X, and our earth is slightly shifted off-center. Opposite the earth is its equant point, which is the center around which the planetary deferent is actually believed to rotate.

Key elements of the Ptolemaic model of a geocentric universe. (Image from Wikimedia commons, not drawn to scale.)

Figure 14.4: Key elements of the Ptolemaic model of a geocentric universe. (Image from Wikimedia commons, not drawn to scale.)

By all reasonable accounts, the Ptolemaic model was widely successful: It predicted different kinds of celestial motions with high accuracy and was used and taught by generations of astronomers. After more than a millenium, the model was challenged in the 16th century by the heliocentric account of Nicolaus Copernicus’ (see Wikipedia). However, as the Copernican system erroneously assumed circular orbits of the planets around the sun, it was no more accurate than Ptolemy’s system. Despite its merits in simplicity, the heliocentric account faced strong opposition by contemporary natural philosophy and religious doctrine (see the trials of Giordano Bruno and Galileo Galilei).

The consensus on the Ptolemaic model only faded in the 17th century, when empirical observations by Johannes Kepler (see Wikipedia) and his heliocentric model with elliptical planetary orbits provided a superior account.10

Thus, “being right” is rarely the reason for adopting a new model — and requires a verdict that is only available in hindsight. Instead, replacing a dominating scientific model can take a long time and often faces considerable resistance by established authorities. (See T. S. Kuhn (1962) and Lakatos & Feyerabend (1999) for the philosophical background.)

Models in psychology

Models of mental processes are particularly tricky. Is the occurrence of a global pandemic best explained by some invisible virus or by the will of God? When we feel stressed by professional or private challenges, should we look into our personality traits, some lack of skills, or for repressed childhood memories? Our cultural history provides a rich compendium of anecdotal, mythological, religious, and scientific models that offer competing explanations for any turn of events or whim of fate.

An important realization: It is easy to “explain” phenomena in verbal terms. For instance, equipped with some “systems,” like Freud’s three instances (super-ego, ego, id) or the more modern so-called “dual-process” theories (i.e., the interaction of a fast/intuitive and a slow/deliberate system) pretty much everything and its opposite can easily be explained. The problem with this is that explanations that always work for everything are vacuous, as they are impossible to falsify. Thus, a good model is not one that explains everything, but a model that is specific enough to fail and can be replaced by a better model.

Models force us to be explicit about assumptions, parameters, and processes. By checking whether our intuitions about some hypothetical system matches what actually arises from its implementation, models act as insurance policies against speculative leaps. Thus, models provide several benefits that are absent from narrative accounts. They

  • make scientific thinking more reproducible,

  • allow us to avoid common mistakes that may bias our intuitive reasoning, and

  • facilitate the communication about psychological capacities and processes

See Farrell & Lewandowsky (2010) for a more detailed version of this argument.

14.1.4 Goals

What is the goal of modeling?

Most people intuitively anwer that a model must be an accurate copy of the phenomenon it represents. However, while a certain degree of accuracy may be necessary feature of a good model, it cannot be a sufficient one. For instance, the most accurate model of a brain would be the brain itself (or an exact replica of it). However, unless we can construct, examine, and understand the model, having access to the most accurate model is completely intransparent and useless.

By contrast, a mathematical formula that predicts a physical phenomenon is very different from the phenomenon itself. For instance, the decrease of atmospheric temperature~\(T\) with increasing altitude~\(z\) can be described and predicted by the so-called Lapse rate: \(\Gamma = -\frac{dT}{dz}\). However, nothing about this model is particularly warm/cold or high/low.

Thus, the main criterion for evaluating a model is not truth or accuracy, but whether the model is useful. And as usefulness is a functional term, it always depends on the goals and purposes for which the model is designed.

The goal of modeling is succinctly summarized in the following statement (Wickham & Grolemund, 2017):

The goal of a model is not to uncover truth,
but to discover a simple approximation that is still useful.

Wickham & Grolemund (2017), Ch. 23: Model basics

Wickham & Grolemund (2017) further relate this to the famous aphorism that “all models are wrong”:

Since all models are wrong the scientist must be alert to what is importantly wrong.
It is inappropriate to be concerned about mice when there are tigers abroad.

(G. E. P. Box, 1976, p. 792)

The corresponding Wikipedia entry cites the following section (from G. E. Box, 1979):

Now it would be very remarkable if any system existing in the real world could be exactly represented by any simple model. However, cunningly chosen parsimonious models often do provide remarkably useful approximations.
For example, the law PV = RT relating pressure P, volume V and temperature T of an “ideal” gas via a constant R is not exactly true for any real gas, but it frequently provides a useful approximation and furthermore its structure is informative since it springs from a physical view of the behavior of gas molecules.
For such a model there is no need to ask the question “Is the model true?” If “truth” is to be the “whole truth” the answer must be “No.” The only question of interest is “Is the model illuminating and useful?”

Thus, rather than aiming to approximate truth, models are useful constructs that contain intentional simplifications.

The basic distinction between explanation and prediction does not exhaust potential goals of modeling. For instance, the REDCAPE acronym by Page (2018) distinguishes seven different uses of models:

  1. Reason: Identify conditions and deduce logical implications.
  2. Explain: Provide testable explanations for phenomena.
  3. Design: Choose features of institutions, policies, or rules.
  4. Communicate: Relate knowledge and understanding to others.
  5. Act: Guide policy choices and strategic actions.
  6. Predict: Predicting future or unknown (categorical or numerical) phenomena
  7. Explore: Investigate hypothetical cases and possibilities.

Interestingly, Page (2018) also advocates a many-model approach: Rather than searching for the “right” model, we can gain insights by viewing our problem through a multiplicity of lenses.

the same list could also be used to explicate the goals of science in general. Thus, the methodology of modeling

Summarizing possible goals of modeling requires that we spell out different aspects of “usefulness”:

  • scope: address or categorize phenomena
  • accuracy: describing or explaining data
  • predictive power
  • parsimony: abstraction and simplicity
  • aesthetics: elegance and beauty

Some of the later goals (like simplicity or elegance) may only matter when accuracy and predictive power of models is comparable. Also, these goals can be turned into criteria for evaluating models.

Overall, it is not wrong to state that models are essential tools for thinking about the interplay of theory and data. As with any tool, models can be used wisely or unwisely. Figuring out how to use models wisely is the goal and purpose of modeling.

14.1.5 Evaluating models

What determines the merits or success of a model?

  • A model’s quality always depends on its purpose: What is considered “essential” depends on our purposes. For deciding which clothes to pack for a vacation, having a weather model that shows the range of average temperature values for each month may be good enough. However, for predicting the temperature for hiking trip over the next two days, such a coarse model may be inadequate.

Note that both instances of a temperature model would still be judged by the same metric (e.g., a comparison between predicted and measured temperature, where we would ask for smaller deviations for a near-term prediction than for a long-term prediction). However, we often would want to use more than one criterion (e.g., not just temperature, but also expected amounts of sunshine or precipitation). In fact, most real-world models will typically be evaluated on multiple critera.

The range of criteria for evaluating models mirror the aspects of “usefulness” (from above):

  • scope: which phenomena are addressed or categorized
  • accuracy: fit to or distance from data
  • predictive power: ability to predict phenomena of interest
  • parsimony: abstraction and simplicity
  • aesthetics: elegance and beauty

Statistical models

Formal models in many natural sciences often appear in the form of statistical models. Although this book is not on statistics, any interest in formal models needs to briefly discuss three elementary dichotomies in statistics:

  1. Sample vs. population: When formulating hypotheses or theories, we typically address populations of entities. Howerver, our data rarely contains observations of entire population. Instead, we collect data from samples and analyze them in order to infer characteristics of the underlying population.

  2. Data as “information given” (experienced or observed) vs. model as an abstract description of data: A (verbal/mathematical/statistical) model describes (given) data, but can also be used to generate new data.

  3. Explanation vs. prediction: Correspondingly, data can be used to explain or verify a model, or to predict new/unobserved data.

Overall, the activities of data collection and modeling are not iterative and mutually exclusive, but rather describe a cyclical process.

Data in models

Data serves two very distinct functions in models:

  • generate hypotheses
  • evaluate and test predictions

Importantly, data often is a finite resource. Thus, it is very important to use the available data in a sensible and smart fashion.

Typically: Dividing data into two main parts that are being used for fitting vs. for prediction.

See Chapter 5 Spending our data of Tidy Modeling with R for ways of sampling and splitting data.

  1. For some historical context, note that Kepler’s astronomical work was interrupted by the need to defend his mother, Katharina Kepler, against a charge of practicing witchcraft.↩︎