Consider the following situation: An agent needs to fulfill a mission in a maze of sub-terranean tunnels, but several adversaries are out to kill him. Beyond dexterity and an ability for fast movement, the setting requires a talent for spatial navigation and a keen alertness for observing his enemies which are constantly closing in on him. Before they get too close, he needs to interrupt his mission to kill some of his foes. If these sentences made you think of James Bond, you are watching too many agent movies. Actually, they describe Pac-Man, an action game for video arcades developed by the Japanese company Namco and released in 1980 (see Wikipedia). While the idea of a yellow pizza-shaped blob chewing pills and chasing ghosts named Blinky, Inky, and Pinky may not be your first association to “action game,” the game has all ingredients for days of fun.20
Motivation: Computer gaming as fun (i.e., the opposite of work) vs. playing as an essential aspect of human nature.
Clarify key terminology:
social := multiple agents.
games vs. strategic games
Complexity of social situations
18.1.1 What are social situations?
A social situation can turn a game against nature (e.g., an agent exploiting some resource) into a social game (e.g., an agent cooperating or competing with other agents). However, social games do not necessarily replace games against nature, but are typically added to them. This may suggest that social games are necessarily more complex than individual games, and that models of social situations need to be more complex than models of individual agents. Before we dispell this common, plausible, but false stereotype, we need to take a closer look at the concept and nature of games.
From complex games (life, pac man, team sports) to simple games (e.g., rock-paper-scissors, tic-tac-toe, ultimatum game).
What’s the essence of games? Perhaps impossible to define (see Wittgenstein) and may take a lifetime to master (see chess, go, but also sports), but often easy to show and play. Some key elements of games are:
- some goal
- some obstacles (e.g., other agents competing for the same goal)
- some rules
A key aspect of games: Setting and following rules. In order to play, we first need to set and agree on rules. This may sound paradoxical, but following rules is not the opposite of freedom, but the precondition for playing a game.
Define games := freedom to follow rules (see Homo ludens, and corresponding considerations by Schiller and Wittgenstein)
Topics and terms studied by game theory: cooperation/defection, payoff matrices, and equilibria
In Section 17.3, we described the introduction of multiple agents as an additional level of complexity. When it is difficult to describe the interactions between an individual agent and its environemnt, it seems intuitively plausible that adding other agents renders situations more complex. This is due to the fact that considering social situations raises important questions regarding the motivations, relations and interactions between agents (e.g., are they cooperative or competitive?), social influence (intentions and potential for communication and persuasion?), social information (observing behavior or rewards of others?), and joint impact on environment (do more agents improve or deplete resources differently?).
More fundamentally, adding complexity also renders it unclear what we mean by “the environment.” For instance, from the myopic perspective of any individual agent, the beliefs, goals, strategies, and behavior of other agents can be described as part of the social environment. Thus, like the term “background,” our concept of the environment is always relative to our model’s primary focus, target, or “foreground.”
Also, we must not confuse potential complexity with actual complexity. Allowing for social situations mostly adds potential for complexity, but does not necessarily require more complex models. And even when added complexity requires messier models (e.g., to accommodate changes on additional dimensions), this does not necessarily render life more challenging for agents. For instance, copying the behavior of others can be much simpler than learning what’s best in an environment. Given some boundary conditions, simple strategies can be both smart (when others are well-adapted) or stupid (when superior solutions exist).
Actually, the potential of simple models for complex phenomena should not surprise us. Note that a theory of Pavlovian learning can be fairly successful by only using terms like “dog,” “bell” and “food” (or technical terms like UCS, CS). Importantly, a model of learning can be very good at both describing and predicting the dog’s behavior without modeling complex interactions in neuronal networks.21
Thus, social games allow for more potential complexity (i.e., interactions of agents with environments and with each other). However, from an agent’s or the modeler’s perspective, this does not necessarily make them more complex.
18.1.2 Models of social situations
The basic models of agents and environments discussed in the previous chapter (see Chapter 17 on Dynamic simulations) were typically developed for individuals. For instance, the framework of reinforcement learning (RL, or the more general class of Markov Decision Processes, MDPs) allows an agent to learn a strategy that maximizes some reward in a stable environment. The options in this environment may be stochastic (e.g., a multi-armed bandit, MAB). Provided that the agent has sufficient time to explore all options, the agent is guaranteed to find the best option. However, if fundamental aspects of the environment change (e.g., by adding options or changing their reward functions), a learning agent may no longer converge on the best option.
Social situations typically change an environment in several ways. Depending on the details of the agent-environment interaction (e.g., how are rewards distributed), they may introduce competition between agents, opportunities for cooperation, and new types of information and learning. For instance, the success of any particular agent strategy can crucially depend on the strategies of other agents. Thus, allowing for other agents questions many results that hold for individual situations and calls for additional types of models and modeling.
In this chapter, we provide a glimpse on some important families of models. Specifically, we introduce three basic paradigms of modeling social situations:
As a teenager, I once recruited a friend to play the game to its alleged end after its 256th level. Although we tried hard to stay alive and awake, we never made it to the end.↩︎
Interestingly, prevailing scientific trends bias us towards immediately asking about the dog’s brain, rather than its auditory and olfactory senses, its digestive processes, or the nutritional or macrobiotic properties of its food.↩︎