AI and machine learning (Intro)

Learning outcomes/objective:

The material for the following sessions is based on James et al. (), Molina and Garip (), James et al. (), Chollet and Allaire () and others’ work (including some of my own).

1 What is AI and machine learning?

Source: https://en.wikipedia.org/wiki/Machine_learning

  • “Artificial intelligence (AI) is intelligence - perceiving, synthesizing, and inferring information - demonstrated by machines, as opposed to intelligence displayed by non-human animals and humans. Example tasks in which this is done include speech recognition, computer vision, translation between (natural) languages, as well as other mappings of inputs.” (Wikipedia)

  • “Machine learning (ML) is a field of inquiry devoted to understanding and building methods that ”learn” […] It is seen as a part of artificial intelligence.” (Wikipedia)

    • “Machine learning is programming computers to optimize a performance criterion using example data or past experience.” ()
    • “Machine learning is a specific subfield of AI that aims at automatically developing programs (called models) purely from exposure to training data. This process of turning models data into a program is called learning.” ()
  • Data mining: “Application of machine learning methods to large databases is called data mining. The analogy is that a large volume of earth and raw material is extracted from a mine, which when processed leads to a small amount of very precious material; similarly, in data mining, a large volume of data is processed to construct a simple model with valuable use, for example, having high predictive accuracy.” ()

2 The AI landscape

  • Different Actors with varying interests (Q: What are their interests?)
    • Researchers (public institutions), e.g., LeCun, Hinton, Bengio (Turing Award: Nobel Prize of computing)
    • Companies: Google (Deepmind), Facebook, Microsoft (OpenAI), Baidu, etc. ()
    • Countries
    • Individuals
  • History (; ; ): Skepticism warranted.. who writes the history?
    • Chollet and Allaire ()
      • 1950s (“can we make computers ‘think’”)
      • Symbolic AI: Dominant paradigm from 1950s to 1980s (peak popularity)
        • Idea: handcraft sufficiently large set of explicit rules for manipulating knowledge
          • Works for well-defined, logical problems such as chess but not for complex, fuzzy problems (image classification, speech recognition)
      • Machine learning: Took off in the 1990s
Insights
  • Various actors with varying interests that can often be subsumed under the concept of “competition”. In tehir daily lifes individuals would simply like AI to just work (but are often not aware that AI is working in the background).
  • There are various books that summarize the history of AI. Importantly, theses surveys may be biased towards certain people/contexts.

3 Stats on the field of AI

Insights
  • Massive investments in the last years
  • Ability: Best chess players are now beaten by ML algorithms.. a few years ago already actually (same for GO!)
  • Costs: Costs go down over time but… (Imagenet)
  • …computation underlying training of recently successful models is extremely high -> super computers (for notable DL systems!)
  • Research as moved into industry away from universities
  • Size of datasets necessary to train has grown for notable DL systems!
  • Share of women in AI/Computer science is low.. (possible bias!)

3.1 Language and image recognition capabilities

Source: https://ourworldindata.org/brief-history-of-ai; Data: Data from Kiela et al. (2021) – Dynabench: Rethinking Benchmarking in NLP.

4 Some debates around AI/ML

  • There are various larger interlinked debates in the field
  • [How well] Does AI really work?
  • Interpretable machine learning (e.g. )
  • Bias of ML algorithms (training data, personnel, minorities) (e.g., , Ch. 15)
    • “I call it a sea of dudes” (Margaret Mitchell, a founding member of Microsoft’s “cognition” group)
    • Garbage in, garbage out; reproduction of inequalities
  • Weaponization (e.g., , Ch. 16)
    • General adversarial networks (GANS) + adversarial attacks (e.g., , Ch. 13, 212)
    • Book: “Weapons of Math Destruction” by Cathy O’Neill
  • Technological singularity, Super intelligence

5 Machine learning tasks in the social sciences (Grimmer et al. 2021)

  • Within social sciences ML is used for various tasks as summarized by Grimmer, Roberts, and Stewart ()
  • We’ll focus on prediction! (beware of ambiguity of “prediction”)

Source: Grimmer et al. 2021

References

Alpaydin, Ethem. 2014. Introduction to Machine Learning. MIT Press.
Buchanan, Bruce G. 2005. “A (Very) Brief History of Artificial Intelligence.” AIMag 26 (4): 53–53.
Chollet, Francois, and J J Allaire. 2018. Deep Learning with R. 1st ed. Manning Publications.
Domingos, Pedro. 2015. The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books.
Grimmer, Justin, Margaret E Roberts, and Brandon M Stewart. 2021. “Machine Learning for Social Science: An Agnostic Approach.” Annu. Rev. Polit. Sci. 24 (1): 395–419.
James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning: With Applications in R. Springer Texts in Statistics. Springer.
Marcus, Gary, and Ernest Davis. 2019. Rebooting AI: Building Artificial Intelligence We Can Trust. Knopf Doubleday Publishing Group.
Metz, Cade. 2022. Genius Makers: The Mavericks Who Brought AI to Google, Facebook, and the World. United Kingdom: Penguine Random House.
Molina, Mario, and Filiz Garip. 2019. “Machine Learning for Sociology.” Annu. Rev. Sociol., July.
Molnar, Christoph. 2022. “Interpretable Machine Learning.” https://christophm.github.io/interpretable-ml-book/.
Roser, Max. 2022. “The Brief History of Artificial Intelligence: The World Has Changed Fast – What Might Be Next?” https://ourworldindata.org/brief-history-of-ai.
Zand, Bernhard, Christoph Scheuermann, and D E R Spiegel. 2018. “Pedro Domingos on the Arms Race in Artificial Intelligence.” https://www.spiegel.de/international/world/pedro-domingos-on-the-arms-race-in-artificial-intelligence-a-1203132.html.