AI and machine learning (Intro)
Learning outcomes/objective:
- Learn about definitons of AI/ML and gain statistical insights.
The material for the following sessions is based on James et al. (2013), Molina and Garip (2019), James et al. (2013), Chollet and Allaire (2018) and others’ work (including some of my own).
1 What is AI and machine learning?
“Artificial intelligence (AI) is intelligence - perceiving, synthesizing, and inferring information - demonstrated by machines, as opposed to intelligence displayed by non-human animals and humans. Example tasks in which this is done include speech recognition, computer vision, translation between (natural) languages, as well as other mappings of inputs.” (Wikipedia)
- “the effort to automate intellectual tasks normally performed humans” (Chollet and Allaire 2018, 2) (includes chess computers!)
“Machine learning (ML) is a field of inquiry devoted to understanding and building methods that ”learn” […] It is seen as a part of artificial intelligence.” (Wikipedia)
- “Machine learning is programming computers to optimize a performance criterion using example data or past experience.” (Alpaydin 2014, 3)
- “Machine learning is a specific subfield of AI that aims at automatically developing programs (called models) purely from exposure to training data. This process of turning models data into a program is called learning.” (Chollet and Allaire 2018, 307)
Data mining: “Application of machine learning methods to large databases is called data mining. The analogy is that a large volume of earth and raw material is extracted from a mine, which when processed leads to a small amount of very precious material; similarly, in data mining, a large volume of data is processed to construct a simple model with valuable use, for example, having high predictive accuracy.” (Alpaydin 2014, 2)
2 The AI landscape
- Different Actors with varying interests (Q: What are their interests?)
- Researchers (public institutions), e.g., LeCun, Hinton, Bengio (Turing Award: Nobel Prize of computing)
- Companies: Google (Deepmind), Facebook, Microsoft (OpenAI), Baidu, etc. (Zand, Scheuermann, and Spiegel 2018)
- Countries
- Individuals
- History (Domingos 2015; Buchanan 2005; Roser 2022): Skepticism warranted.. who writes the history?
- Chollet and Allaire (2018, 2f)
- 1950s (“can we make computers ‘think’”)
- Symbolic AI: Dominant paradigm from 1950s to 1980s (peak popularity)
- Idea: handcraft sufficiently large set of explicit rules for manipulating knowledge
- Works for well-defined, logical problems such as chess but not for complex, fuzzy problems (image classification, speech recognition)
- Idea: handcraft sufficiently large set of explicit rules for manipulating knowledge
- Machine learning: Took off in the 1990s
- Chollet and Allaire (2018, 2f)
Insights
- Various actors with varying interests that can often be subsumed under the concept of “competition”. In tehir daily lifes individuals would simply like AI to just work (but are often not aware that AI is working in the background).
- There are various books that summarize the history of AI. Importantly, theses surveys may be biased towards certain people/contexts.
3 Stats on the field of AI
- See Our World in Data for various graphs
- Ability: Chess ability of the best computers
- Costs: Cost to train an AI system to equivalent performance on ImageNet
- Computation used to train notable artificial intelligence systems (FLOPS)
- Affiliation of research teams
- Affiliation of researchers behind notable AI systems
- Number of datapoints used to train notable artificial intelligence systems
- Share of women among new artificial intelligence and computer science PhDs in the US and Canada
Insights
- Massive investments in the last years
- Ability: Best chess players are now beaten by ML algorithms.. a few years ago already actually (same for GO!)
- Costs: Costs go down over time but… (Imagenet)
- …computation underlying training of recently successful models is extremely high -> super computers (for notable DL systems!)
- Research as moved into industry away from universities
- Size of datasets necessary to train has grown for notable DL systems!
- Share of women in AI/Computer science is low.. (possible bias!)
- See, e.g., Our World in Data, Statista and OECD.ai for more statistics
- AI timelines: What do experts in artificial intelligence expect for the future?
3.1 Language and image recognition capabilities
4 Some debates around AI/ML
- There are various larger interlinked debates in the field
- [How well] Does AI really work?
- “Rebooting AI” (Marcus and Davis 2019), etc.
- How should AI be tested? (e.g., competitions etc.)
- Interpretable machine learning (e.g. Molnar 2022)
- Bias of ML algorithms (training data, personnel, minorities) (e.g., Metz 2022, Ch. 15)
- “I call it a sea of dudes” (Margaret Mitchell, a founding member of Microsoft’s “cognition” group)
- Garbage in, garbage out; reproduction of inequalities
- Weaponization (e.g., Metz 2022, Ch. 16)
- General adversarial networks (GANS) + adversarial attacks (e.g., Metz 2022, Ch. 13, 212)
- Book: “Weapons of Math Destruction” by Cathy O’Neill
- Technological singularity, Super intelligence
5 Machine learning tasks in the social sciences (Grimmer et al. 2021)
- Within social sciences ML is used for various tasks as summarized by Grimmer, Roberts, and Stewart (2021)
- We’ll focus on prediction! (beware of ambiguity of “prediction”)
References
Alpaydin, Ethem. 2014. Introduction to Machine Learning. MIT Press.
Buchanan, Bruce G. 2005. “A (Very) Brief History of Artificial Intelligence.” AIMag 26 (4): 53–53.
Chollet, Francois, and J J Allaire. 2018. Deep Learning with R. 1st ed. Manning Publications.
Domingos, Pedro. 2015. The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books.
Grimmer, Justin, Margaret E Roberts, and Brandon M Stewart. 2021. “Machine Learning for Social Science: An Agnostic Approach.” Annu. Rev. Polit. Sci. 24 (1): 395–419.
James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2013. An Introduction to Statistical Learning: With Applications in R. Springer Texts in Statistics. Springer.
Marcus, Gary, and Ernest Davis. 2019. Rebooting AI: Building Artificial Intelligence We Can Trust. Knopf Doubleday Publishing Group.
Metz, Cade. 2022. Genius Makers: The Mavericks Who Brought AI to Google, Facebook, and the World. United Kingdom: Penguine Random House.
Molina, Mario, and Filiz Garip. 2019. “Machine Learning for Sociology.” Annu. Rev. Sociol., July.
Molnar, Christoph. 2022. “Interpretable Machine Learning.” https://christophm.github.io/interpretable-ml-book/.
Roser, Max. 2022. “The Brief History of Artificial Intelligence: The World Has Changed Fast – What Might Be Next?” https://ourworldindata.org/brief-history-of-ai.
Zand, Bernhard, Christoph Scheuermann, and D E R Spiegel. 2018. “Pedro Domingos on the Arms Race in Artificial Intelligence.” https://www.spiegel.de/international/world/pedro-domingos-on-the-arms-race-in-artificial-intelligence-a-1203132.html.