Week 5 Theory and Learning Analytics

Learning analytics are about learning, so theories of learning should not be ditched despite the data deluge that has been fueling the growth of learning analytics.

Since the dawn of the field, there have been voices arguing for the importance of learning theory in the design, development, and implementation of learning analytics. In 2015, a special section was published by the Journal of Learning Analytics on the relation between learning theory and learning analytics. In the guest editorial, Wise and Shaffer (2015, p. 5) write:

It is an exhilarating and important time for conducting research on learning, with unprecedented quantities of data available. There is danger, however, in thinking that with enough data, the numbers speak for themselves. In fact, with larger amounts of data, theory plays an ever-more critical role in analysis.

The notion that theory matters even more in “big data” research in education goes against what was proposed in a controversial Wired article titled “The end of theory: The data deluge makes the scientific method obsolete” (Anderson, 2008). Now there is little disagreement that learning theory is essential for learning analytics.

5.1 Learning Analytics and Learning Theory

Briefly speaking, there are essentially two ways theory is important for the field of learning analytics.

1. Theory Use in Learning Analytics

As a field of research and practice, learning analytics work naturally draws from all sorts of theories. As summarized by Wise and Shaffer (2015, p. 9):

  • Theory gives a researcher guidance about which variables to include in a model
  • Theory gives a researcher guidance about what potential confounds, subgroups, or covariates in the data to account for
  • Theory gives a researcher guidance as to which results to attend to
  • Theory gives a researcher a framework for interpreting results
  • Theory gives a researcher guidance about how to make results actionable
  • Theory helps a researcher generalize results to other contexts and populations

For example, self-regulated learning (SRL) is widely used in learning analytics and MOOC research. SRL as a learning theory has informed data collection, data transformation, data mining, and result interpretation.

2. Theory Building in Learning Analytics

Even more exciting to me is the possibility of building new theories of learning and teaching, based on fine-grained data analysis enabled by advanced computational methods. Learning analytics and educational data mining research could give rise to new theories. For example, whether keystroke analysis can shed light on our understanding of writing processes? Whether temporal analysis can uncover “productive” patterns of collaborative discourse?

Equally valuable are theories of learning analytics usage in emerging socio-technical contexts. For example, as new learning analytics tools are put in the hands of instructors to aid their pedagogical decisions, domain theories of analytics use can be created (e.g., van Leeuwen, 2015).

After exploring cases and proposing possible project ideas last week, it is a great time for us to think about learning theory relevant to our project ideas!

5.2 Week 5 Learning Activities

5.2.1 Read, annotate, and discuss

This week’s readings speak in particular to theory use in learning analytics. Reading #2 provides a concrete example of using a well-established learning theory (self-regulated learning in this case) in a research study. Reading #3 (optional) is the controversial Wired article mentioned above.

  1. Skim: Wise and Schaffer (2015) - download link. This reading is a guest editorial about learning analytics and learning theory. Please skim through it and no need to annotate closely.
  2. Read & Annotate: Winne (2017) - download link. This reading discusses a very established learning theory – Self-Regulated Learning (SRL) – that is widely adopted in learning analytics. Please read and take time to “notice” how the author discusses SRL and connects SRL constructs with data.
  3. Skim: Kizilcec, Pérez-Sanagustín, and Maldonado (2017) - download link (Note: This file is accessible via the UMN Libraries.) The third reading is a study that examines SRL in MOOCs. Please skim through it to see how the authors used SRL theory to grapple with the learning phenomenon.

As always, please annotate and interact as you like. Please use tags intentionally in your annotations to help you retrieve ideas potentially useful for our course projects.

As a reminder, please strive to stick to the weekly participation schedule:

5.2.2 Meet Learning Analytics Experts

Dr. David Williamson Shaffer is the Vilas Distinguished Professor of Learning Sciences at the University of Wisconsin-Madison. He is known for his work on game-based learning and Epistemic Network Analysis. Below is a keynote speech he delivered at the 2018 Learning Analytics and Knowledge conference in Sydney, where he summarized his idea of “quantitative ethonography” and ways to tackle theory scarcity in the age of “big data.”

5.2.3 Special Interest Group (SIG) Signup

I will distribute a SIG signup form on Slack. Please fill the form with your preferred topics.

As a reminder:

“With support from the instructor, each SIG is expected to take a lead on designing learning activities, presenting key ideas, and facilitating discussion. Each SIG will meet with the instructor one week in advance to finalize their course plan.” (From course syllabus)

Feel free to ask questions on Slack if you’re unsure about any SIG topics.

5.2.4 Continue to advance your project idea

Please continue to articulate or advance your project idea. Use this week’s readings and discussions to help you more clearly conceptualize the learning phenomenon you’re interested in. Do some literature search to find out theories relevant to your project. Share in the #projects channel things you find!

Have a great week!

References

Kizilcec, René F, Mar Pérez-Sanagustín, and Jorge J Maldonado. 2017. “Self-Regulated Learning Strategies Predict Learner Behavior and Goal Attainment in Massive Open Online Courses.” Computers & Education 104: 18–33. https://doi.org/10.1016/j.compedu.2016.10.001.

Winne, Philip H. 2017. “Learning Analytics for Self-Regulated Learning.” In Handbook of Learning Analytics, edited by Charles Lang, George Siemens, Alyssa Wise, and Dragan Gasevic, First, 241–49. Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.021.

Wise, Alyssa Friend, and David Williamson Schaffer. 2015. “Why Theory Matters More Than Ever in the Age of Big Data.” Journal of Learning Analytics 2 (2): 5–13. https://doi.org/10.18608/jla.2015.22.2.