A.2 Course objectives: what is this course for?

A.2.1 The Big Picture

Want the nutshell version? At the end of this class, you probably won’t know what a Plackett-Burman design is. But if someone says to you:

“Oh, it’s a two-level factor design that you use in place of a fractional factorial when you don’t have \(2^{k-p}\) runs, and it creates partial confounding between the main effects and some interactions.”

…you will understand what they mean and whether you want to do it.

The goal is that at the end of this course you will be able to:

  1. Perform the design and analysis of a basic experiment
  2. Work with a boss or colleague to learn and use a new experimental design

The second one is just as important as the first, if not more so! Experimental design is an enormous field in which many people happily spend their entire careers toiling away. We will barely scratch the surface in this course. The goal is to scratch the surface in enough places so you know where the surface is. And to learn how to use a shovel.

To say this with less metaphor and more detail: there’s a list of skills that you will work on in this course, and a list of concepts or topics that you will meet.

A.2.2 Skills

You’ll work on these skills basically all the time, but especially in projects, which focus on applying concepts. The “big ideas” are:

  1. Use experimental design vocabulary to describe a research problem or experimental design. Given a research problem, identify its characteristics (by asking the right questions to figure them out!). Given the description of a design, describe what its characteristics mean in the context of the research problem.
  2. Use a model to describe the relationships between variables. Interpret the model in context.
  3. Perform inference using regression or ANOVA. Determine what inference tools are appropriate, and use some basic fixes if conditions aren’t met. Put the results in context.
  4. Choose between experimental design options based on context, and describe the consequences of your choice. Explain your choice to someone else.

These get translated into a list of specific skills. Each project gives you an opportunity to check off some of these skills; some of the skills appear in multiple projects, so you’ll have multiple opportunities to demonstrate them. Types of projects we may do in the course include:

  • Do-it-yourself experiment
  • Role-playing as consultants and clients
  • Teaching a new topic/extension
  • Writing a case study (with a consulting skit plus technical explanation)

You can see a table of which skills show up in which projects [here] (you can also use this to track when you check off each skill!).

A.2.3 Concepts

These topics are building blocks of the study of experimental design. They’ll allow you to choose and use experimental designs, as well as understand new designs and techniques.

You’ll work on these concepts in practice problems, and assess your understanding of them in Assessments.

The goal is that, for each technique we encounter, you can do the following:

  • Define it
  • Identify whether it’s happening in a given design or research problem, whether it has to be, and whether it’s a good thing
  • Describe research problems where you would need it, and experimental designs that incorporate it
  • Demonstrate how it connects to or affects the model
  • Demonstrate how it connects to or affects inference
  • Show or identify how it’s reflected in R code, if at all

The conceptual work of the course is divided into six chunks, or modules, each with a different focus:

  1. Experimental fundamentals: vocabulary, models, inference
  2. ANOVA: setup, interpretation, multiple factors
  3. Factorial designs: principles, setup, finding relevant factors
  4. Reducing runs: fractional factorial designs, D-optimality
  5. Response surface designs (RSDs): setup and interpretation, design regions
  6. Related runs: blocking, random effects, split-plot designs

As we start thinking about each topic, I’ll also provide a more detailed description of what you need to know. Let me take this moment, though, to remind you that I’m neither omniscient nor omnipotent. Things might change :)