Chapter 5 Experiments & Observational Studies
5.1 Video
This 11 minute video has both an example of an observational study (looking at marine life on remote islands) and an experiment (studying different treatments to help arthritis patients). An older version of this video series (that we had on old-school VHS tapes) used the Physician’s Health Study (aspirin/heart attacks).
https://www.learner.org/series/against-all-odds-inside-statistics/designing-experiments/
5.2 Association
Two variables are associated if values of one variable tend to be related to the values of the other variable. For example, I might notice that students who report studying for more hours per week also tend to be the students with the highest G.P.A.
The variables are causually associated if changing the value of one variable influences the value of the other variable.
This means if we manipulate one variable, we can change the other variable. For example, a farmer might notice that if they increase the amount of fertilizer used on a field, the yield of corn will increase. (Which is the explanatory variable and which is the response variable?)
Of course, this trend might not hold forever. If the farmer uses too much fertilizer, at some point it will actually cause the yield to decrease, as the farmer might actually kill the crop.
Just because two variables are associated doesn’t mean that there is a causal relationship. For example, suppose you own an ice cream store near the beach. You might notice that the months of the year that have the highest ice cream sales are also the months of the year with the highest number of shark attacks! This doesn’t mean eating ice cream before swimming will make sharks more likely to attack. Here, there is a confounding factor, or lurking variable that is associated with both the explanatory variable (shark attacks) and response variable (ice cream sales), in this case the time of year or the temperature. You will sell more ice cream in the summer, when more people go swimming, and thus there is a greater chance of a shark attack.
5.3 Osteoarthritis Study
Will taking a dietary supplement such as glucosamine and/or chrondroitin help bring relief to patients suffering from joint pain due to osteoarthritis? Are these supplements as effective than a commonly prescribed prescription drug (Celebrex)?
A sample of \(n=662\) patients suffering from moderate to severe pain in the knee due to osteoarthritis was recruited. They were randomly assigned to one of 5 levels of the treatment:
glucosamine
chrondroitin sulfate
both glucosamine and chrondroitin sulfate
Celebrex (a NSAID, non-steroidal anti-inflammatory drug )
placebo
Both the participants in the study and the doctors & nurses that they interacted with were unaware of which level of the treatment was being received.
The study’s primary outcome measure was a 20-percent reduction in pain scores (using the WOMAC pain scale). After the data was collected and analyzed statistically, it was shown that none of the treatments were significantly better than the placebo.
More details of the study are found here:
5.4 Designed Experiments: Single Group Design
Let us reconsider the Osteoarthritis example? How come the researchers didn’t select a single group design for this experiment.
In other words, how come the experiment wasn’t just: gather a large sample of subjects with osteoarthritis, give the subject a treatment (both dietary supplements), and measure to see if the desired outcome (reduction in knee pain) occurs?
The problem is that the effect of the explanatory variable (treatment) on the response variable (reduction in knee pain) cannot be separated from other extraneous variable(s). These extraneous variable(s) may or may not even be known to the researchers and are called confounding variables.
The presence of confounding variables would make it impossible to attribute a reduction in pain to the dietary supplements. Maybe everyone in the study decided to exercise more or improve their eating habits.
5.5 Observational Study
In an observational study, no control is placed on the independent variables. Common examples of observational studies are sample surveys. Even though no control of the independent variables are used, this is often an appropriate design. For example, if I want to know the opinions of the American public on health insurance, I would not want to manipulate the subjects into answering a certain way.
A retrospective study has participants recall past events, while a prospective study follows the participants into the future and records events. Generally a prospective study is preferred, when it is feasible.
in a case-control study, cases are those with a particular attribute and controls are those without that attribute. If I wanted to see if there was a relationship between being home-schooled and successfully graduating from college, I would compare “cases” (college students who were home-schooled) with “controls” (college students who were not home-schooled). Whether a student was home-schooled or not as a child was not assigned randomly, but occurred naturally. Another example would be studying the incidence of mouth cancer between people who chew tobacco versus those who do not.
Case-control studies are often used in situations where an experiment that uses random assignment is impossible, unethical, or too expensive/time consuming to conduct.
5.6 Experiemental Study
In a planned experiment, the researcher do manipulate, or control, the values of independent variable(s). In the arthritis study, this was done by randomly assigning the subjects to receive one of 5 possible levels of the treatment. If there is a difference in the response between the comparison groups, we can attribute the difference to the treatment. In the aspirin study, there was a statisically significant decrease in rate of heart attacks when aspirin was taken, while the arthritis study did not yield a statistically significnat decrease in knee pain.
5.7 Single Factor Design
Randomization is typically used in planned experiments to assign experimental units to different levels of an independent variable, or factor. This random selection used to be done with random number tables, but now is typically done via random number generators on computers.
Consider a single factor design. We are studying the effectiveness of a `patch’ to help people quit smoking. Suppose we have three levels of the factor: a patch with a high dose of the active agent, a patch with a low dose of the active agent, and a patch with no dose of the active agent, which serves as a placebo. If we had \(n=300\) subjects, we could randomly assign 100 to each level of the dose and determine what level of dose is most effective.
5.8 Types of Randomzied Experiments
– Randomized comparative experiment, we randomly assign cases to different treatment groups and then compare results on the response variable.
– Matched Pairs experiment, each case gets both treatments in random order and we look at individual differences in the response variable between the two treatments. Sometimes cases get “paired up”, such as pairing you with the person in the study most similar to you. Twins are ideal for this design, with one twin getting one treatment and the other the second treatment.
– Repeated Measures are studies where each case is measured repeatedly, typically over time. For example, if I was testing a medication to give relief to patients suffering from chronic back pain, I might be interested in both how quickly the patient obtained relief and how long the relief lasted.
5.9 Factorial Design
Now consider a more complicated two factor design, where we want to study both the dose (high/low/placebo) and method of delivery (patch/gum). We have 3 levels of dose (what level of dose is best?) and 2 levels of method (is the patch better/worse than the gum?), so we have \(3 \times 2=6\) different treatments. With a smaple of \(n=300\), we would randomly assign 50 subjects to: patch-high, patch-low, patch-placebo, gum-high, gum-low, gum-placebo. We would determine what combination of dose and method is most effective.
We could have three or more factors as well in such a factorial design.
5.10 Randomized Block Design
The designs we have considered so far are completely randomized designs, where the randomization of subjects to groups happens without any restriction. In my smoking example, the researcher would have complete control over whether a subject received the patch or the gum and what level of dose they received.
Often there are extraneous variables, or factors, of interest that we cannot control via randomization. In the smoking example, we might want to compare heavy smokers (more than one pack/day) versus light smokers (less than one pack/day). It would not be appropriate (or possible) to randomly assign subjects to smoke a certain amount of cigarettes, so we would use the technique called blocking.
If I had 120 heavy smokers and 180 light smokers in my study, looking at a patch with either a high/low/no dose, I would use the level of smoking as a block, randomly assign 40 heavy smokers to each dosage group, and randomly assign 60 light smokers to each dosage group.
Variables such as gender, race, age, etc. which cannot be randomly controlled or ethically assigned are commonly used as blocking variables. It would not be ethical to randomly assign you to smoke a pack of cigarettes a day for an experiment, as we’ve long established that smoking is not good for you. If age group was being used as a blocking variable, I cannot randomize how old you are.
5.11 Single-Blind Study
Often in medical studies, particularly clinical trials involving a placebo, a single-blind or double-blind experiment is used.
In a single-blind study, the subjects do not know if they are receiving the treatment or placebo (i.e. they do not know if they are taking real or fake medicine). Knowledge of whether you are in the treatment group or not could affect the accuracy and the integrity of the trial.
5.12 Double-Blind Study
A double-blind study extends this concept to keeping the treatment providers (such as nurses and physicians) blind as well. This is to prevent any conscious or subconsious bias in the way these providers deal with the subjects in the study.
The Physicians’ Aspirin Study was a double-blind study; the aspirin/placebo was mailed to the subjects by a third party that never directly interacted with the subjects.
5.13 Single- and Double-Blindness
It is not always desirable or possible to utilize single- or double-blindness in a study. Several years ago, I had appendicitis and had surgery to remove my appendix. I underwent laparascopic surgery, as opposed to the more invasive ‘open’ appendectomy.
There were studies published when the laparascopic technique was first developed, comparing it to the traditional open appendectomy. To my knowledge, these studies were based on observational studies and not on a randomized experiment.
Theoretically, it would be possible, if probably not ethical, to have a randomized experiment. I could have agreed to let randomization decide which tecnique was used for my surgery, therefore having a single-blind experiment. I doubt I would have agreed to such a plan, whereas I could imagine myself participating in a clinical trial for a drug (such as the aspirin study).
Double-blindness would not be possible. The surgical team would need to know what type of surgery they would perform.