1.9 Units of observation and units of analysis
Units of observation and units of analysis are important, but similar, concepts that need to be distinguished.
Lemma 1.10 (Designs) Consider this RQ:
What is the problem with comparing 100 hair strands from one blond man, to 100 hair strands from one brunet man?In Australian 20-something men, is the average thickness of head hair strands the same for blonds and brunets (Vaughn et al. 2009)?
In this study, only one man of each hair colour is represented. There are 200 observations, but only two people are compared, so little is learnt about 20-something men in general.
We learn a lot about two men specifically. The Population is represented by just two men... so we don't learn much about the population of men in general.
In this study, each individual hair is a unit of observation: the hair strands are what must be measured to obtain 'thickness of head hair strands'.
But each blond hair comes from the same man, so each of those hairs have essentially lived their life together: They are washed at the same time, with the same shampoo, exposed to the same amount of sunlight and exercise, share genetics, etc. However, different people do their own thing and have their own genetics.
A similar, but different, concept is the unit of analysis.
In the hair-thickness study each person is a unit of analysis. Importantly, the size of sample in the study is the number units of analysis; so here, there are only two examples of the population in the study. The size of the sample is just two.
Example 1.25 (Units of analysis) In the hair-strand study, each hair strand is a unit of observation: measurements of hair strand thickness are taken from individual hair strands.
However, the unit of analysis is the person: the hair strands from each man share a lot in common. The men themselves would share little in common, and we are interested in comparing men.
Example 1.26 (Units of analysis) Consider a study comparing the percentage of females and males wearing sunglasses at a specific beach.
People in a group at the beach will probably not be operating 'independently': groups of people tend to behave similarly (but perhaps not identically). For example, a couple will often both be either wearing or not wearing sunglasses.
The researchers have two options; they could either
Use the people groups as the unit of analysis (some of which will be groups of one), and record data from just one person in any group.
Ideally, the researchers would specify before-hand which group member from which to take data (e.g., the person closest to the researchers when the group is spotted).
Alternatively, the researchers may decide not to use data from groups at all, and only gather data from individuals.
Example 1.27 (Units of analysis) A study compares two brands of car tyres. Four tyres of Brand A are allocated to each of Cars 1--5. Four tyres of Brand B are allocated to each of Cars 6--10.
After 12 months, the amount of wear is recorded on each tyre. The unit of observation is the tyre: the amount of wear is measured on each tyre.
The unit of analysis is the car: the brand of tyre is allocated to the car and all wheels on the car get the same tyre. Tyres on any one car 'live their life together': They all are exposed to the same day-to-day use, the same drivers, have driven very similar distances, under the same conditions, etc.A report on the Spectrum website reported the following:
Seven years ago, Peter Kind [...] was reading a study about fragile X syndrome, a developmental condition characterized by severe intellectual disability and, often, autism [...] Kind was surprised when he noticed a potentially serious statistical flaw.
The research team had looked at 10 neurons from each of the 16 mice in the experiment, a practice that in itself was unproblematic. But in the statistical analysis, the researchers had analyzed each neuron as if it were an independent [individual observation]. That gave them 160 data points to work with, 10 times the number of mice in the experiment.
``The question is, are two neurons in the brain of the same animal truly independent data points? The answer is no,'' Kind says.
--- Spectrum report
There were 16 units of analysis (mice) so the sample size is 16 mice, but the authors treated the \(16\times 10 = 160\) neurons as the sample size. The 10 neurons from each mouse have a lot in common: the genetic information was the same for all 10 neurons from each mouse.
A total of 160 neurons from 16 mice is very different to a study of 160 neurons from 160 genetically-different mice.The units of observation and units of analysis may be the same, and often are the same. However, they are sometimes different too, and it is crucial to be able to identify these situations. Importantly, studies compare units of analysis, not units of observation.
Example 1.28 (Units of analysis) A study compared two school physical activity (PA) programs. Each of 44 children (whose parents agreed for their children to participate in the study) were allocated to one of two PA programs. The improvement in children's fitness was measured for every student in the study after six months.
The units of observation are the individual students, as the the fitness measurements are taken from the students. The units of analysis are also the individual students, as the PA program was allocated to each student individually.Lemma 1.11 (Units of analysis and observation) A study compared two school physical activity (PA) programs. Program 1 was allocated to be used at School A, while Program 2 was allocated to School B. In each school, 22 children (with parental consent) were observed and the improvement in children's fitness was measured for each student after six months.
What are the units of analysis and unit of observation?The following short video may help explain some of these concepts: