30 Experiment
Things to consider:
-
Validity
- statistical conclusion validity
- internal validity
- construct validity: whether you operationalize your construct correctly
- external validity: generalization
-
Heterogeneity of treatment effects
- average (aggregate) effect can be artificially constructed.
Mechanisms
More things to consider
- cost
- control
- realism
- ethics
The Principles of Humane Experimental Technique by Russell and Burch (1959): 3Rs
- Replace: with natural (less invasive)
- Refine: less harmful
- Reduce: min number of participants
Before running experiments, to make sure that you did not tamper with the data, or hypotheses (i.e., change them after the fact), you should always pre-register your research. A website that allows you to do this and automatically provides you with time stamp is aspredicted
This way then you can show to reviewers that you did hypothesize before running analysis.
To combat replication crisis in psychology, we can advocate for open research in which researchers public their data and analyses. Popular websites include:
To see whether authors temper with their data, we can use p-curve to inspect. 1 2 3 4 5
Interaction Effects Need Interaction Controls (basically, you can’t just control for main effects).