Chapter 5 Science
5.1 Scientific Integrity
You have a responsibility to me, each other, the institutions that support our work, and the broader scientific community to uphold the highest standards of scientific accuracy and integrity. By being in the lab you agree to adhere to professional ethical standards. There is never an excuse for fabricating or misrepresenting data. If you have any questions, or in the unlikely event that you have concerns about a research practice you have seen in the lab, please talk to me immediately.
It is also important that you prioritize the accuracy of your work while in the lab. Unintentional errors due to inattentiveness or rushing can be extremely damaging and produce results that turn out to be incorrect. Although there is always a pressure for a high quantity of research, it is critical that everything we do is of the highest quality. Please double-check your work frequently. However, mistakes happen, and it is part of being human. Please feel secure in admitting your mistakes so we can all learn from them. There is no blame culture in the lab.
5.2 Reproducible Science
Reproducible research is research that can be exactly reproduced. This is related to replicability, in that it has to do with your ability to get the same results again, but it refers specifically to getting the same results given the same set of data. That is, if I was to take your raw data and follow the same steps that you took in your analysis, would I get exactly the same outcome? Replicability is something different: If you find an experimental effect, would I be able to find the same effect if I was to conduct an independent replication of your experiment?
I expect that all of our research will be, at minimum, reproducible (I hope that it will also be replicable!).
Conducting reproducible research is more difficult than it sounds, because it requires that you are organised and possess sufficient foresight to document each step of your research process. Extensive note-taking (i.e., as much as you can manage) will help improve the reproducibility of your work.
If you are a BSc student, you can increase reproducibility by documenting all stages of your analysis in sufficient detail such that anyone could follow your steps exactly. If using SPSS or JASP, make notes of what menu options you chose and in what order. Did you exclude any data? If so, document why. Did you transform any variales? if so, document why.
If you are an MSc student, I would be expecting you to start to explore using statistical programming languages such as R for your analysis so that you have a script of your analysis code. If you are a PhD student, I absolutely expect you to be coding your own analysis. Of couse, I am always on hand to help out if needed.
5.3 Open Science
As a lab we engage actively in the principles of Open Science. This refers to the movement to make all aspects of the scientific process open and accessible to all, for free. Working in accordance to open science practices increases the transparency, accessibility, and the impact of our work. It allows other researchers to utilise our methods, access our data, and read about our research easily.
In our work, we strive to achieve all three badges recently introduced by the Center for Open Science:
5.3.1 Open Data
We share all of our anonymised raw data online at the Open Science Framework. This allows all readers of our work to be able to access our data, either to verify our analyses (and hence conclusions) or for further independent analysis. Here is an example from a recent publication: https://osf.io/n3jgt/.
5.3.2 Open Materials
We share all of our materials used in our experimental protocol. In our lab, we use the free software PsychoPy to program our experiments. Once programmed, we share the code of the expeiment online so that anyone can download it and run our experiment on their own machine. Here is an example from a recent publication: https://osf.io/e2nx8/.
In addition to this, we also share the materials used to analyse our data. Althought this is tricky to do in SPSS (which most undergraduates will be using), it is very easy to do using R or the more user-friendly software JASP. The reason is because this analysis produces a script (i.e., some code) which allows an independent person to access our raw data, and then step through the code to see how our final results were obtained. Here is an example from a recent publication: https://osf.io/8kt56/.
If you have read the Simmons et al. (2012) — False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant paper from the “Essential Readings” chapter, you are probably of the opinion that researchers should decide ahead of time what exclusion criteria they plan to adopt and which analyses they plan to conduct on their data. An elegant solution to this is to pre-register your hypotheses, experiment plan, and data analysis procedures. Put simply, this means that you declare ahead of time what you plan to do, and only then do you go and do it.
Of course, it’s always nice to be able to prove that you did indeed plan things ahead of time. This is where online pre-registration resources such as https://aspredicted.org/ come in handy. Researchers complete an online form by answering 9 questions about the design, hypotheses, and analysis of their study. This form is then saved online and time-stamped to prove the date that the registraton occured.
Here is an example completed form:
5.4 Publication of Our Work
All work conducted in the lab should be of such a quality that it is of a publishable standard. That is, the question should be of scientific importance, the study should be designed so that the outcome will be informative in respect to the question, and the data should be of the highest quality (this is aided by good experimenter conduct & control during the running of the study). I expect you to aim for these standards during your project, whether you are a BSc student or a post-doc.
In the lab, we decide the order of authorship on a publication submission by completing the following table (recommended by the APA):
As you will see, points are achieved for different aspects of the project process. Once the table is completed, authorship order is determined by the number of points achieved. This is a fair and transparent way to determine authorship.
As principle investigator of the lab, I expect to be an author on all papers produced by lab members. But please note that I do not get automatic authorship; rather, my expectation is that you should involve me in your work to such a degree that I have earned authorship on the paper.