2 Introduction to Research

2.1 What is Research

So you’re taking a class on research methods, probably because it was a required part of your program. Let’s start at the beginning then. Before we get to methods, let’s talk about research.

You do research every day. You look at pictures on Yelp to figure out where you want to go to lunch. You check the reviews for movies before deciding what to see. You figure out the best route to go shopping. None of that required a government grant or a lab to conduct, but it is all research.

Research is just the systematic study of materials and sources in order to reach conclusions. It existed long before it became codified in textbooks like this, and it’ll exist long after this class as well. Humans are a curious species. We’re always looking for new ways of doing things and ways to improve old processes. That requires research.

The core process of doing research is collecting data and looking at it for patterns. Yes, that can mean running a nationally representative survey, but anytime you look at pictures on Yelp you’re looking at data. And as you look at that data, you see whether the pattern of pictures looks appetizing or unappetizing.

For a non-food example, if you take the bus to work you might decide to catch the 8:17 instead of the 8:02 because the earlier bus is typically more crowded. You may have noticed that pattern over the course of two weeks and decided to change your behavior. The city might look at tickets for buses over months and draw the same conclusion, and so adjust their service to try and even out the number of travelers on each bus. We can gather a lot of different data, some formal some informal, and draw conclusions based on the patterns we see.

This textbook is designed to make you better at research, better at recognizing data, and better at finding patterns. It is going to help you identify information that can help you make better decisions. It will help you to gather information to answer question and it will show you what to do with data once you have it. And, just as importantly, it’ll help you be able to evaluate the research others perform to consider whether it is valid or reliable.

Doing research is more accessible now than ever before with the growth of affordable tools and the ability to communicate over the internet. These trends have culminated in the growth of what is called “citizen science”, and the ability of amateurs to contribute collectively or individually to significant scientific debates.

One of my favorite citizen scientists is Jane Jacobs, who used her observations about changes in the neighborhoods around her to upend decades of city planning orthodoxy through her book The Death and Life of Great American Cities. Citizen scientists are making contributions to conservation, biotechnology, astronomy and dozens of other fields. This book will provide you some tools to make your own contributions, if you want.

Research also takes a lot of mental power. You may want to contribute to new discoveries, but you may also just want to learn from the research others are doing. All research is not created equal, and being able to parse and interpret the findings you see reported everyday is important. Let’s take this headline from The Guardian.

What does the headline actually mean? Should we all adopt the Mediterranean diet (lots of olive oil, legumes, fruits, and vegetables, some consumption of fish, dairy, and wine, and not much non-fish meat products)? Dietitians and health scientists have been working for decades to work out exactly what foods people should consume to promote health. It would appear from that headline we have an answer. Maybe we do, this book will help you to evaluate whether you should take advice from headlines and articles like that. We’ll talk a lot about the mechanics that went into the production of that headline and others so you can be a better consumer of research in your day-to-day life.

This book is designed for introductory classes in the social sciences. Eventually the text may have expanded enough to cover advanced topics as well, in which case this sentence will hopefully be edited to reflect that fact. The book is specifically written with political science and public administration in mind, but the ideas and lessons can apply to other social sciences fields as well. Some parts of this book may be of use to students in the hard sciences, but not a lot. The methods and subject matters are quite different, but research in the social sciences is still somewhat guided by that done in the hard sciences.

2.2 Natural and Social Sciences

The social sciences look to the natural sciences (also known as hard sciences) as a guide because much of what we know about how to do good research comes from them. The natural sciences generally encompass physics, biology, astronomy, chemistry and others, and are considered ‘hard’ (not referring to their difficulty) because of their methodological rigor. Great, I’m using terms I have to define in order to define a term. Why do they have more methodological rigor? In laymen’s terms, they can be more exacting and specific in constructing a research study. They don’t have to settle for “good enough” when they’re developing their research as much as we do in the social sciences.

As a result, the hard sciences have been able to develop a set of laws about the universe that apply to any context. Take for instance Newton’s law of gravity, which states every mass exerts an attractive force on every other mass. There is a gravitational pull between you and your dog, although you’re both too small for it to be recognized. This is true everywhere. There’s less gravity on the moon, but those planets have different masses. If I drop a rock or a bowling ball or a piece of paper from a building, the law of gravity will be the same (although other variables will change the rate they fall too). Natural scientists have been able to go very very deep into minutia, understanding what is inside of an atom, because rocks don’t change their opinions between 11 and 11:05 am.

The social sciences are considered the “soft” sciences, because they cannot reach the same level of methodological rigor. That term again, methodological rigor – we’ll return to that in just a minute. The social sciences study humans, their interactions, opinions, identities, and lots else and includes sociology, anthropology, archaeology, economics, geography, management, media studies, political science, psychology and many many more.

So methodological rigor. Let’s define that a little more clearly. Methodological rigor refers to how stringent or specific one can be in structuring their research. Scientists in the hard sciences are able to create a space where they can control anything that might impact their outcome of interest. If they want to measure how fast a rock falls from a building, they can. They can do that from a building, or set it up in a lab and control the air density and humidity and wind and anything that might change the rate that a rock would fall from some height. They can isolate the exact thing they want to study with incredible precision.

Humans are not rocks. Each human is more different than two rocks, and scientists have a difficult time getting us to sit still. If we want to understand what makes humans happy, think about everything we’d have to control for to isolate happiness. We’d have to control for the individuals age, and their career. Their family history probably makes a difference. Are they married, what were their parents like? How was their childhood? Childhood traumas have been shown to have long term impacts, so we’ll need to account for that. People’s happiness changed between and within days. Have they just had a meal? People are probably less happy as they get hungrier. How are we even measuring happiness? Do we just ask people whether they’re happy or not? Ask them to use a 0-100 scale? Or maybe we measure the amount of dopamine in their system? That’s why it’s really complicated to understand something like “what makes humans happy”.

Measuring the speed of a rock dropped from a building is comparatively easier. We can standardize the rock. We can measure the humidity and wind. We don’t have to disagree about how to measure speed or what we mean by rock. There’s no cultural constructs there. A rock dropped from a building is easy. Studying humans is hard. Which isn’t to argue you should all quit and go study physics instead (well, maybe you should, but you don’t have to leave). The natural sciences can do a lot more complicated stuff because they have moved on from the speed of falling rocks and they’re working on understanding the physical reality of the source of all matter in the universe and other things that I have really no idea about Us social scientists are still sitting around a table trying to figure out what we mean by happiness so that we can get started.

And that’s why there is no law in the social sciences that can approach Newton’s law of gravity or any others. We just don’t know enough about humans to really understand them. And that’s despite being humans the whole time, and trying to study them in more or less concerted ways for 2 millenia. Why do some marriages fail and others survive? Is oatmeal healthy to eat for breakfast? How much should the public spend on schools? What is the best form of government to encourage human flourishing?

2.3 What do we know?

Social scientists are working hard on the questions listed above, but it’s difficult. There are just so many factors that affect all those outcomes of interest, and we can’t force humans to live in a lab in order to eliminate any other factors. But we do the best we can.

Just because the social sciences don’t have anything as certain as the law of gravity doesn’t mean we don’t know anything. Part of the difficulty in trusting scientific findings is understanding how science learns. Science learns through replication and revision. All of the things that make people different, their age, gender, upbringing, education, culture, etc. shape a human. We shouldn’t expect most things to apply to humans the same way as the law of gravity. And social sciences often get in trouble because of the way that news headlines love to announce new findings, which when there is a contradictory headline later undermines confidence in all of science.

What are we to make of these two headlines?

One can understand people shrugging their shoulders when scientists try to tell them that sugar is the root of bad health, or that masks reduce the spread of airborne diseases, or that a universal basic income will increase the wealth of a community.

But what such reactions miss is that no one study can be expected to generate universal truth. The law of gravity was not the result of one study (or even an apple hitting Newton on the head). It took centuries of refinement and adjustment and mistakes to eventually come to such a profound truth.

So what do we know in the social sciences? For one thing, we have a pretty good idea that spending more on schools improves student outcomes. There have been studies that have shown it doesn’t have a positive effect, and that things outside school spending are more important (stable homes, educated parents, god nutrition). But a researcher Kirabo Jackson looked at all the recent studies that looked at school spending and student outcomes and synthesized their findings and found clear evidence that yes, spending more on schools is a positive thing. That study isn’t perfect, and the studies it uses aren’t perfect, but social science is making progress at answering big questions whether we realize it or not.

2.4 Rest of the Book

In the following chapters we’ll go over the steps social scientist use in their research, the different ways they collect data to answer their questions, and what it means to answer a question in the social sciences.

So that covers what is research and a little bit about the bigger question of what is science. And it covers the big goal of this book - to make you better at discovering new facts and understanding the world around you. The rest of the book will focus on the methods part – how to develop a project, how to collect data, and what to do with that data once you have it collected.