3 Types of research
In the last chapter we talked about the ways that research is all around us. You do it yourself almost every day in small and big ways, but we’re not really here to help you become more rigorous in your search for the best tacos in town. Looking at yelp is research, it’s just not really the type of research we’re going to talk about today. In the first section of this chapter we’ll talk about different types of research. Then we’ll describe different fields of research within social sciences, and finally we’ll discuss the steps of doing research.
3.1 Types
I’m going to break types of research into three categories, which probably don’t match the way they’re described in other textbooks.
A lot of the research you do in your daily life could probably be called secondary research. You have a question (“where are the best tacos?”, “when did the Civil War start?”, “is coffee bad for my heart?”) and so you seek an answer. That’s still research, it just doesn’t involve the collection of new data or a lot of detailed steps. Google and other search engines are incredible tools that will direct you towards an answer to your questions. What you’re doing there is secondary research, using the research of others to answer your question. Your collecting, reviewing, or synthesizing existing research, not creating new data to answer a question.
You can be better at secondary research by identifying reputable sources, accessing multiple opinions, and understanding how they produced their findings. That’s part of the research we’re talking about in this class, but only a small part. We’ll return to secondary research in a later chapter, because in order to be really good at it you have to understand how to do the original research yourself. Secondary research thus involves reviewing the research of others and is motivated by getting an answer to a question.
You can only do secondary research if someone has already researched the question you have. Another type of research people do is what could be called applied research, or research that is intended for immediate public dissemination. The idea of applied research is that there is a very clear connection between the research question and the importance of the research. Imagine you’re in a sorority and you’re planning dinner for the new pledges, so you poll everyone to ask which of three options they’d pick. You gather the data and you get an answer – the most common answer is tacos. Why do you care about the answer? Because you needed to know where to go for dinner.
Would people be open to changing the colors on the United States flag? I don’t know, and based on a quick Google search no one has answered the question. No one has polled Americans to find out whether they think red white and blue is a little dated (or maybe just too similar to France). I don’t know why we’d want to change the colors, maybe so everyone has to go buy a new flag. I can’t get an answer to that question based on secondary research though. I have to collect original data if I want an answer.
Polls are a great example of applied research. Who is currently winning the race for president? How do people feel about policies designed to slow climate change? How much trust do citizens have in their government? Those are all questions you can find written about in wonky news sources like the [NY Times], Vox, 538, or others. Why do we care? Because we want to know who is winning the race, or peoples views towards certain policies.
Take another example. A radio station wants to know the demographics of its listeners so that it can make sure the commercials they run are matched to who listens. There isn’t an esoteric question to answer, but they need to collect data to improve their business operations.
The research question and the importance are very directly linked. Thus, applied research involves original research, not just reviewing what others have done, but like secondary research it is motivated to get an answer.
The third type is the least common, but is also generally the focus of a textbook like this. Academic research is the type of research that your professors do most of the time. What differentiates public research from academic research? Public research is concerned with providing new facts, academic research is concerned with testing theories and seeking explanations.
I could spend thousands of dollars to run a new poll with a very rigorous research design to understand exactly what percentage of Americans would support new taxes. If I did that research I might be able to get it published in popular sources like the New York Times, but I could never get it published in an academic journal – and those are the papers that get professors tenure.
Why? Good polls tell us something about the world at this moment, but sciences goal, both the social and hard sciences, is to tell us something about the world beyond this moment. More accurately, it’s concerned with explaining the causes of the phenomena we see. Scientists weren’t just concerned with tracking that rocks fell from buildings, but wanted to identify the force that explained why that occurred (gravity). Similarly, social scientists aren’t just concerned with knowing what percentage of people are in poverty (although that is important) they want to identify the cause of poverty so that those conditions can be changed.
My poll might find that 46% of Americans plan to vote. What academic research is concerned with is the ‘why?’ Why did 46% say that, why did one person say yes and the other no, what does that help us to understand about the society? What we want to understand is the causes of the phenomena we see every day so that we can better understand the world of tomorrow.
Let’s say we did a study and found that 32% of elementary age children are significantly overweight. That’s good to know, it gives everyone an idea of the status of the health of children at this moment. What would be more important to know is why. If we know why 32% of children are significantly overweight and the other 68% aren’t, we can make changes that affect the future. Is it a lack of recess in schools, do children not have enough access to fruits and vegetable, are the foods they’re eating changing – understanding why is just as important as knowing the what, so that we know where to make changes.
Change is the only constant. I do most of my research on urban policy, which sort of means I study cities and the changes they undergo. If I looked at data on all of the neighborhoods in a city a decade ago and look again a decade later some would have gotten richer and others would have gotten poorer.
Change in Income in Nashville 2000 to 2018
That map might look interesting, and it might be important for people living in those communities to know. But unless we can provide an explanation for the change, we haven’t really learned anything. Neighborhoods change. People change. Demographics change. Everything changes. Are the neighborhoods shaded darker going to keep getting richer, or will there be a return to the mean and they’ll get poorer in the next decade? Did something happen that changed the fate of those neighborhoods, or was it just random decisions by a lot of different actors that lead to a new geography of the city? Those questions might seem unnecessary. Some neighborhoods are getting poorer, they need more support! Who cares about the why!? But if we’re going to try and figure out what neighborhoods will get poorer in the future, or want to change the future, we have to understand the underlying causes of those changes. That’s what academic research is trying to untangle. Not just what’s going on, but why, so that we can try to get more control of the future. Thus, academic research involves original research, like applied research, but is focused on developing theories as much as it is getting an answer to a question.
3.2 What’s theory got to do with it?
Let’s first define theory, because the way it’s used in science and the way it’s used in everyday conversation are slightly different. In everyday conversation you might hear the word theory used as the equivalent of “hunch” or “idea”: “oh, that’s just your theory”. In the sciences it means a bit more. A theory in the sciences is a well-substantiated explanation for a set of observations. A law is accepted as true by scientists, it is confirmed fact. A theory is on its way to becoming a law, it just needs more observations to be fully accepted.
The social sciences have plenty of theories, and fewer explanations that can be accepted as laws because as we discussed in the previous chapter humans just make it hard to get consistent findings.
For instance, researchers in in political science and public administration often use:
- elite theory which posits that a small minority of elites be they the wealthy or those that drive the creation of policy, holds most of the power in society even within democratic systems.
- democratic peace theory argues that democracies generally do not go to war or have armed conflicts with other democracies
- representative bureaucracy argues that governing bodies throughout society should be representative of the community they serve or govern.
Those theories and others get applied in different studies to continue testing them and refining them. For instance, I might want to study whether counties with growing Latinx/Latino/Latina populations see changes in who is elected for county wide offices. If I just do that research and report the results it could just be applied research. The public has an interest in knowing who their elected representatives are. If I use the research to test representative bureaucracy as a way of explaining my results, my research is now venturing into academic territory. The theoretical argument is all about explaining whatever I find, whether representation changes in those counties or not.
3.3 Inputs to research
The most difficult thing to accept and internalize in developing a research project is that it is iterative, not linear. We like linear processes like following a recipe. You can follow those steps and you get the end and then you have cookies. Following the research recipe isn’t that clean. It will be a lot of one step forward two steps back, which is progress, but can be frustrating.
Research values novelty. One should not spend a lot of time gathering original data in order to answer a question that has already been answered. So in order to develop a research question worth researching, it is really important to understand what has already been studied on that topic. I’ve learned this from experience, both in my own research and teaching, but you really can’t develop a good research question without doing a lot of reading.
If you’re starting to develop a research project, start with the things you care about. You’re going to spend a lot of time studying it and reading about the subject – it should be something you enjoy. Think about the things you observe in the world, the odd processes or changes you see around you. And think about the things you know a lot about. Whatever your interests are, whether they’re video games or hiking or reading, try to embed that interest in the research.
But again, you have to do a lot of reading. If I was walking down the street and someone stopped me and forced me to come up with a research question in sociology I would surely stumble and I might eventually stammer something out like “why do people leave online communities?” I don’t know much about that, it sounds kind of interesting. And online communities are a somewhat recent development (in comparison to say churches) so maybe the research will be interesting. Almost certainly not. I can guarantee a lot of research has been done on that question. That doesn’t mean I have to abandon the idea, it just means I need to start by reading all the research that has been done, and continue to refine my question. As I read thorough the literature on online group membership I’ll probably find answers to questions that never occurred to me, and as I read I might find questions that haven’t been answered yet.
A similar pattern occurred as I began my PhD. I wanted to study big important questions, and when I got to my program I was given a lot of freedom to decide what I would do research on. I decided I’d start by answering a question I constantly heard debated by policymakers: do sports stadiums create economic activity. I was going to be the researcher that answered the question. But pretty immediately I discovered that, actually, about 100 other researchers had already answered that question (it’s a definitive no, stadiums don’t create any economic activity, they’re a really bad deal for cities). The fact that the public didn’t regard it as a settled question doesn’t mean that researchers hadn’t already answered it. Did I give up? No, I kept reading the literature and I started to uncover related questions that hadn’t been answered yet. I ended up doing my dissertation on minor league baseball stadiums and their impact on the neighborhoods where they are located, which wasn’t the most important question ever but it hadn’t been answered before.
One recommendation I would make as you start reading the literature in a given area is to keep an annotated bibliography. As you read new articles to down a few sentences summarizing them - those few sentences can often be gleaned just from the abstract of the article. That way you’ll have a record of what you read, and as the project changes you can go back and wont have to search through the literature over and over. As your project begins to gain focus, you can pull the relevant articles from your annotated bibliography and begin to build out your paper. I would also recommend using a computer program like Zotero where you can save the details of articles and generate the bibliography of papers later. I don’t know the difference between MLA or APA or any system, because I make the computer do it for me. In the video below I walk through these things with a brief demonstration for anyone starting out a new research project.
One problem you will face in reading about your topic is accessing the articles that are relevant to your topic. If you go to a journals webpage, you’ll see that you can buy the article for probably $30. $30! That’s as much as a book, and you probably won’t even be sure if the article is good before buying it. There have been a lot of arguments made against these paywalls particularly given that taxpayers fund most of the research that is then sent to these journals. Researchers aren’t paid directly for the researcher they publish, which we give the copyright over to journals because it helps us to get tenure; journals then charge for people to read the research, and universities pay subscription fees so their researchers have access. It’s a circular economy, with me working for free, and journals making out like bandits. When you find an article you want to read see if you can access it through your library, but you can also check a few different websites where people post articles in order to “free” them. You can also just reach out to the author of the article and request a copy. Authors generally have the right to share the article even though they’ve given the journal the copyright over its contents, and most researchers are just happy to see someone engage with their work.
Once you’ve got a research question that’s worth studying and hasn’t been answered before, it’s time to answer it yourself. That’ll mean collecting data though, to answer that question. I should probably start by trying to figure out if data already exists that was collected by someone else first. You can do a lot of research in political science based on surveys that are posted online by Gallup or Pew. It’ll be easier to do the research if I can find the data from the start. If I can’t find the data, I’ll be faced with the choice of changing my research question to match the data that’s available or collecting my own data. Collecting your own data can be expensive and difficult, but if you’re interested in breaking new ground in your research it might be necessary.
The two steps are thus iterative. Knowing the literature on a topic will help you to develop new questions and lead you towards data necessary to answer them. But looking at data may help you to generate new questions and lead you back to the literature to understand how it can be used.
3.4 Units
As you settle on a research question, and begin to look around for data to answer it, it is good to be explicit about your unit of analysis. The unit of analysis is whatever entity of body you wish to be able to answer your question about at the end of your study. Related, there is also the unit of observation, which is whatever unit you are measuring phenomena at. The unit of analysis and observation can be the same (they often are), but they can also be different.
Your unit of analysis (and observation) can be nations, cities, neighborhoods, individuals, or any other such grouping. Let’s use a few examples. If our research question is ‘why are some nations rich?’ we can answer that by collecting data for different nations, or we could use survey data about the individuals within different countries to make a comparison. Our unit of analysis is the same (countries), but our unit of observation (countries in the first, individuals in the second) can change.
Often we can study similar subjects using different units of observation or analysis. If I want to broadly study volunteering, I could collect data to understand volunteering rates for individuals, cities, states, or countries. The unit of observation will all depend upon what data I either collect or is available. And the unit of analysis can change as well, because I might want to study predictors of why some individuals volunteer and others don’t, or I may want to understand why different countries have different rates of volunteering. Which is all to say that it’ll depend on what you’re actually studying, but you should be explicit from the start about who you are studying.
3.5 Writing research
This is sort of where I get frustrated with myself as a teacher, or more specifically frustrated at myself for you. I just laid out three types of research. One you definitely do, which is secondary research. The second, applied research, is something you’ll see in the world all around you, and there’s some chance you might end up doing in your professional life. And the third, academic research, is generally inaccessible, uncommon, and probably not something you’ll ever do in your life. And yet, here we are, in a class on research, about to start talking about how to write a highly structure research paper using a format you’ll never use again.
Why? Why am I going to do that? In part it’s so that you can better understand the field you’re studying. It’s important to understand what it means not just to be curious about politics or sociology, but to understand what it means to study that field.
Beyond that, the best way to learn something new is to break it apart. This will be a bit like learning to drive by first building a car from parts. You could just move straight to secondary research or driving the car, but for you to really understand why things are working or where a breakdown might be, you have to understand the underlying parts. Each part of your car is important for getting you from point A to B, and each step that goes into research is important to getting you from your question to the right answer.
So when you see a headline in the future like
You’ll be able to better understand how that headline was made. The people that wrote that headline were using some academic research that is being translated to the public to use as secondary research. Where did it come from though, what did the researchers do to know that was the right answer? If it’s good research, they probably followed a process like what we’re about to lay out.
3.5.1 Introduction
If you look at published research in an academic journal it will typically follow a basic structure with 5 sections. The introduction explains the subject of your research and clearly identifies your research question. It provides a bit of background about the subject the current relevancy of it or maybe recent events that heightened its importance.
A good introduction thus has two purposes. First, it should explain to the reader what the paper is about. The thing you learn as you continue to write is the value of being clear in the introduction. Tell the reader what you do in the paper, the order of information you’re going to present, and what they takeaway is going to be. There shouldn’t be any surprise endings or twists. Just give them everything up front.
You can see that is done in the article excerpted and annotated below. This is from an article I published in 2020 Evaluating the impact of short-term rental regulations on Airbnb in New Orleans. The title should generally give you an idea of what the paper is about. It’s not the best paper I’ve ever written, but it’s short and so it’s easy to identify the structure of what I’m describing there. Read the complete introduction below to see how I describe the purpose of the paper in a direct manner with some background to prepare the reader.
3.5.2 Literature Review
There are two themes that should be described in any literature review. The two aren’t separate sections, they’re both intertwined.
- What has been done in the area of your research before
- What I need to know as a reader to understand what you’re going to do.
You want to prove to the reader that you’re aware of what’s been done in the area of your research before so they’ll believe your research is informed and new. The worst feedback you can get from a reviewer is that someone has already done the same study you’re attempting to publish. With any research question you identify you’ll find that a lot has been done before, and that’s fine. But describe what has been done so that I can better understand what makes yours different or unique.
And you’re also trying to make sure I can understand the background of your topic. What are the key words you’re going to be using? How have other people studied the issue? It’s all the background I need to understand your new contribution. Imagine you’re explaining a movie to someone so that they can see the sequel with you. Who are the main characters, what was the story, where did it leave off? Get them excited to see the sequel because you’re going to finally answer the question that was left lingering by all those past researchers!
You’ll want to use the literature you review to build hypotheses for your article. A hypothesis is a statement of what you expect to find. Hypothesis: toddlers that drink milk will be taller as adults.
That statement about toddlers and milk might be right or wrong, and that’s okay! That’s what the paper is building towards, proving whether the hypothesis is correct or not. Because right or wrong, if that hypothesis hasn’t been tested before we’re learning something new. But the hypothesis will be a lot better and more reasonable if it’s based on existing literature. Why would I think milk would help toddlers grow? I’d want to base that prediction on studies about milks effect on the bodies and what non-milk drinkers might consume and anything else that would be relevant.
The literature review for the paper below is 10 paragraphs in all, but I want to just pull out two. What the literature review is trying to do is just get the reader ready. Again, it’s the “previously on…” intro to your favorite show.
3.5.3 Methodology
The introduction and literature review are all used to set up your new study. Now you can explain how you’ll do whatever new and impressive thing you’re about to do in the methodology section. Describe the data you collected and how you’ll analyze it. Essentially you want to draw the reader a road map so they can understand exactly how to redo your study. It’s similar to the chain of custody in evidence for criminal cases. How did you find this information, where did it come from, why should we trust that this data is good? You don’t just wake up and find data on the side of the road, it had be collected somehow and the way it’s collected could impact whether we trust it or not.
One of the big concerns in science is replicability. We’ll talk about that later, but the study design section is a nod towards it. If I wanted to redesign your study, recreate the experiment with similar subjects in a similar setting, how would I do that? In science, we don’t just trust your word for how you generated your results. Tell us how you generated them, so that we can consider whether there were any potential problems present.
You can start writing the methodology section as you begin the research. As you start you should have a design in mind at the beginning of any project, including what data you are going to collect, how will you collect it, and how you will analyze it. Answers to those questions might evolve as you conduct the research, but you can begin by setting it out as a research design, describing your reserach plan, and then revise it as you write the paper. Regardless, your collection and analysis should be guided by a research design, whether formally written or just a mental plan.
Where did you get the data, and what are you gonna do with it now that you have it? What’s written below may or may not make sense at this stage, but I’m including it just to illustrate the way that researchers attempt to write clearly and directly in describing their studies.
3.6 Results
Once you’ve explained how you conducted your study, you can go ahead and tell the reader what you found in the results section. Exactly what you’ll say here will differ based on what you studied, but there isn’t a lot more to say at this stage.
3.6.1 Discussion
The paper then concludes with a discussion of the significance of the results and their implications. You found something, why do we care? How does that change the field? Should policymakers react? Should scientists react? You’ll often start with an overview of what the paper found, before launching into some of the more specific takeaways you want readers to get.
3.7 Summary
This chapter has covered some of the different ways we do research, and one way (a formal paper) that we report our research. It might seem a little overwhelming to think about how to write up your research results before you even know how to do research. And that’s fine, this is something of getting a fly over of the forest before we start to look more closely at the trees. It’s good to have an idea of what your final paper might look like, before we get started. Now we can begin to get a little more detailed about how we fill in all those words between the title and the final period on a research paper.