Research Design

Researchers –

As you know, your next piece that you will be turning in is the Research Design. Please do not confuse this with what you put into your proposal. The research design is all about the methodology you will use to conduct your study – it has nothing to do with the TYPES of sources you use. For example, if your study is a case study, you will tell me what a case study is and why it is the right methodology. Then, outline your plan for the case study. If you are doing a historical or document review, you will tell me all about that methodology and why it is the right one. Some of you included part of this in your proposal – so go back to your proposal and read my notes again on the copy I uploaded!

Based on your proposals, I expect your Research Design section to be only about 2 (maybe 3) paragraphs long. That is it!

You will also want to have a subheader for Limitations of the Study. Here are some limitations you will want to consider:

• Reliability of sources/data. You are using second-hand data and you have to trust the sources are reliable – this is why your data should ONLY be scholarly or peer-reviewed.
• Classification. Much of the data for a study in the field of homeland security is classified, so you will not have access to it.
• Vast amount of data. This is true because of the various kinds of terrorism, lack of an agreed upon definition, or the number of incidents.
• Limitation of Time. This is especially true for a case study! Your study is bound by time, so you may not be considering all available information.
• Researcher Bias. Folks, we all have this! This is the unconscious bias we bring with us. Some of you saw comments about this in your proposals – we go into this thinking we know the answer – that is bias. This impacts every researcher, so I will be looking for this! You must include something about research bias!

So, here is a simple outline of what you want to be talking about:

Part 1 (1 paragraphs) – Tell me about the methodology and why it is the right methodology. What are the benefits and limitations of this methodology.

Part 2 (1 paragraphs)- Tell me how you will apply this methodology. If you are using variables from another study or have specific things you are looking for, you may need a second paragraph.

Part 3 (usually 1 paragraph) – Limitations of the Study.

I am attaching some samples of Research Design that you can review. One of the research design examples is for a comparative case study and the other for a document review (the student also conducted a risk analysis, but for those of you doing historical, policy or document reviews, the basics are here).

Remember – you must use scholarly references for this section! You cannot talk about research methodology without references! – remember these were done with earlier versions of APA!

///

Lesson Material:

Let us first review what is outlined in the End of Program Manual (EOP) for the Research Design section of your thesis:

Research Design/Methodology: Describes how the student will test the hypothesis and carry out his/her analysis. This section describes the data to be used to test the hypothesis, how the student will operationalize and collect data on his/her variables, and the analytic methods that to be used, noting potential biases and limitations to the research approach. It should include

• identification and operationalization (measurement) of variables;
• a sampling plan (i.e., study population and sampling procedures, if appropriate);
• justification of case studies used;
• data collection/sources (secondary literature, archives, interviews, surveys, etc.);
• a summary of analysis procedures (pattern-matching, etc.); and
• the limitations of study and bias discussion.

RESEARCH STRATEGIES

A review of basic terminology explains that a research methodology refers to the nature of your inquiry on a topic. The purpose of your research may range from a desire to solve a problem, explore an issue, or generate new knowledge. You will then select a research approach, which refers to whether you will address your topic through qualitative, quantitative, or a mixed approach. The tool(s) that you select within your methodology is called a research design. Qualitative research is associated with certain tools as is quantitative research. As a result of conducting your literature review and theoretical framework, you will have identified several research designs in the literature. The lesson overview contains eight examples of research designs recently used in theses.

ACADEMIC RESEARCH METHODS

There are only three academic research methods. They are qualitative, quantitative, and mixed methods. Here is a brief overview of each:

Qualitative Research Overview

Qualitative research is designed to reveal a target audience’s range of behavior and the perceptions that drive it with reference to specific topics or issues. It uses in-depth studies of small groups of people to guide and support the construction of hypotheses. The results of qualitative research are descriptive rather than predictive.

Qualitative research methods originated in the social and behavioral sciences: sociology, anthropology, and psychology. Today, qualitative methods in the area of marketing research include in-depth interviews with individuals, group discussions (from two to ten participants is typical); diary and journal exercises; and in-context observations. Sessions may be conducted in person, by telephone, via video conferencing and the Internet.

Why Qualitative Research Works

Source: http://www.qrca.org/?page=whatisqualresearch

Quantitative Research Overview

Your goal in conducting a quantitative research study is to determine the relationship between one thing [an independent variable] and another [a dependent or outcome variable] within a population.

Quantitative research designs are either descriptive [subjects usually measured once] or experimental [subjects measured before and after a treatment]. A descriptive study establishes only associations between variables; an experimental study establishes causality.

Quantitative research deals in numbers, logic, and an objective stance (emphasis added). Quantitative research focuses on numeric and unchanging data and detailed, convergent reasoning rather than divergent reasoning [i.e., the generation of a variety of ideas about a research problem in a spontaneous, free-flowing manner].

Its main characteristics are:

• The data is usually gathered using structured research instruments.
• The results are based on larger sample sizes that are representative of the population.
• The research study can usually be replicated or repeated, given its high reliability.
• Researcher has a clearly defined research question(s) to which objective answers are sought.
• All aspects of the study are carefully designed before data is collected.
• Data are in the form of numbers and statistics, often arranged in tables, charts, figures, or other non-textual forms.
• Project can be used to generalize concepts more widely, predict future results, or investigate causal relationships.
• Researcher uses tools, such as questionnaires or computer software, to collect numerical data.

The overarching aim of a quantitative research study is to classify features, count them, and construct statistical models in an attempt to explain what is observed.

Source: http://libguides.usc.edu/writingguide/quantitative

Mixed Methods Research Overview

Mixed-methods research comes out of an environment in which professional researchers have a tendency to use exclusivity in the use of either quantitative research or qualitative research approaches as described at the American Educational Research Association website. One characteristic of mixed-methods research is that this approach seeks to minimize the weaknesses and draw from the strengths of an exclusively qualitative or quantitative research methodology. Another characteristic is that this methodology recognizes the value of knowledge as constructed through qualitative means such as perceptions, as well as experience based on factual aspects of the world in which people live. Another key characteristic of the mixed-method research approach is that it rejects the dualism that sets qualitative or fact-based and quantitative or subjectively based methodologies as having value only in exclusivity from each other.

Source: http://www.ehow.com/list_7646151_characteristics-mixedmethod-design-research.html

Note: Many students say they are conducting mixed methods research when in fact, they are merely using either a qualitative or quantitative research method. This is because while most studies are either qualitative or quantitative, many include a modicum of a secondary method as well. However, in most cases, this does not constitute mixed methods research, which must include proportionate amounts of both qualitative and quantitative data to be classified as such.

KEY QUALITATIVE RESEARCH METHODS

The following section is excerpted in part from Schloss and Smith Conducting Research and Leedy and Ormrod Practical Research Planning and Design.

Qualitative research is characterized by an emphasis on describing, understanding, and explaining complex phenomena. This type of research focuses on relationships, patterns and configurations among factors; or the context in which activities occur. The focus is on understanding the full multi-dimensional, dynamic picture of the subject of study. Qualitative methods are useful not only in providing rich descriptions of complex phenomena, but in constructing or developing theories or conceptual frameworks, and in generating hypotheses to explain those phenomena (Schloss & Smith, 1999, pp. 85 – 91).

Its approaches contrast with quantitative methods that aim to divide phenomena into manageable, clearly defined pieces, or variables. Quantifying is useful for separating phenomena into distinct and workable elements of a conceptual framework. But, when we focus research on what we already know how to quantify, (that is, what can be reliably quantified), we may miss factors that are key to a real understanding of the phenomena being studied. The downside of quantification is that it does not always support (as well as qualitative methods) understanding of complex, dynamic, and multi-dimensional wholes. A mixed method approach may provide the ideal solution, using statistics to discover facts that can be explained by more nuanced description.

The selection of method, or combination of methods, must be tailored to the questions being studied and the setting for research. Typical methods include:

• Naturalistic inquiry and participant observation: Naturalistic inquiry, or ethnography, has its roots in anthropology and sociology and involves long-term exposure to a setting or a group of people. Extensive use of unstructured observations and conversations documented by detailed field notes form the basis for this type of research, often considered the purest form of qualitative research. Naturalistic inquiry is used when situations are unique or complex, when the level of uncertainty about the questions to ask is high and when there is little or no theory to direct the investigator.
• Case study research: Case studies are the preferred strategy when ‘how’ or ‘why’ questions are being posed, when the investigator has little control over events, and when the focus is on a contemporary phenomenon within some real-life context. The case study is especially appropriate when the boundaries between phenomenon and context are not clearly evident. The case study copes with the technically distinctive situation in which there will be many more variables of interest than data points, and as one result relies on multiple sources of evidence, with data needing to converge in a triangulating fashion.
• Archival Research: Archival research involves investigators carrying out primary research that is unobtrusive in nature. With this kind of method, the investigator is working to collect data that was created for reasons beyond the research project (Berg & Lune, 2012). Archival research is inclusive of information contained in both public and private archives and may include documents obtained from schools, medical facilities, crime reports, birth and death records, etc. Archival research takes place at both regional and national sites throughout the world. The National Archives serves as an excellent starting point for researchers as it provides insight on archive locations and the types of records they might hold. With this kind of research, curious individuals would be good to make use of the archivist on-site. To be sure these knowledgeable individuals will happily provide researchers with information on what archives have yet to be reviewed in the hopes that the materials are put to good use. When visiting archives its best to call ahead and plan your visit so the materials can be prepared for you prior to your arrival. Asking for finding aids and archive lists in advance of your arrival can help speed up your on-site experience. In addition to visiting national or international archives some archival material can also be found online. Some reputable sources include the UK’s National Archives, Internet History Sourcebooks Project at Fordham University, and the New York Times Article Archive.
• Structured observations of meetings and events: This involves attending meetings of the group that you wish to research. This can also be extended to observation of individuals in their daily work routine or on special tasks. The purpose of observing is to learn what is going on at the meeting and witness the group dynamic in process. This can be a rich information source as it can give researchers insight into the group.
• Social Historical Research: Social historical research as the name implies focuses on examining elements of history in context (Berg & Lune, 2012). Ultimately, it involves looking at data made up of records and historical accounts in order to determine what happened during a specific time period. Based on the facts discovered the researchers then work to develop theoretical explanations surrounding historical events (Berg & Lune, 2012).
• Content analysis of documents: This is a non-intrusive form of research. This involves reviewing documents, memos or other pieces of written information for content and themes. By examining the written word, the researcher is studying one type of communication that occurs in the selected sample. During the conduct of content analysis, the research works to reduce the content of the document down to a set of categories that are representative of the research interest (Singleton & Straits, 2010). Types of documents that can be coded include written documents, new articles, speeches, interviews, biographical information, memos, policies, and even photographs, video and audio recordings. When conducting content analysis, researchers develop codes and attach these to notes or transcript information. There are a number of qualitative research software options that researchers can use to help them code and draw connections between their data points. After the codes are categorized researchers are then able to see and evaluate patterns and compare these to commonly held theories or prior research findings. For an example of how content analysis can be carried out see the GAO’s “Content Analysis: A Methodology for Structuring and Analyzing Written Material.”
• Collection and analysis of other archival, administrative and performance data: This method also is non-intrusive. Information that has been previously collected, or secondary data, is reviewed to gain a better understanding into the topic. This information is historical provides insight into the past.
• Focus groups: Focus groups usually explore specific issues. The focus group brings together individuals chosen to meet a specific profile. They may be homogenous along some dimensions and heterogeneous along others and a structured, yet informal, setting is used to explore a limited number of questions. Focus groups, unlike individual interviews, provide the added dimension of the interactions among members. Focus groups are often combined with more quantitative approaches such as surveys that can be administered at different points in the group discussion and even used as grist for additional discussion.
• Cognitive interviews: Cognitive interviews are typically used in survey development. One-to-one interviews are conducted (with people meeting the criteria for completing a particular survey) as the individuals complete the instrument being tested. This method helps investigators understand how people perceive and interpret language and their own experiences as they refine the survey instruments.
• Mail and telephone surveys: Mail and telephone surveys are a method of collecting information by sending surveys via email or postal mail. Participants return completed forms to the researcher or an outside vendor. Surveys may ask respondents to rate items on a scale (a Likert scale of 1-5 is such an example). Some surveys also allow respondents to write their feelings or attitudes about a particular event or to elaborate in more detail on an item, or to express suggestions (Leedy & Ormrod, 2005, pp. 94 – 97).

KEY QUANTITATIVE METHODS

When carrying out quantitative research there are a number of steps involved. Like with qualitative research we first start off with theory. Next, we work to deduce a hypothesis based on the theory. It is important to note that some quantitative research does not require the specification of a hypothesis. Instead, “the theory acts loosely as a set of concerns in relation to which the social researcher collects data” typically, hypotheses are specified in experimental research as this involve hypothesis testing (Bryman, 2012, p. 161). From here, we then move into the research design phase in the research process. This is where we work to operationalize (or measure) our concepts. Once this has been determined we then work to determine what research sites we plan to use and also determine our research subjects/respondents. After the subjects have been determined the investigator can then administer the research instruments and collect their data. After the data has been collected the investigator then moves into the data processing and analysis phases. Finally, they are then able to write up their findings and conclusions.

Survey Research

One of the most popular ways to collect data is through a survey. Think about how often you are asked to take a survey…when you leave a store, when you purchase something online, after talking to your cell phone carrier on the phone they ask if you are willing to take a survey, etc. They are everywhere.

Why? The general features are easy to work with.

• They are versatile – Surveys can be designed to study almost any social issue and we can use them to learn about people, organizations, criminal behavior, etc.
• They are efficient. Data can easily be collected from a large number of people relatively inexpensively and quickly.
• Often times they can be generalizable as we are usually able to get information from a representative sample of a large population (Bachman & Schutt, 2012).

Please note that probably the most important thing to know about a survey is the development of the survey questions is very important. This will affect the reliability and validity of any data collected. Second to this is how the interview or survey is administered; this too is important and relates directly to the reliability and validity of the survey results. Keep in mind that survey questions are answered as part of a questionnaire, not in isolation from other questions. The context created by the questionnaire and how the questionnaire is administered affects how individual questions are interpreted; and whether they are even answered. Finally, we must give very careful attention to design of the questionnaire as a whole, as well as to each individual question that it includes.

Secondary Data Analysis

Another method that we can pull from to carry out quantitative research is through secondary data analysis. There is a tremendous amount of research done each year that makes use of this method. This type of research uses pre-existing data to perform the study in question rather than going out and collecting new data for the inquiry. In effect, the data have already been collected but they will be analyzed in a different way or can be used to answer a different research question than was originally intended by those who collected the data in the first place. This happens all of the time and it is a great way to do research.

The major types of secondary data analysis used in social science research include surveys; official statistics; official records; and historical documents.

The most common sources of secondary data are social science survey data collected in studies funded by federal and state government. Other common sources include official records maintained by government agencies for administrative, rather than research, purposes – for example, police department arrest data. Often what happens is a government agency – like the National Institute of Justice – will fund data collection and research. After the data is collected and several reports are written, the government agency then asks for a clean set of the data collected; they then hold this for other researchers that may want to analyze the data. Other researchers like you and I can request a portion of the data to answer certain research questions. In fact, the government agency encourages use of the data to help with our understanding of various social science and social policy questions. They will even sometimes fund researchers’ projects that analyze secondary data.

As you come to work with your data rather than just looking at what you have, you should also be thinking about what it means in relation to the theory or theories you are using in your research. Data analysis is something that should be looked at in conjunction to a researcher’s hypothesis (if carrying out an experimental study) or theoretical framework (Singleton & Straits, 2010). These frameworks “guide the collection and analysis of data, indicating what variables should be measured a statistically controlled” (Singleton & Straits, 2010, p. 53). The analysis portion of research then helps provide validation to current theoretical holdings, provides insight on theoretical gaps, and helps determine new areas of research that should be explored.

After researchers complete their research projects many are typically required to archive their work for preservation, recall, and sharing. While it is always good to save ones work, researchers that have created large data sets often come to share their data with others in the spirit of collaboration. The Global Terrorism Database at the University of Maryland, RAND Database of Worldwide Terrorism Incidents, Data.gov, GIS Geography, Human Trafficking Indicators, USDP/PRIO Armed Conflict Dataset are all examples of data sets that have been shared by researchers. This information is something that anyone can use in research looking to use secondary or existing data sets.

References

This information was excerpted in part from Center for Organization, Leadership, and Management Research (COLMR), a Center of Excellence of the VA Health Services Research and Development Service (HSR&D). (n.d.). Qualitative Research Approaches. Retrieved from http://www.colmr.research.va.gov/mgmt_research_in_…

Bachman, R. & Schutt, R. (2012). Fundamentals of Research in Criminology and Criminal Justice. Sage.

Berg, B. & Lune, H. (2012). Qualitative Research Methods for the Social Sciences. 8th ed. Pearson.

Bryman, A. (2012). Social Research Methods 4th ed. Oxford University Press.

Cottam, M. (1994). Images and Intervention: U.S. Policies in Latin America. University of Pittsburg Press.

Prunckun, H. (2015). Scientific Methods of Inquiry for Intelligence Analysis. 2nd ed. Lanham, Maryland: Roman & Littlefield.

Schloss, P.J. & Smith, M.A. (1999). Conducting Research. Saddle River, NJ: Prentice.

Singleton, R.A. & Straits, B.C. (2010). Approaches to Social Research. 5th ed. Oxford University Press.