Selecting an Appropriate Research Design for the Study: Qualitative, Quantitative or Mixed Methods

Introduction

Educational research requires the individual (or group) conducting a study to determine how the collected data should best be understood: from a more traditional (quantitative) perspective, or a more interpretive (qualitative) mindset. This is, in fact, one of the first choices facing any researcher prior to study design (Cohen, Manion & Morrison, 2007). Choosing a quantitative method of study often (though certainly not always) indicates that the researcher is convinced that social sciences consist of behaviors controlled by reliable, natural laws that apply consistently, similar to those found in the natural sciences (Bogdan & Biklen, 2007). On the other hand, a researcher’s selection of qualitative methods shapes the research in a way that allows for differences in behavior when examining phenomena in the social world (Creswell, 2009). These seemingly contradictory views help shape our understanding of the world and both possess strengths and weaknesses (Creswell, 2009) that must be understood clearly before selecting one or the other as the method used in the design of a dissertation.

Based on the summary of quantitative and qualitative methods provided in the last paragraph, it may appear that the two methods share nothing in common, and some authorities express that opinion (see, e.g., Gall, Borg & Gall, 2007). In effect, many believe that nearly everything in the world—including behavior—is explainable based on objective reality. According to this belief, reality can be described numerically and studied statistically (Creswell, 2009). In contrast, proponents of qualitative methods assert that there is no single ‘reality’ in social sciences, but individuals develop their own interpretations of what is real and act accordingly (Gall et al., 2007, p. 28). This does seem to indicate a clear distinction between the collection of data that are analyzed statistically and data that are interpreted from a much more subjective point of view. However, it is also true that an analysis of qualitative data can provide the basis for identifying valid themes useful in other settings (Bogdan & Biklen, 2007).

It is certainly true, as described by Gall et al. (2007), that “qualitative methods play a discovery role, while quantitative research plays a confirmatory role” (p. 29). However, this distinction indicates that each method simply provides a researcher with a different set of tools to examine (or identify) a theme in the social environment from two different perspectives (Bogdan & Biklen, 2007). In order to validate such themes or determine if they apply to the entire class, quantitative methods may be used. Significantly, while many authorities believe that the two main research methods share nothing in common, Creswell (2009) insisted that the use of qualitative and quantitative methods is not necessarily mutually exclusive, since a combination of both methods may be desirable in some instances. Combining the two methods is called, logically, mixed methods (Creswell, 2009). Regardless of what methods are used, or whether the goal is to discover or confirm, it is vital that research results are based on data that are reliable and valid.

This paper briefly reviews the three methods used in education research—quantitative, qualitative, and mixed methods—and explains why a qualitative method was selected for use in the proposed dissertation. Additionally, two previously written qualitative dissertations are reviewed to identify strengths in those two presentations. These dissertations are similar to the proposed dissertation design (while not necessarily dealing with the exact topic proposed) with the objective of showing how qualitative research methods are preferred in the context of the proposed study. Finally, an overview of the proposed dissertation is presented, including a justification for selecting a qualitative method for the study.

Defining the Three Approaches

Qualitative

All forms of research are based on a specific set of assumptions (Creswell, 2009). In the case of qualitative research, it is assumed that reality is subjective and based on a constructivist point of view, wherein personal interpretations of experiences and the surrounding environment defines meaning (Gall et al., 2007). For the qualitative researcher, broad generalizations are not possible, since each case may vary based on individual or student grouping constructs that are unique (Bogdan & Biklen, 2007). Therefore, it is assumed prior to the collection of data that the results of the study will not establish one all-encompassing ‘reality’ applicable in every other setting.

Validity and Reliability

According to Creswell (2009), reliability does not play an important role in qualitative research, but it is still important to develop conclusions that are generalizable. Without the ability to apply the results in some way to other settings, there is little value in the research. Unlike quantitative research—in which numerical data and analysis is easily replicated by another researcher—it is understood that two different qualitative researchers may obtain different results (Gay et al., 2006). Consequently, many qualitative researchers utilize multiple data sources (or triangulate) in order to maintain consistency (Creswell, 2009; Lichtman, 2013).

Data Collection

Unlike quantitative data, which is termed ‘hard’, Bogdan and Biklen (2007) portray qualitative data as ‘soft’. This is because information that contains detailed descriptions of a particular behavior or activity is not readily translated into numerical or statistical terms. The authors (Bogdan & Biklen) described interviews as the most valuable process to the qualitative researcher, as it produces, customarily, valuable data, and interviews are also very adaptable (unlike typically rigid questionnaires or surveys) (see also Horrocks & King, 2010). That being the case, the adaptability of this method lends itself to adjusting the process as it is ongoing, a benefit that does not exist in quantitative research. Thus, the full complexity of the phenomenon being studied is revealed. Specifically, data collection may take the form of interviews, focus groups, reflective journals, field notes, or observations (Creswell, 2009).

Data Analysis

The process of data analysis in qualitative research was described by Bogdan and Biklen (2007) as similar to a funnel. The start of the research process (like the top of a funnel) is very wide—allowing for a variety of possibilities. Then, as the study proceeds, the process gradually becomes more and more focused and eventually becomes very specific at the end (like the bottom of the funnel) (Lichtman, 2013). This is very different from quantitative research, which begins the process with a very limited focus (e.g., a hypothesis) and merely attempts to prove or disprove that hypothesis (Creswell, 2009). It is impossible to develop a limiting hypothesis prior to beginning a qualitative research process since, as Bogdan and Biklen (2007) explained, the researcher “...does not assume that enough is known to recognize important concerns before undertaking the research” (p. 7). As the research develops, then, the focus narrows until the theme or themes are clearly identified.

Ethical Issues

Ethical issues are often more of a concern in qualitative studies simply because the researcher has more direct contact with individuals or groups (Flick, 2009). It is vital to protect the confidentiality of individuals as well as to assure them that they will not be negatively affected as a result of their participation in the study (Gay, Mills & Airasian, 2006). It is also critical for the researcher to keep personal views to him or herself and respect the opinions of all participants (Horrocks & King, 2010). Since studies often involve diverse groups of individuals, any hint of discrimination in multicultural education must be avoided.

Quantitative

Based on a positivist perspective, quantitative methods are utilized “to describe current conditions, investigate relationships, and study cause-effect phenomena” (Gay et al., 2006, p. 10). Unlike the qualitative researcher, there is no attempt to understand the subjective feelings of individuals or groups, but he or she is focused solely on determining either underlying causes of a phenomenon or specific facts (Creswell, 2009). Quantitative research assumes that any phenomenon can be studied and explained using scientific methods (Gall et al., 2007). Regardless of the subject under study, the quantitative researcher believes that it can be measured and described in quantifiable terms.

Validity and Reliability

For quantitative studies, validity and reliability are very important elements which are used to determine the quality of the research (Gay et al., 2006). There are two parts of validity in any quantitative study—external and internal—used to demonstrate that the researcher actually measured what he set out to measure (Creswell, 2009). The reliability of a study applies to the ability to reproduce the results of the research (Cohen et al., 2007).

Data Collection

Data is collected in quantitative research largely by means of surveys or questionnaires which are administered in a variety of ways, including online (Bogdan & Biklen, 2007). In some cases, a researcher will develop their own, original questionnaires or surveys designed specifically to address the research variables; but at other times, existing questionnaires or research instruments are utilized (Creswell, 2009). In quantitative research, closed-ended questions are typically employed, with the goal of restricting the number of possible answers (which enables more effective measurement) (Gay et al., 2006).

Data Analysis

Scientific methods drawn from the natural science field are used to analyze collected data which result in broad generalizations or forecasts that are applied to a particular phenomenon (Creswell, 2009). More precisely, statistical analysis is employed to reach an objective conclusion.

Ethical Issues

Although quantitative researchers may not have the level of direct contact that is required in qualitative studies, Cohen et al. (2007) reminded such researchers that ethical considerations are still important. Whenever humans are included in research, their dignity, anonymity, and sense of self must be protected. To accomplish this, Christians (2000) introduced four guiding principles for all studies:

(1) informed consent (subjects must agree voluntarily to participate and this agreement must be based on full and open information); (2) deception (deliberate misrepresentation is forbidden); (3) privacy and confidentiality (primarily by safeguarding against unwanted exposure; data can be made public only behind a shield of anonymity; and no-one deserves harm or embarrassment as a result of insensitive research practices); and (4) accuracy. (Cited in Mustafa, 2011, p. 27)

While Christians’ guidelines were written with qualitative research in mind (because that method involves more human contact) it is also applicable to quantitative studies.

Mixed Methods

Research that employs mixed methods may be structured from a standpoint of testing a theory or developing a new theory (Creswell, 2009). In fact, the purpose of the study may determine which of these methods receive greater emphasis. Regardless of the motivation, a researcher typically uses mixed methods as a way to direct their “theoretical lens” (Creswell, 2009) in a more effective manner.

Validity and Reliability

In mixed methods studies, reliability assesses whether or not the results can be repeated by another researcher. Validity, as in the case of purely quantitative research, is used to determine if the study actually measured what was intended (Creswell, 2009; Gay et al., 2006).

Data Collection

Mixed methods’ data collection utilizes the techniques described for qualitative and quantitative methods (Cohen et al., 2007). The use of both qualitative and quantitative data collection methods are used when the researcher considers the practice appropriate, such as when either method alone is insufficient to accomplish a study’s purpose. Another factor to consider is the order in which data are collected and this is determined by the priority assigned to each by the researcher (Creswell, 2009).

Data Analysis

Data analysis for mixed methods simply consists of a combination of various elements utilized for quantitative and qualitative methods (Creswell, 2009). This may mean using a survey to collect quantitative data while also including interviews for qualitative assessments. But, as Creswell noted, at times a researcher may ‘quantify’ the qualitative data in order to compare the two data sets.

Ethical Issues

Similar to the other two methods of research, mixed methods require a careful consideration of ethical issues (Flick, 2009). Indeed, Flick noted that respect for individuals will prevent a research from causing any harm (intentional or unintentional) to participants. One of the overriding principles in research using humans is informed consent, which means giving potential participants any detail of the study that directly impacts them (Lichtman, 2013). This allows any potential participant to exit the study at any point in the process, especially if they believe some negative outcome may result from participation (Creswell, 2009).

Overview of Two Selected Qualitative Studies

The first dissertation reviewed is titled Perceptions of Eighth Grade Students on the Transition from Preadolescence to Adolescence in a Selected Middle School (Bush, 2013). This qualitative study examined the way adolescent students viewed their transition from elementary school to middle school, or preadolescence into adolescence. More precisely, the purpose of the study was to investigate any changes (positive or negative) in social development, motivation, or academic achievement. Assessments were made for all students without demographic restrictions or limitations so that a picture of both male and female students could be acquired. Bush (2013) based the research design on Rubin and Rubin (2011) who recommended qualitative research methods for educational research studies.

Bush (2013) utilized an ethnographic case study methodology to collect the data from student and teacher participants. As described by Creswell (2009), an ethnographic study includes in-depth interviews in addition to observing the participants over the course of the study in order to collect data. One of the strengths of case study research is the fact that the researcher is able to refine and fine-tune the preliminary research plans as the process unfolds (Creswell, 2009). In contrast to a strictly experimental format, the researcher does not influence or maneuver the setting, but simply attempts to observe and understand phenomena in natural settings. Additionally, the qualitative evidence allows for more inductive reasoning which allowed the researcher to infer from the results how students may react in similar settings. Lichtman (2013) also stated that qualitative evidence collected from case studies is appealing since it may provide data that is so well documented that it allows the development of new theories related to the topic being studied. In the case of the Bush (2013) dissertation, this topic is changes in student behavior during transition from elementary to middle school.

According to the study’s author (Bush, 2013), the case study method was deemed to be most appropriate for the research since it is one of the most effect methods used in research when the goal is to understand a complex issue more fully or, in some cases, to build upon existing knowledge. While this case study utilized only a limited dataset, it was nonetheless able to provide a detailed analysis of important contextual information. The case study method has been successfully used for quite some time by researchers in various disciplines, particularly by social scientists (Horrocks & King, 2010). Its use in the field of education research is particularly useful, since it operates directly in the environment being studied and it also uses a variety of sources in the process (Creswell, 2009). In addition, according to Bush (2013) the time and resources needed for other types of research were not considered available to complete the study. This is always a factor to consider when designing a study.

Bush (2013) set out to determine what changes (if any) are made by students—in particular, the students’ own perception—transitioning from elementary school to middle school in one school district in Louisiana. An additional element included was tracking grades over the period of the study. The total number of participants used for this study was six, including males and females. Students were selected from both successful as well as unsuccessful students (determined by grade point average as assessed by the researcher). Within a parental communication plan, parents had to sign agreements to allow their children to participate and they were thoroughly informed of all activities and questions that involved their children. The study was carried out over a two-year period, from the 2010 to the 2012 school year.

The case study process involved multiple elements. The interview procedure for the Bush (2013) study consisted of private, one-on-one interviews with each student which lasted approximately 25-40 minutes. The questions asked of the students were open-ended and included topics such as beliefs, values and disposition toward moving on to middle school. Observations were also included as part of the study and this involved a minimum of five hours of observation of each student performing their regular daily schedule and routine in school (including classroom as well as in social settings outside the classroom). Bush (2013) made regular notes in a journal to maintain an accurate record of these events.

Additionally, Bush (2013) conducted interviews with one middle school and one elementary school teacher familiar with the students involved in the study. The purpose of these two interviews was to collect data related to teachers’ perceptions of the students’ transition and compare their assessment with those of the students. Adding teachers’ comments was designed to provide another layer of detail to the data which helped to triangulate the information collection as a whole (Bogdan & Biklen, 2007). Overall, the method used and the structure of this study (Bush, 2013) provide many useful ideas that may be applied to the future proposed dissertation.

Study Two

The second dissertation reviewed is called Characteristics of Classroom Contexts, Self-Processes, Engagement, and Achievement across the Transition from Middle School to High School, and was written by Tomback (2007). This researcher also selected qualitative methodology, in particular the case study method. Case studies are widely considered one of the best tools for conducting research in this type of setting (Cohen et al., 2007). While the period of transition in this study is later in the students’ life than the study by Bush (2013), the reason for selecting this dissertation for review was its use of similar methods and its applicability to the future proposed study. At the outset, Tomback’s (2007) study included a much larger number of participants than included in the Bush (2013) study. Specifically, Tomback (2007) examined the transitions of 160 participating students, over a one-year period (2003-2004).

However, many of the areas for which data was collected were very similar to the study conducted by Bush (2013). For example, this study assessed how these students felt about their own changed behavior in addition to their feelings toward their teacher's quality (whether they were supportive, etc.). Tomback (2007) utilized a framework for studies related to student transition developed by Connell and Wellborn (1991). This model especially incorporates student opinions regarding how well (or poorly) they are supported by their teachers—before, during, and after transition. Furthermore, Connell and Wellborn (1991) included specific tools to qualitatively measure student feelings through the use of narratives. As in the Bush (2013) study, Tomback (2007) realized how valuable it is to obtain students’ opinions in their own words through direct interviews. Both researchers understood that the results of their studies could be applied in other settings and to other students approaching a transitioning period in their academic lives. These were not simply frivolous academic exercises, but were intended to provide a future framework to benefit other schools, teachers, and students.

Similar to Bush (2013), Tomback (2007) included students from the higher performing academic groups as well as those performing at a lower level. Classroom teachers were also included in the interview process, which added depth to the data collected for analysis.

Plans for the Dissertation

The tentative topic of the proposed dissertation is: Perceptions of Middle School Female Students on the Issues of Transition from Elementary to Middle School in a Title I Middle School. This topic was selected as the result of consistent feedback regarding the changes observed in female students transitioning from elementary to middle school. Such feedback comes from parents as well as educators, and both groups are—in their own way and for different reasons—concerned with the perceived changes they notice during this time in a student’s academic life. The underlying purpose for studying this topic is to provide additional resources to all stakeholders involved in this environment: female students, parents, and educators.

Based on the review of the two included dissertations, it is clear that qualitative methods provide a firm framework for collecting data as well as obtaining comments directly from the students. In particular, the ethnographic case study method employed by Bush (2013) provides a strong motivation for application of this methodology in the future dissertation. Moreover, the use of qualitative methods will allow the proposed dissertation to provide a more in-depth, detailed description of female students’ personal experiences and feelings about middle school transition. As in the case of both the Bush (2013) and Tomback (2007) studies, case study methods utilizing interviews and observations will be utilized to examine Perceptions of Middle School Female Students on the Issues of Transition from Elementary to Middle School in a Title I Middle School.

Rationale/Justification for Qualitative Method

The development of qualitative research methods came about as a result of the desire on the part of researchers to investigate the complex interactions between humans that form the basis for certain phenomena (Creswell, 2009). Thus, qualitative methods are used to look at all aspects of a given event, resulting in an inclusiveness implying that each situation is unique. As a result, many variables are considered for each case studied, including all applicable influences observed. By developing a clearer understanding of a particular phenomenon, qualitative researchers hope to be able to understand related phenomena.

Interestingly, Bogdan and Biklen (2007) explained qualitative methods as an attempt to “objectively study the subjective states of their subjects” (p. 33). Therefore, the benefit of qualitative research is that it allows a much deeper understanding of a phenomenon than is possible through the relatively restricted statistical method (Creswell, 2009). That being said, it is also a challenge for qualitative researchers to maintain a state of objectivity while, at the same time, being immersed in the completely subjective data they are collecting (Bogdan & Biklen, 2007).

According to Creswell (2009), qualitative research typically involves attempting to discover the meaning of a particular phenomenon by interviewing participants or by observing their behavior. A qualitative approach features case studies, participant observation, open interviews, etc. (Gay et al., 2006). In effect, a quantitative approach, as proposed for this study, is more objective, and closely related to traditional empirical research in its logical positivism, while a qualitative methodology is often subjective, based on a more phenomenological tradition that originated in the social sciences (Lichtman, 2013).

One of the strengths of qualitative research is the fact that the researcher is able to refine and fine-tune the preliminary research plans as the process unfolds. In contrast to a strictly experimental format, the qualitative researcher does not influence or manipulate the setting, but simply attempts to observe and understand phenomena in natural settings (Gay et al., 2006). Importantly, as the qualitative study progressed, the graduate students as a group may seem receptive, but individuals may have reservations about participating or not feel that the study is even important. It will be essential, following the collection of the qualitative data (which would be implemented by means of interviews using mostly open-ended questions), to construct a meaning that can be beneficial to others. Qualitative research and data analysis is therefore unique, since meaning can vary even within the group(s) being studied (Maxwell, 2013). Unlike quantitative analysis of data, qualitative analysis is decidedly intuitive (Creswell, 2009). It is thus vital that the feelings and opinions of the students who will be participating are placed in the proper context, and are not assessed based on preconceived assumptions.

In trying to determine the most effective means to pursue the objectives of the proposed study, it was decided—partly because of budgetary constraints and partly due to the benefits of qualitative methodology itself—to undertake an ethnographical case study of one Title 1 Middle School. This method was selected for three primary reasons: 1) it provides a strategic way to explore the complex problems involved in the proposed study; 2) it permits the use of both interviews with the students as well as in-depth observations of the several aspects of the dynamics of school life; and 3) it allows the use of a variety of data-gathering methods, which is an essential element for the proposed inquiry. Regardless of motivation, it is critical for a researcher producing a case study to conduct it in such a way that the results can be easily communicated to the reader. There is a need for anyone reading the research to be able to understand, with little effort, what the study attempts to show and what the conclusions are.

Similar to the Bush (2013) study, the data collection method for this qualitative study will be primarily semi-structured, open-ended questions asked during face-to-face interviews with the students (Creswell, 2009). In-depth interviews, employing only open-ended questions, enables the students to express their opinions regarding the topic being studied and essentially determine the direction of the interview (Horrocks & King, 2010). Many times, asking direct questions results in a diminished or limited response and fails to provide the deeper meaning that the researcher is looking for in the interview process. Freedom of expression is enhanced and pressure on a participant to answer as they think the interviewer wants is eliminated (Bogdan & Biklen, 2007). The relaxed atmosphere that results from this type of interview also serves to reduce possible apprehension that the student participants may feel during the observation portion of data collection (Horrocks & King, 2010).

According to Maxwell (2013), qualitative research is most often based on a relativistic or constructivist theory that hypothesizes that there is no objective reality. On the other hand, the user of qualitative methods anticipates multiple realities that are created by individual human beings who experience a phenomenon of interest (Gay et al., 2006). Thus, in the context of the proposed study, it is anticipated that the student participants will have their own unique opinions regarding transitioning to middle school. In fact, that will be a necessary element of the study to make sure that personal assessment by each student is integrated into the collected data.

Conclusion

This paper briefly reviewed the three research methods used in education research—quantitative, qualitative, and mixed methods—and explained why a qualitative method was selected for use in the proposed dissertation. Additionally, two previously written qualitative dissertations were reviewed to identify strengths of those two presentations and provide additional motivation for utilizing a qualitative method. These dissertations are similar to the proposed dissertation design (while not necessarily dealing with the exact topic proposed) with the objective of showing how qualitative research methods are preferred in the context of educational studies I n general and the proposed study in particular. Finally, an overview of the proposed dissertation was presented, including a justification for selecting a qualitative method for the study.

Qualitative researchers can often improve the quality of analysis by increasing the amount of information used to investigate a problem, frequently by increasing the number of observations. Researchers committed to the study of social phenomena that choose not to use quantitative procedures cannot afford to ignore sources of bias and inefficiency created by methodologically unreflective research designs. The topics they study are every bit as important, and often more important, than those analyzed by quantitative scholars. Descriptive and causal inferences made by qualitative researchers deserve to be as sound as those made by any other researcher. This point will be kept in mind as the proposed dissertation proceeds.

To make valid deductions, qualitative researchers quite often will need to be more familiar with methodological issues than quantitative researchers. They also must be more self-conscious when designing research and more unambiguous when reporting substantive results. Anyone reading published results should not have to reformulate qualitative studies to ensure they are scientifically valid. Fortunately, the suitable methodological issues for qualitative researchers to understand are exactly the ones that all other scientific researchers need to follow. Valid inference is possible only when the inherent logic underlying that research is understood and followed. The proposed study will comply with these standards.

References

Bogdan, R.C., & Biklen, S.K. (2007). Qualitative research for education: An introduction to theories and methods (5th ed.). Boston: Allyn and Bacon. Bush, B.D. (2013). Perceptions of Eighth Grade Students on the Transition from Preadolescence to Adolescence in a Selected Middle School. A dissertation submitted to the faculty of The University of Louisiana, Monroe.

Christians, C. (2000). Ethics and Politics in Qualitative Research. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of Qualitative Research (pp. 133-155). Thousand Oaks, CA: Sage.

Cohen, L., Manion L., & Morrison, K. (2007). Research methods in education (6th ed.). London: Routledge.

Connell, J.P., & Wellborn, J.G. (1991). Competence, autonomy, and relatedness: A motivational analysis of self-system processes. In M.R. Gunnar & L.A. Sroufe (Eds.), Self-processes and development (Vol. 23). Hillsdale, New Jersey: Lawrence Erlbaum Associates.

Creswell, J.W. (2009). Research design: Qualitative, quantitative and mixed methods approach (3rd ed.). Thousand Oaks, CA: Sage Publications.

Dörnyie, Z. (2007). Research methods in applied linguistics. Oxford: Oxford University Press.

Flick, U. (2009). An Introduction to Qualitative Research (4th ed.). London: Sage.

Gall, M.D., Borg, W.R., & Gall, J.P. (2007). Educational Research: An Introduction (8th ed.). New York: Longman.

Gay, L.R., Mills, G.E., & Airasian, P.W. (2006). Educational research: Competencies for analysis and application (8th ed.). Upper Saddle River, NJ: Prentice-Hall.

Horrocks, C., & King, N. (2010). Interviews in qualitative research. London: Sage.

Lichtman, K. (2013). Qualitative Research in Education (3rd ed.). Thousand Oaks, CA: Sage.

Maxwell, J.A. (2013). Qualitative Research Design: An Interactive Approach (3rd ed.).Thousand Oaks, CA: Sage.

Mustafa, R.F. (2011). The P.O.E.Ms of Educational Research: A beginners’ Concise Guide. International Education Studies, 4(3), 23-30.

Rubin, I.S., & Rubin, H.J. (2011). Qualitative interviewing: The art of hearing data. Thousand Oaks, CA: Sage.

Tomback R.M. (2007). Characteristics of Classroom Contexts, Self-Processes, Engagement, and Achievement Across the Transition from Middle School to High School. Dissertation submitted to the Faculty of the Graduate School of the University of Maryland. Retrieved from http://drum.lib.umd.edu/bitstream/1903/6661/1/umi-umd-4117.pdf