qualitative research is more difficult to analyze

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Am J Pharm Educ
• v.74(8); 2010 Oct 11

Presenting and Evaluating Qualitative Research

The purpose of this paper is to help authors to think about ways to present qualitative research papers in the American Journal of Pharmaceutical Education . It also discusses methods for reviewers to assess the rigour, quality, and usefulness of qualitative research. Examples of different ways to present data from interviews, observations, and focus groups are included. The paper concludes with guidance for publishing qualitative research and a checklist for authors and reviewers.

INTRODUCTION

Policy and practice decisions, including those in education, increasingly are informed by findings from qualitative as well as quantitative research. Qualitative research is useful to policymakers because it often describes the settings in which policies will be implemented. Qualitative research is also useful to both pharmacy practitioners and pharmacy academics who are involved in researching educational issues in both universities and practice and in developing teaching and learning.

Qualitative research involves the collection, analysis, and interpretation of data that are not easily reduced to numbers. These data relate to the social world and the concepts and behaviors of people within it. Qualitative research can be found in all social sciences and in the applied fields that derive from them, for example, research in health services, nursing, and pharmacy. 1 It looks at X in terms of how X varies in different circumstances rather than how big is X or how many Xs are there? 2 Textbooks often subdivide research into qualitative and quantitative approaches, furthering the common assumption that there are fundamental differences between the 2 approaches. With pharmacy educators who have been trained in the natural and clinical sciences, there is often a tendency to embrace quantitative research, perhaps due to familiarity. A growing consensus is emerging that sees both qualitative and quantitative approaches as useful to answering research questions and understanding the world. Increasingly mixed methods research is being carried out where the researcher explicitly combines the quantitative and qualitative aspects of the study. 3 , 4

Like healthcare, education involves complex human interactions that can rarely be studied or explained in simple terms. Complex educational situations demand complex understanding; thus, the scope of educational research can be extended by the use of qualitative methods. Qualitative research can sometimes provide a better understanding of the nature of educational problems and thus add to insights into teaching and learning in a number of contexts. For example, at the University of Nottingham, we conducted in-depth interviews with pharmacists to determine their perceptions of continuing professional development and who had influenced their learning. We also have used a case study approach using observation of practice and in-depth interviews to explore physiotherapists' views of influences on their leaning in practice. We have conducted in-depth interviews with a variety of stakeholders in Malawi, Africa, to explore the issues surrounding pharmacy academic capacity building. A colleague has interviewed and conducted focus groups with students to explore cultural issues as part of a joint Nottingham-Malaysia pharmacy degree program. Another colleague has interviewed pharmacists and patients regarding their expectations before and after clinic appointments and then observed pharmacist-patient communication in clinics and assessed it using the Calgary Cambridge model in order to develop recommendations for communication skills training. 5 We have also performed documentary analysis on curriculum data to compare pharmacist and nurse supplementary prescribing courses in the United Kingdom.

It is important to choose the most appropriate methods for what is being investigated. Qualitative research is not appropriate to answer every research question and researchers need to think carefully about their objectives. Do they wish to study a particular phenomenon in depth (eg, students' perceptions of studying in a different culture)? Or are they more interested in making standardized comparisons and accounting for variance (eg, examining differences in examination grades after changing the way the content of a module is taught). Clearly a quantitative approach would be more appropriate in the last example. As with any research project, a clear research objective has to be identified to know which methods should be applied.

Types of qualitative data include:

• Audio recordings and transcripts from in-depth or semi-structured interviews
• Structured interview questionnaires containing substantial open comments including a substantial number of responses to open comment items.
• Audio recordings and transcripts from focus group sessions.
• Field notes (notes taken by the researcher while in the field [setting] being studied)
• Video recordings (eg, lecture delivery, class assignments, laboratory performance)
• Case study notes
• Documents (reports, meeting minutes, e-mails)
• Diaries, video diaries
• Observation notes
• Press clippings
• Photographs

RIGOUR IN QUALITATIVE RESEARCH

Qualitative research is often criticized as biased, small scale, anecdotal, and/or lacking rigor; however, when it is carried out properly it is unbiased, in depth, valid, reliable, credible and rigorous. In qualitative research, there needs to be a way of assessing the “extent to which claims are supported by convincing evidence.” 1 Although the terms reliability and validity traditionally have been associated with quantitative research, increasingly they are being seen as important concepts in qualitative research as well. Examining the data for reliability and validity assesses both the objectivity and credibility of the research. Validity relates to the honesty and genuineness of the research data, while reliability relates to the reproducibility and stability of the data.

The validity of research findings refers to the extent to which the findings are an accurate representation of the phenomena they are intended to represent. The reliability of a study refers to the reproducibility of the findings. Validity can be substantiated by a number of techniques including triangulation use of contradictory evidence, respondent validation, and constant comparison. Triangulation is using 2 or more methods to study the same phenomenon. Contradictory evidence, often known as deviant cases, must be sought out, examined, and accounted for in the analysis to ensure that researcher bias does not interfere with or alter their perception of the data and any insights offered. Respondent validation, which is allowing participants to read through the data and analyses and provide feedback on the researchers' interpretations of their responses, provides researchers with a method of checking for inconsistencies, challenges the researchers' assumptions, and provides them with an opportunity to re-analyze their data. The use of constant comparison means that one piece of data (for example, an interview) is compared with previous data and not considered on its own, enabling researchers to treat the data as a whole rather than fragmenting it. Constant comparison also enables the researcher to identify emerging/unanticipated themes within the research project.

STRENGTHS AND LIMITATIONS OF QUALITATIVE RESEARCH

Qualitative researchers have been criticized for overusing interviews and focus groups at the expense of other methods such as ethnography, observation, documentary analysis, case studies, and conversational analysis. Qualitative research has numerous strengths when properly conducted.

Strengths of Qualitative Research

• Issues can be examined in detail and in depth.
• Interviews are not restricted to specific questions and can be guided/redirected by the researcher in real time.
• The research framework and direction can be quickly revised as new information emerges.
• The data based on human experience that is obtained is powerful and sometimes more compelling than quantitative data.
• Subtleties and complexities about the research subjects and/or topic are discovered that are often missed by more positivistic enquiries.
• Data usually are collected from a few cases or individuals so findings cannot be generalized to a larger population. Findings can however be transferable to another setting.

Limitations of Qualitative Research

• Research quality is heavily dependent on the individual skills of the researcher and more easily influenced by the researcher's personal biases and idiosyncrasies.
• Rigor is more difficult to maintain, assess, and demonstrate.
• The volume of data makes analysis and interpretation time consuming.
• It is sometimes not as well understood and accepted as quantitative research within the scientific community
• The researcher's presence during data gathering, which is often unavoidable in qualitative research, can affect the subjects' responses.
• Issues of anonymity and confidentiality can present problems when presenting findings
• Findings can be more difficult and time consuming to characterize in a visual way.

PRESENTATION OF QUALITATIVE RESEARCH FINDINGS

The following extracts are examples of how qualitative data might be presented:

Data From an Interview.

The following is an example of how to present and discuss a quote from an interview.

The researcher should select quotes that are poignant and/or most representative of the research findings. Including large portions of an interview in a research paper is not necessary and often tedious for the reader. The setting and speakers should be established in the text at the end of the quote.

The student describes how he had used deep learning in a dispensing module. He was able to draw on learning from a previous module, “I found that while using the e learning programme I was able to apply the knowledge and skills that I had gained in last year's diseases and goals of treatment module.” (interviewee 22, male)

This is an excerpt from an article on curriculum reform that used interviews 5 :

The first question was, “Without the accreditation mandate, how much of this curriculum reform would have been attempted?” According to respondents, accreditation played a significant role in prompting the broad-based curricular change, and their comments revealed a nuanced view. Most indicated that the change would likely have occurred even without the mandate from the accreditation process: “It reflects where the profession wants to be … training a professional who wants to take on more responsibility.” However, they also commented that “if it were not mandated, it could have been a very difficult road.” Or it “would have happened, but much later.” The change would more likely have been incremental, “evolutionary,” or far more limited in its scope. “Accreditation tipped the balance” was the way one person phrased it. “Nobody got serious until the accrediting body said it would no longer accredit programs that did not change.”

Data From Observations

The following example is some data taken from observation of pharmacist patient consultations using the Calgary Cambridge guide. 6 , 7 The data are first presented and a discussion follows:

Pharmacist: We will soon be starting a stop smoking clinic. Patient: Is the interview over now? Pharmacist: No this is part of it. (Laughs) You can't tell me to bog off (sic) yet. (pause) We will be starting a stop smoking service here, Patient: Yes. Pharmacist: with one-to-one and we will be able to help you or try to help you. If you want it. In this example, the pharmacist has picked up from the patient's reaction to the stop smoking clinic that she is not receptive to advice about giving up smoking at this time; in fact she would rather end the consultation. The pharmacist draws on his prior relationship with the patient and makes use of a joke to lighten the tone. He feels his message is important enough to persevere but he presents the information in a succinct and non-pressurised way. His final comment of “If you want it” is important as this makes it clear that he is not putting any pressure on the patient to take up this offer. This extract shows that some patient cues were picked up, and appropriately dealt with, but this was not the case in all examples.

Data From Focus Groups

This excerpt from a study involving 11 focus groups illustrates how findings are presented using representative quotes from focus group participants. 8

Those pharmacists who were initially familiar with CPD endorsed the model for their peers, and suggested it had made a meaningful difference in the way they viewed their own practice. In virtually all focus groups sessions, pharmacists familiar with and supportive of the CPD paradigm had worked in collaborative practice environments such as hospital pharmacy practice. For these pharmacists, the major advantage of CPD was the linking of workplace learning with continuous education. One pharmacist stated, “It's amazing how much I have to learn every day, when I work as a pharmacist. With [the learning portfolio] it helps to show how much learning we all do, every day. It's kind of satisfying to look it over and see how much you accomplish.” Within many of the learning portfolio-sharing sessions, debates emerged regarding the true value of traditional continuing education and its outcome in changing an individual's practice. While participants appreciated the opportunity for social and professional networking inherent in some forms of traditional CE, most eventually conceded that the academic value of most CE programming was limited by the lack of a systematic process for following-up and implementing new learning in the workplace. “Well it's nice to go to these [continuing education] events, but really, I don't know how useful they are. You go, you sit, you listen, but then, well I at least forget.”

The following is an extract from a focus group (conducted by the author) with first-year pharmacy students about community placements. It illustrates how focus groups provide a chance for participants to discuss issues on which they might disagree.

Interviewer: So you are saying that you would prefer health related placements? Student 1: Not exactly so long as I could be developing my communication skill. Student 2: Yes but I still think the more health related the placement is the more I'll gain from it. Student 3: I disagree because other people related skills are useful and you may learn those from taking part in a community project like building a garden. Interviewer: So would you prefer a mixture of health and non health related community placements?

GUIDANCE FOR PUBLISHING QUALITATIVE RESEARCH

Qualitative research is becoming increasingly accepted and published in pharmacy and medical journals. Some journals and publishers have guidelines for presenting qualitative research, for example, the British Medical Journal 9 and Biomedcentral . 10 Medical Education published a useful series of articles on qualitative research. 11 Some of the important issues that should be considered by authors, reviewers and editors when publishing qualitative research are discussed below.

Introduction.

A good introduction provides a brief overview of the manuscript, including the research question and a statement justifying the research question and the reasons for using qualitative research methods. This section also should provide background information, including relevant literature from pharmacy, medicine, and other health professions, as well as literature from the field of education that addresses similar issues. Any specific educational or research terminology used in the manuscript should be defined in the introduction.

The methods section should clearly state and justify why the particular method, for example, face to face semistructured interviews, was chosen. The method should be outlined and illustrated with examples such as the interview questions, focusing exercises, observation criteria, etc. The criteria for selecting the study participants should then be explained and justified. The way in which the participants were recruited and by whom also must be stated. A brief explanation/description should be included of those who were invited to participate but chose not to. It is important to consider “fair dealing,” ie, whether the research design explicitly incorporates a wide range of different perspectives so that the viewpoint of 1 group is never presented as if it represents the sole truth about any situation. The process by which ethical and or research/institutional governance approval was obtained should be described and cited.

The study sample and the research setting should be described. Sampling differs between qualitative and quantitative studies. In quantitative survey studies, it is important to select probability samples so that statistics can be used to provide generalizations to the population from which the sample was drawn. Qualitative research necessitates having a small sample because of the detailed and intensive work required for the study. So sample sizes are not calculated using mathematical rules and probability statistics are not applied. Instead qualitative researchers should describe their sample in terms of characteristics and relevance to the wider population. Purposive sampling is common in qualitative research. Particular individuals are chosen with characteristics relevant to the study who are thought will be most informative. Purposive sampling also may be used to produce maximum variation within a sample. Participants being chosen based for example, on year of study, gender, place of work, etc. Representative samples also may be used, for example, 20 students from each of 6 schools of pharmacy. Convenience samples involve the researcher choosing those who are either most accessible or most willing to take part. This may be fine for exploratory studies; however, this form of sampling may be biased and unrepresentative of the population in question. Theoretical sampling uses insights gained from previous research to inform sample selection for a new study. The method for gaining informed consent from the participants should be described, as well as how anonymity and confidentiality of subjects were guaranteed. The method of recording, eg, audio or video recording, should be noted, along with procedures used for transcribing the data.

Data Analysis.

A description of how the data were analyzed also should be included. Was computer-aided qualitative data analysis software such as NVivo (QSR International, Cambridge, MA) used? Arrival at “data saturation” or the end of data collection should then be described and justified. A good rule when considering how much information to include is that readers should have been given enough information to be able to carry out similar research themselves.

One of the strengths of qualitative research is the recognition that data must always be understood in relation to the context of their production. 1 The analytical approach taken should be described in detail and theoretically justified in light of the research question. If the analysis was repeated by more than 1 researcher to ensure reliability or trustworthiness, this should be stated and methods of resolving any disagreements clearly described. Some researchers ask participants to check the data. If this was done, it should be fully discussed in the paper.

An adequate account of how the findings were produced should be included A description of how the themes and concepts were derived from the data also should be included. Was an inductive or deductive process used? The analysis should not be limited to just those issues that the researcher thinks are important, anticipated themes, but also consider issues that participants raised, ie, emergent themes. Qualitative researchers must be open regarding the data analysis and provide evidence of their thinking, for example, were alternative explanations for the data considered and dismissed, and if so, why were they dismissed? It also is important to present outlying or negative/deviant cases that did not fit with the central interpretation.

The interpretation should usually be grounded in interviewees or respondents' contributions and may be semi-quantified, if this is possible or appropriate, for example, “Half of the respondents said …” “The majority said …” “Three said…” Readers should be presented with data that enable them to “see what the researcher is talking about.” 1 Sufficient data should be presented to allow the reader to clearly see the relationship between the data and the interpretation of the data. Qualitative data conventionally are presented by using illustrative quotes. Quotes are “raw data” and should be compiled and analyzed, not just listed. There should be an explanation of how the quotes were chosen and how they are labeled. For example, have pseudonyms been given to each respondent or are the respondents identified using codes, and if so, how? It is important for the reader to be able to see that a range of participants have contributed to the data and that not all the quotes are drawn from 1 or 2 individuals. There is a tendency for authors to overuse quotes and for papers to be dominated by a series of long quotes with little analysis or discussion. This should be avoided.

Participants do not always state the truth and may say what they think the interviewer wishes to hear. A good qualitative researcher should not only examine what people say but also consider how they structured their responses and how they talked about the subject being discussed, for example, the person's emotions, tone, nonverbal communication, etc. If the research was triangulated with other qualitative or quantitative data, this should be discussed.

Discussion.

The findings should be presented in the context of any similar previous research and or theories. A discussion of the existing literature and how this present research contributes to the area should be included. A consideration must also be made about how transferrable the research would be to other settings. Any particular strengths and limitations of the research also should be discussed. It is common practice to include some discussion within the results section of qualitative research and follow with a concluding discussion.

The author also should reflect on their own influence on the data, including a consideration of how the researcher(s) may have introduced bias to the results. The researcher should critically examine their own influence on the design and development of the research, as well as on data collection and interpretation of the data, eg, were they an experienced teacher who researched teaching methods? If so, they should discuss how this might have influenced their interpretation of the results.

Conclusion.

The conclusion should summarize the main findings from the study and emphasize what the study adds to knowledge in the area being studied. Mays and Pope suggest the researcher ask the following 3 questions to determine whether the conclusions of a qualitative study are valid 12 : How well does this analysis explain why people behave in the way they do? How comprehensible would this explanation be to a thoughtful participant in the setting? How well does the explanation cohere with what we already know?

CHECKLIST FOR QUALITATIVE PAPERS

This paper establishes criteria for judging the quality of qualitative research. It provides guidance for authors and reviewers to prepare and review qualitative research papers for the American Journal of Pharmaceutical Education . A checklist is provided in Appendix 1 to assist both authors and reviewers of qualitative data.

ACKNOWLEDGEMENTS

Thank you to the 3 reviewers whose ideas helped me to shape this paper.

Appendix 1. Checklist for authors and reviewers of qualitative research.

Introduction

• □ Research question is clearly stated.
• □ Research question is justified and related to the existing knowledge base (empirical research, theory, policy).
• □ Any specific research or educational terminology used later in manuscript is defined.
• □ The process by which ethical and or research/institutional governance approval was obtained is described and cited.
• □ Reason for choosing particular research method is stated.
• □ Criteria for selecting study participants are explained and justified.
• □ Recruitment methods are explicitly stated.
• □ Details of who chose not to participate and why are given.
• □ Study sample and research setting used are described.
• □ Method for gaining informed consent from the participants is described.
• □ Maintenance/Preservation of subject anonymity and confidentiality is described.
• □ Method of recording data (eg, audio or video recording) and procedures for transcribing data are described.
• □ Methods are outlined and examples given (eg, interview guide).
• □ Decision to stop data collection is described and justified.
• □ Data analysis and verification are described, including by whom they were performed.
• □ Methods for identifying/extrapolating themes and concepts from the data are discussed.
• □ Sufficient data are presented to allow a reader to assess whether or not the interpretation is supported by the data.
• □ Outlying or negative/deviant cases that do not fit with the central interpretation are presented.
• □ Transferability of research findings to other settings is discussed.
• □ Findings are presented in the context of any similar previous research and social theories.
• □ Discussion often is incorporated into the results in qualitative papers.
• □ A discussion of the existing literature and how this present research contributes to the area is included.
• □ Any particular strengths and limitations of the research are discussed.
• □ Reflection of the influence of the researcher(s) on the data, including a consideration of how the researcher(s) may have introduced bias to the results is included.

Conclusions

• □ The conclusion states the main finings of the study and emphasizes what the study adds to knowledge in the subject area.

• Skip to main content
• Skip to primary sidebar
• Skip to footer

Online Metrics

Google Analytics Courses and Consulting

July 14, 2015 By Paul Koks Leave a Comment

Six Challenges of Qualitative Data Analysis

In an ideal world there is both valuable quantitative as well as qualitative data available to you.

You can’t say that one data source is better than the other. They complement each other and provide you with a more accurate picture of what’s going on and why.

Both data sources are very helpful in the field of conversion optimization.

Well thought out hypothesis – based on quantitative and qualitative data – are important to define the best A/B test experiments .

In this article I share six common problems with qualitative data that you should know.

Sampling-Related Problems

The first three limitations are sampling-related issues.

1. Limited Sample Size

Contrary to quantitative data where you often have a great amount of data available, is sample size one of the challenges of qualitative data.

If you browse on the internet, you find out there is no general agreement on the ideal sample size for qualitative research.

It is very costly to perform extensive qualitative research with hundreds of participants.

And is it really needed to question so many people to get valuable insights?

Watch this video to get a better understanding of this topic:

Two tips about your sample size:

• Rule of thumb: you need more participants if new participants keep on providing you with relevant, new insights.
• Be flexible; don’t rigidly set the number of participants at the start.

2. Sampling Bias

Sampling bias definition by Wikipedia :

“In statistics, sampling bias is a bias in which a sample is collected in such a way that some members of the intended population are less likely to be included than others.”

In other words, your qualitative sample will never include a representative overview of all the different people that come to your website.

It’s important to keep that in mind when interpreting test results.

3. Self-Selection Bias

Do you like to participate in surveys? A few of you might say “Yes” and others think “Arghhhh”.

This is the exact problem here. It’s a free choice to participate in a research study or not.

On the other side, quantitative data is gathered from most people whether they like it or not.

Just sign up for Hotjar , set up a heatmap and the data will be collected for you.

Ok, I don’t talk about the tech-savvy people here. ;-)

Sampling and self-selection biases are closely related and limit the usefulness of qualitative data.

Observation Biases

The second group of problems with qualitative data include observational biases.

4. Hawthorne Effect

The Hawthorne Effect can best be described as:

“Participants in behavioral studies change their behavior or performance in response to being observed.”

For example, your opinion about a particular website might be different when you know you are being observed if compared to when you (don’t know) you are being observed.

I recommend to watch this video (it clearly explains the Hawthorne Effect and its background):

5. Observer-Expectancy Effect

Let’s say you are running a survey and function as an observer in the research room. You are walking around and observe the participants.

Do you think you won’t influence the results?

It is known that researcher’s beliefs or expectations causes him or her to uncon­sciously influ­ence the par­tic­i­pants of an experiment. This is called the observer-expectancy effect.

6. Artificial Scenario

Most experiments include pre-set goals in a specific environment. And you can’t get feedback on things you don’t ask.

For example, you run an experiment for an ecommerce website .

Your goal is to find out whether the form (where people leave their personal information) functions well or if anything needs to be improved.

In this case it is such a focused goal so that you won’t learn about other valuable things through this study.

The participant might have a lot of other things to say, but without asking them you won’t know it.

Conclusions

As you can see, there are a many challenges with qualitative data.

However, marketers can perform extremely well if they use this data in combination with quantitative data to form strong A/B test hypothesis.

Refrain from changing your website on just a small set of qualitative responses.

Instead, enrich your conversion optimization framework with all data sources that are available to you and get more out of your testing efforts.

What’s your experience with qualitative data? Do you use it in combination with quantitative data?

Reader Interactions

Leave a reply cancel reply.

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed .

Online Metrics enhances your data quality and insights so that you can improve your business results. Learn more

Market Research

Qualitative Research: Understanding the Goal and Benefits for Effective Analysis

As market trends evolve at lightning speed in the age of digital transformation, having an intimate understanding of consumer desires and motivations is more critical than ever. Enter qualitative research – the knight in shining armor of deep-dive data analysis. In this blog post, we’ll be exploring the profound purpose and impressive benefits behind qualitative research, unveiling how it anchors effective market analysis and strategy development. Brace yourselves for a mesmerizing journey into the realm of potent insights that power consequential decisions and breed groundbreaking innovation.

The primary goal of qualitative research is to obtain insights into participants’ experiences and understanding of the world. This type of research provides rich descriptions and explanations of processes in identifiable local contexts. Qualitative research has several benefits including providing an in-depth understanding, being flexible and adaptable, and generating descriptive data that can be used to create new theories using the inductive method. 

Qualitative Study’s Importance

Qualitative research holds a significant place in the realm of social science research and is integral for understanding the complexities of human behavior, experiences, and social interactions. Unlike quantitative research which focuses on numerical data and statistical analysis, qualitative research collects non-numerical data and emphasizes interpreting meaning from social contexts.

The importance of qualitative research lies in its ability to provide rich descriptions and explanations of processes in identifiable local contexts. It allows researchers to gain insights into participants’ experiences and understand the world as another person experiences it. This deeper understanding paves the way for more comprehensive analyses and the development of theories that accurately represent the intricacies of human life.

For instance, imagine a sociologist interested in studying how individuals cope with unemployment during economic downturns. By conducting qualitative research , these sociologists can immerse themselves in the lives of unemployed individuals, observe their daily routines, conduct in-depth interviews, and analyze their personal narratives. This approach goes beyond simply quantifying unemployment rates; it provides an intimate understanding of how people navigate through difficult situations and sheds light on the emotional, psychological, and societal impacts.

In addition to providing rich insight into human experiences, qualitative research offers numerous other benefits that contribute to effective analysis.

• Qualitative research is essential in social science research as it allows for a deeper understanding of human behavior and social interactions. Its focus on non-numerical data collection and interpretation of meaning helps researchers gain insights into participants’ experiences and contextual factors. Qualitative research also provides rich descriptions and explanations of processes in identifiable local contexts, leading to the development of comprehensive analysis and accurate theories. Overall, qualitative research offers numerous benefits that contribute to effective analysis in social science research.

Goals & Benefits Driving Research

The goals of qualitative research are multifaceted. One primary objective is to investigate the meanings people attribute to their behavior and interactions within specific social contexts. This focus on subjective interpretations helps uncover individual perspectives that may be overlooked by quantitative methods alone. Additionally, qualitative research aims to explore social phenomena that are not easily measurable or quantifiable.

Qualitative research also generates descriptive data that requires rigorous methods of analysis. Researchers employ various techniques such as thematic analysis or grounded theory to identify patterns, themes, and categories within their data. These analytical approaches ensure systematic interpretation while maintaining the integrity of participants’ lived experiences.

Beyond these goals, qualitative research offers several benefits that aid in reliable analysis. Firstly, it provides an in-depth understanding of complex social issues by capturing the nuances and subtleties of human behavior. This depth allows researchers to generate rich descriptions and explanations that facilitate a comprehensive comprehension of social phenomena.

For example, consider a study exploring the experience of minority students in predominantly white institutions. Through qualitative research methods like interviews and focus groups, researchers can delve into the students’ lived experiences, their perceptions of inclusion or exclusion, and their strategies for navigating through institutional challenges. This level of detail paints a holistic picture that goes beyond quantitative statistics such as enrollment numbers.

Another advantage of qualitative research is its flexibility and adaptability. Researchers can modify their data collection methods to account for new insights or unexpected findings during the research process. This responsiveness allows for deeper exploration and ensures that no valuable information is left unexamined.

However, it is essential to acknowledge that qualitative research also has its limitations. These include the limited scope and generalizability of findings due to the smaller sample sizes typically used in qualitative studies. Additionally, there is a potential for researcher bias since the individuals collecting and analyzing the data play an active role in shaping the research process.

Nonetheless, while objectivity may be seen as a myth in qualitative research, researchers should be honest and transparent about their own biases and assumptions. Reflexivity, which involves acknowledging and critically examining one’s subjectivity throughout the research process, is integral to ensuring integrity and minimizing undue influence.

• According to a report from the Journal of Social Issues, as of 2022, around 45% of psychological studies used qualitative methods, signaling strong recognition in the field for its unique insights into human behavior.
• A study conducted by the Market Research Society confirmed that out of all market research carried out worldwide, approximately 20% utilize qualitative methodologies. This highlights its crucial role in understanding customer behaviors and motivations.
• The National Center for Biotechnology Information (NCBI) indicated that nearly 70% of health research incorporates some elements of qualitative research, underscoring its importance in contributing to our understanding of complex health issues and interventions.

Comprehensive Approaches

When conducting qualitative research , adopting comprehensive approaches is essential for capturing the richness and depth of data required for effective analysis. These approaches involve a holistic perspective that considers multiple dimensions and contexts. One commonly used comprehensive approach is triangulation , which involves using multiple data sources, methods, or perspectives to cross-verify findings. By triangulating data, researchers can enhance the reliability and validity of their analysis.

Another important approach is thick description , which focuses on providing detailed and vivid accounts of participants’ experiences and contexts. This technique enables researchers to capture the nuances and complexities of social phenomena, ensuring a comprehensive understanding of the research topic. Thick descriptions typically include vivid narratives, dialogue excerpts, and detailed observations, providing readers with a rich portrayal of the study’s context.

Researchers may also adopt an iterative process in their analysis, where data collection and analysis occur simultaneously. This approach allows for constant refinement and adjustment of research questions and methods based on emerging findings. Through iteration, researchers can dive deeper into the topic, uncover unexpected insights, and explore various angles that contribute to a more comprehensive analysis.

It’s worth noting that comprehensive approaches in qualitative research require flexibility and openness to embracing emergent themes and unexpected directions. As researchers immerse themselves in the data, they should be willing to adapt their strategies accordingly.

Participant Engagement & Topic Exploration

Participant engagement plays a crucial role in qualitative research as it fosters a deeper understanding of participants’ perspectives and experiences. Effective engagement encourages open dialogue and trust between the researcher and participants, allowing for richer data collection. One way to promote participant engagement is through active listening . By attentively listening to participants’ stories, concerns, and viewpoints, researchers can demonstrate empathy and create a safe space for open expression.

Another aspect that greatly enhances participant engagement is establishing rapport . Building rapport involves creating a comfortable environment where participants feel at ease to share their thoughts and experiences. This can be achieved through transparent communication, respect for participants’ autonomy, and genuine interest in their stories. Researchers should establish a positive and respectful relationship with participants, positioning themselves as partners rather than authoritative figures.

In qualitative research, topic exploration is a dynamic and iterative process that allows researchers to uncover new insights and dimensions of the phenomenon under study. This involves probing deeper into participants’ responses, asking follow-up questions, and exploring unexpected avenues that emerge during data collection. By being open to revisiting research questions and digging deeper into topics, researchers can uncover valuable insights and gain a more comprehensive understanding of the subject matter.

It’s important to note that participant engagement and topic exploration go hand in hand. Engaged participants are more likely to provide rich and detailed responses, leading to enhanced exploration of the research topic. Conversely, skillful topic exploration can foster deeper engagement from participants by demonstrating genuine interest and curiosity in their perspectives.

Effective Data Accumulation Methods

In qualitative research, the collection of rich and meaningful data is a crucial step toward understanding the complexities of human experiences. To ensure effective analysis, researchers need to employ appropriate data accumulation methods that capture the depth of participants’ perspectives and insights. Let’s explore some strategies that can facilitate this process.

One common method used in qualitative research is participant observation. This involves immersing oneself in the research setting, actively observing, and taking detailed notes on behaviors, interactions, and cultural nuances. By being present in the natural context, researchers gain a deeper understanding of the social dynamics and can document valuable data that may go unnoticed otherwise.

For instance, imagine a researcher interested in studying the experiences of healthcare workers in a hospital. Through participant observation, they can shadow these workers, witness their daily routines, the challenges they face, and even engage in conversations during breaks. This method provides an intimate look into their lives and generates valuable insights.

Another effective technique is in-depth interviews. These interviews allow researchers to establish a personal connection with participants and delve into their thoughts, feelings, and motivations regarding the research topic. It’s crucial to create an open and comfortable environment where participants feel safe sharing their views openly.

Additionally, focus groups are utilized as a powerful data accumulation method. Bringing together a small group of individuals who share similar characteristics or experiences allows for stimulating discussions that uncover diverse perspectives. Participants can build upon each other’s ideas and provide deeper insights collectively.

Having explored effective data accumulation methods like participant observation, in-depth interviews, and focus groups, let’s now dive into another important aspect of qualitative research – harnessing sensory inputs & eliciting verbal responses.

Harnessing Sensory Inputs and Eliciting Verbal Responses

Qualitative research aims to understand phenomena from the perspective of individuals involved. One way to achieve this is by harnessing sensory inputs and eliciting verbal responses, allowing participants to express themselves fully. This approach taps into a range of human senses and encourages participants to describe their experiences vividly.

For instance, researchers might utilize photovoice techniques, where participants capture images related to the research topic using cameras or smartphones. These visual representations allow participants to share their perspectives in a unique and powerful way.

Imagine a study exploring the impact of urbanization on community well-being. Participants could be asked to take pictures of spaces they feel contribute positively or negatively to their quality of life. These images can then be used as stimuli for further discussion, sparking conversations about the emotional and sensory aspects of the built environment.

In addition to visuals, researchers can also engage participants’ sense of hearing through audio recordings. By recording interviews, focus group discussions, or even ambient sounds in a particular environment, researchers can capture subtle nuances that may not be conveyed through written transcripts alone.

By harnessing sensory inputs and giving participants the space for verbal expression, qualitative researchers foster an environment where rich and nuanced data can be collected. This multi-sensory approach enables a deeper understanding of individuals’ experiences and allows us to gain insights beyond mere words.

Parsing and Conclusion Derivation from Data

In qualitative research, one of the primary goals is to parse and derive meaningful conclusions from the collected data. Unlike quantitative research which relies on statistical analysis, qualitative research involves obtaining rich descriptions of participants’ experiences and understanding the world as another person experiences it. The process of parsing and deriving conclusions from qualitative data requires a meticulous examination of the data, identification of patterns, themes, and connections, and an inductive approach to theory development.

Qualitative researchers immerse themselves in the data collected through methods such as interviews, observations, and focus groups. They carefully analyze transcripts, field notes, or documents to identify recurring themes or significant incidents that shed light on the research question. Through this process of coding and categorizing, researchers start to make sense of the data and identify key findings that can be used to develop theories or inform specific contexts.

For example, imagine a researcher conducting an ethnographic study exploring the experiences of undocumented immigrants in their journey toward citizenship. Through interviews and participant observation, they gather compelling stories and narratives about the challenges faced by these individuals. By carefully analyzing these stories for common themes such as navigating legal systems or facing social stigma, the researcher can derive conclusions about the complex processes involved in seeking legal status.

“Analyzing qualitative data is like piecing together a puzzle. Each interview or observation provides a unique piece that contributes to the overall picture.”

However, it is important to note that deriving conclusions from qualitative data is not a simple linear process. It requires reflexivity on the part of the researcher to acknowledge their own biases and assumptions that may influence their interpretation of the data. Reflexivity encourages researchers to critically reflect on how their own subjectivity affects their analysis and conclusions.

Advantages & Drawbacks of This Research Type

Qualitative research offers several advantages that contribute to its effectiveness in providing rich insights into social phenomena. First and foremost, it allows researchers to gain an in-depth understanding of the experiences, perspectives, and meanings that individuals attribute to their behavior and interactions. This depth of understanding is often difficult to achieve through quantitative research methods alone.

Moreover, qualitative research is known for its flexibility and adaptability. Researchers can modify their research design or data collection methods as they delve deeper into the field, responding to emerging themes or new areas of investigation. The open-ended nature of qualitative research also enables participants to express themselves freely and provide nuanced responses, offering a more comprehensive view of complex social phenomena.

On the other hand, there are some drawbacks to consider when conducting qualitative research. One challenge is the limited scope and generalizability of findings. Due to the small sample sizes typically involved in qualitative studies, it can be challenging to extrapolate findings to broader populations or contexts. Additionally, there is potential for researcher bias as interpretations of qualitative data are subjective and influenced by researchers’ perspectives and assumptions.

Despite these limitations, the benefits of qualitative research outweigh its drawbacks in many cases. By providing detailed insights into participants’ experiences, qualitative research contributes valuable knowledge that can inform policy decisions, improve interventions, and enhance our understanding of social phenomena.

Unlock the power of qualitative research with Discuss

In a world driven by meaningful connections, Discuss stands at the forefront of qualitative research, empowering you to delve deeper, understand better, and innovate with confidence. Elevate your research game—choose Discuss for insights that go beyond the surface. Navigate cultural nuances effortlessly. Our platform is designed to facilitate cross-cultural research, helping you understand and appreciate the local context that shapes consumer behavior around the world.

Why Discuss ?

• Unparalleled Access: Connect with your target audience effortlessly, breaking down geographical barriers and ensuring your research is truly representative.
• Real-time Collaboration: Seamlessly share ideas, gather feedback, and refine your approach on the fly.
• Rich Multimedia Insights: Witness authentic reactions, emotions, and body language that add layers of depth to your qualitative findings.
• Data-Driven Decision Making: Make informed decisions backed by real, human-driven data.

Sign Up for our Newsletter

Related articles.

Navigating Tomorrow: A Glimpse into the Future of Market Research in 2024

Author: Jim Longo, Co-founder & Chief Strategy Officer As a veteran with over 30 years in…

The Importance of a Market Research Analyst: Key Benefits and Skills

As the world becomes increasingly data-driven, understanding market trends is more crucial than ever for business…

Forrester and G2 Reports Explained

In this short video, Discuss’ Chief Growth Officer, Adam Mertz, highlights three recent industry reports and…

Qualitative vs. Quantitative Data: 7 Key Differences

Qualitative data is information you can describe with words rather than numbers. 

Quantitative data is information represented in a measurable way using numbers. 

One type of data isn’t better than the other. 

To conduct thorough research, you need both. But knowing the difference between them is important if you want to harness the full power of both qualitative and quantitative data. 

In this post, we’ll explore seven key differences between these two types of data. 

#1. The Type of Data

The single biggest difference between quantitative and qualitative data is that one deals with numbers, and the other deals with concepts and ideas. 

The words “qualitative” and “quantitative” are really similar, which can make it hard to keep track of which one is which. I like to think of them this way: 

• Quantitative = quantity = numbers-related data
• Qualitative = quality = descriptive data

Qualitative data—the descriptive one—usually involves written or spoken words, images, or even objects. It’s collected in all sorts of ways: video recordings, interviews, open-ended survey responses, and field notes, for example. 

I like how researcher James W. Crick defines qualitative research in a 2021 issue of the Journal of Strategic Marketing : “Qualitative research is designed to generate in-depth and subjective findings to build theory.”

In other words, qualitative research helps you learn more about a topic—usually from a primary, or firsthand, source—so you can form ideas about what it means. This type of data is often rich in detail, and its interpretation can vary depending on who’s analyzing it. 

Here’s what I mean: if you ask five different people to observe how 60 kittens behave when presented with a hamster wheel, you’ll get five different versions of the same event. 

Quantitative data, on the other hand, is all about numbers and statistics. There’s no wiggle room when it comes to interpretation. In our kitten scenario, quantitative data might show us that of the 60 kittens presented with a hamster wheel, 40 pawed at it, 5 jumped inside and started spinning, and 15 ignored it completely.

There’s no ifs, ands, or buts about the numbers. They just are. 

#2. When to Use Each Type of Data

You should use both quantitative and quantitative data to make decisions for your business. 

Quantitative data helps you get to the what . Qualitative data unearths the why .

Quantitative data collects surface information, like numbers. Qualitative data dives deep beneath these same numbers and fleshes out the nuances there. 

Research projects can often benefit from both types of data, which is why you’ll see the term “mixed-method” research in peer-reviewed journals. The term “mixed-method” refers to using both quantitative and qualitative methods in a study. 

So, maybe you’re diving into original research. Or maybe you’re looking at other peoples’ studies to make an important business decision. In either case, you can use both quantitative and qualitative data to guide you.

Imagine you want to start a company that makes hamster wheels for cats. You run that kitten experiment, only to learn that most kittens aren’t all that interested in the hamster wheel. That’s what your quantitative data seems to say. Of the 60 kittens who participated in the study, only 5 hopped into the wheel. 

But 40 of the kittens pawed at the wheel. According to your quantitative data, these 40 kittens touched the wheel but did not get inside. 

This is where your qualitative data comes into play. Why did these 40 kittens touch the wheel but stop exploring it? You turn to the researchers’ observations. Since there were five different researchers, you have five sets of detailed notes to study. 

From these observations, you learn that many of the kittens seemed frightened when the wheel moved after they pawed it. They grew suspicious of the structure, meowing and circling it, agitated.

One researcher noted that the kittens seemed desperate to enjoy the wheel, but they didn’t seem to feel it was safe. 

So your idea isn’t a flop, exactly. 

It just needs tweaking. 

According to your quantitative data, 75% of the kittens studied either touched or actively participated in the hamster wheel. Your qualitative data suggests more kittens would have jumped into the wheel if it hadn’t moved so easily when they pawed at it. 

You decide to make your kitten wheel sturdier and try the whole test again with a new set of kittens. Hopefully, this time a higher percentage of your feline participants will hop in and enjoy the fun. 

This is a very simplistic and fictional example of how a mixed-method approach can help you make important choices for your business. 

#3. Data You Have Access To

When you can swing it, you should look at both qualitative and quantitative data before you make any big decisions. 

But this is where we come to another big difference between quantitative vs. qualitative data: it’s a lot easier to source qualitative data than quantitative data. 

Why? Because it’s easy to run a survey, host a focus group, or conduct a round of interviews. All you have to do is hop on SurveyMonkey or Zoom and you’re on your way to gathering original qualitative data. 

And yes, you can get some quantitative data here. If you run a survey and 45 customers respond, you can collect demographic data and yes/no answers for that pool of 45 respondents.

But this is a relatively small sample size. (More on why this matters in a moment.) 

To tell you anything meaningful, quantitative data must achieve statistical significance. 

If it’s been a while since your college statistics class, here’s a refresh: statistical significance is a measuring stick. It tells you whether the results you get are due to a specific cause or if they can be attributed to random chance. 

To achieve statistical significance in a study, you have to be really careful to set the study up the right way and with a meaningful sample size.

This doesn’t mean it’s impossible to get quantitative data. But unless you have someone on your team who knows all about null hypotheses and p-values and statistical analysis, you might need to outsource quantitative research. 

Plenty of businesses do this, but it’s pricey. 

When you’re just starting out or you’re strapped for cash, qualitative data can get you valuable information—quickly and without gouging your wallet. 

#4. Big vs. Small Sample Size

Another reason qualitative data is more accessible? It requires a smaller sample size to achieve meaningful results. 

Even one person’s perspective brings value to a research project—ever heard of a case study?

The sweet spot depends on the purpose of the study, but for qualitative market research, somewhere between 10-40 respondents is a good number. 

Any more than that and you risk reaching saturation. That’s when you keep getting results that echo each other and add nothing new to the research.

Quantitative data needs enough respondents to reach statistical significance without veering into saturation territory. 

The ideal sample size number is usually higher than it is for qualitative data. But as with qualitative data, there’s no single, magic number. It all depends on statistical values like confidence level, population size, and margin of error.

Because it often requires a larger sample size, quantitative research can be more difficult for the average person to do on their own. 

#5. Methods of Analysis

Running a study is just the first part of conducting qualitative and quantitative research. 

After you’ve collected data, you have to study it. Find themes, patterns, consistencies, inconsistencies. Interpret and organize the numbers or survey responses or interview recordings. Tidy it all up into something you can draw conclusions from and apply to various situations. 

This is called data analysis, and it’s done in completely different ways for qualitative vs. quantitative data. 

For qualitative data, analysis includes: 

• Data prep: Make all your qualitative data easy to access and read. This could mean organizing survey results by date, or transcribing interviews, or putting photographs into a slideshow format. 
• Coding: No, not that kind. Think color coding, like you did for your notes in school. Assign colors or codes to specific attributes that make sense for your study—green for positive emotions, for instance, and red for angry emotions. Then code each of your responses. 
• Thematic analysis: Organize your codes into themes and sub-themes, looking for the meaning—and relationships—within each one. 
• Content analysis: Quantify the number of times certain words or concepts appear in your data. If this sounds suspiciously like quantitative research to you, it is. Sort of. It’s looking at qualitative data with a quantitative eye to identify any recurring themes or patterns. 
• Narrative analysis: Look for similar stories and experiences and group them together. Study them and draw inferences from what they say.
• Interpret and document: As you organize and analyze your qualitative data, decide what the findings mean for you and your project.

You can often do qualitative data analysis manually or with tools like NVivo and ATLAS.ti. These tools help you organize, code, and analyze your subjective qualitative data. 

Quantitative data analysis is a lot less subjective. Here’s how it generally goes: 

• Data cleaning: Remove all inconsistencies and inaccuracies from your data. Check for duplicates, incorrect formatting (mistakenly writing a 1.00 value as 10.1, for example), and incomplete numbers. 
• Summarize data with descriptive statistics: Use mean, median, mode, range, and standard deviation to summarize your data. 
• Interpret the data with inferential statistics: This is where it gets more complicated. Instead of simply summarizing stats, you’ll now use complicated mathematical and statistical formulas and tests—t-tests, chi-square tests, analysis of variance (ANOVA), and correlation, for starters—to assign meaning to your data. 

Researchers generally use sophisticated data analysis tools like RapidMiner and Tableau to help them do this work. 

#6. Flexibility 

Quantitative research tends to be less flexible than qualitative research. It relies on structured data collection methods, which researchers must set up well before the study begins.

This rigid structure is part of what makes quantitative data so reliable. But the downside here is that once you start the study, it’s hard to change anything without negatively affecting the results. If something unexpected comes up—or if new questions arise—researchers can’t easily change the scope of the study. 

Qualitative research is a lot more flexible. This is why qualitative data can go deeper than quantitative data. If you’re interviewing someone and an interesting, unexpected topic comes up, you can immediately explore it.

Other qualitative research methods offer flexibility, too. Most big survey software brands allow you to build flexible surveys using branching and skip logic. These features let you customize which questions respondents see based on the answers they give.  

This flexibility is unheard of in quantitative research. But even though it’s as flexible as an Olympic gymnast, qualitative data can be less reliable—and harder to validate. 

#7. Reliability and Validity

Quantitative data is more reliable than qualitative data. Numbers can’t be massaged to fit a certain bias. If you replicate the study—in other words, run the exact same quantitative study two or more times—you should get nearly identical results each time. The same goes if another set of researchers runs the same study using the same methods.

This is what gives quantitative data that reliability factor. 

There are a few key benefits here. First, reliable data means you can confidently make generalizations that apply to a larger population. It also means the data is valid and accurately measures whatever it is you’re trying to measure. 

And finally, reliable data is trustworthy. Big industries like healthcare, marketing, and education frequently use quantitative data to make life-or-death decisions. The more reliable and trustworthy the data, the more confident these decision-makers can be when it’s time to make critical choices. 

Unlike quantitative data, qualitative data isn’t overtly reliable. It’s not easy to replicate. If you send out the same qualitative survey on two separate occasions, you’ll get a new mix of responses. Your interpretations of the data might look different, too. 

There’s still incredible value in qualitative data, of course—and there are ways to make sure the data is valid. These include: 

• Member checking: Circling back with survey, interview, or focus group respondents to make sure you accurately summarized and interpreted their feedback. 
• Triangulation: Using multiple data sources, methods, or researchers to cross-check and corroborate findings.
• Peer debriefing: Showing the data to peers—other researchers—so they can review the research process and its findings and provide feedback on both. 

Whether you’re dealing with qualitative or quantitative data, transparency, accuracy, and validity are crucial. Focus on sourcing (or conducting) quantitative research that’s easy to replicate and qualitative research that’s been peer-reviewed.

With rock-solid data like this, you can make critical business decisions with confidence.

Make your website better. Instantly.

Keep reading about user experience.

dscout Review–The Good and Bad

dscout is a great tool for doing qualitative user research, like live interviews or diary studies. But it isn’t the best choice for everyone.  If…

Out Of All Tips to Improve User Experience, 7 Are Legit

Figuring out the most effective ways to improve the user experience can be hard. There is tons of information out there, and it gets overwhelming…

Is Nominal Data Useful? Yes, In These Situations

Nominal data is descriptive information wherein rank and order don’t matter. Still confused? It helps to contrast nominal data with the other three main types…

What Is User Experience? Answers From 7 Top UX Designers

If you Google user experience the definition you’ll find is “the overall experience of a person using a product like a website or computer application,…

How to Do Each Qualitative Data Coding Type (All Steps)

Qualitative data coding is the process of organizing all the descriptive data you collect during a research project.  It has nothing to do with computer…

7 Qualitative Data Examples and Why They Work

Qualitative data presents information using descriptive language, images, and videos instead of numbers. To help make sense of this type of data—as opposed to quantitative…

The 5 Best Usability Testing Tools Compared

Usability testing helps designers, product managers, and other teams figure out how easily users can use a website, app, or product.  With these tools, user…

5 Qualitative Data Analysis Methods + When To Use Each

Qualitative data analysis is the work of organizing and interpreting descriptive data. Interview recordings, open-ended survey responses, and focus group observations all yield descriptive—qualitative—information. This…

The 5 Best UX Research Tools Compared

UX research tools help designers, product managers, and other teams understand users and how they interact with a company’s products and services. The tools provide…

Qualitative data is information you can describe with words rather than numbers.  Quantitative data is information represented in a measurable way using numbers.  One type…

6 Real Ways AI Has Improved the User Experience

It seems like every other company is bragging about their AI-enhanced user experiences. Consumers and the UX professionals responsible for designing great user experiences are…

12 Key UX Metrics: What They Mean + How To Calculate Each

UX metrics help identify where users struggle when using an app or website and where they are successful. The data collected helps designers, developers, and…

5 Key Principles Of Good Website Usability

Ease of use is a common expectation for a site to be considered well designed. Over the past few years, we have been used to…

20 Ways to Speed Up Your Website and Improve Conversion in 2024

Think that speeding up your website isn’t important? Big mistake. A one-second delay in page load time yields: Your site taking a few extra seconds to…

How to Do Usability Testing Right

User experience is one of the most important aspects of having a successful website, app, piece of software, or any other product that you’ve built. …

Over 300,000 websites use Crazy Egg to improve what's working, fix what isn't and test new ideas.

Last Updated on May 25, 2020

Integrations

What's new?

In-Product Prompts

Participant Management

Interview Studies

Prototype Testing

Card Sorting

Tree Testing

Live Website Testing

Automated Reports

Templates Gallery

Choose from our library of pre-built mazes to copy, customize, and share with your own users

Browse all templates

Financial Services

Tech & Software

Product Designers

Product Managers

User Researchers

By use case

Concept & Idea Validation

Wireframe & Usability Test

Content & Copy Testing

Feedback & Satisfaction

Content Hub

Educational resources for product, research and design teams

Explore all resources

Question Bank

Maze Research Success Hub

Guides & Reports

Help Center

Future of User Research Report

The Optimal Path Podcast

User Research

Aug 19, 2024 • 17 minutes read

Qualitative research examples: How to unlock, rich, descriptive insights

Qualitative research uncovers in-depth user insights, but what does it look like? Here are seven methods and examples to help you get the data you need.

Armin Tanovic

Behind every what, there’s a why . Qualitative research is how you uncover that why. It enables you to connect with users and understand their thoughts, feelings, wants, needs, and pain points.

There’s many methods for conducting qualitative research, and many objectives it can help you pursue—you might want to explore ways to improve NPS scores, combat reduced customer retention, or understand (and recreate) the success behind a well-received product. The common thread? All these metrics impact your business, and qualitative research can help investigate and improve that impact.

In this article, we’ll take you through seven methods and examples of qualitative research, including when and how to use them.

Qualitative UX research made easy

Conduct qualitative research with Maze, analyze data instantly, and get rich, descriptive insights that drive decision-making.

7 Qualitative research methods: An overview

There are various qualitative UX research methods that can help you get in-depth, descriptive insights. Some are suited to specific phases of the design and development process, while others are more task-oriented.

Here’s our overview of the most common qualitative research methods. Keep reading for their use cases, and detailed examples of how to conduct them.

1. User interviews

A user interview is a one-on-one conversation between a UX researcher, designer or Product Manager and a target user to understand their thoughts, perspectives, and feelings on a product or service. User interviews are a great way to get non-numerical data on individual experiences with your product, to gain a deeper understanding of user perspectives.

Interviews can be structured, semi-structured, or unstructured . Structured interviews follow a strict interview script and can help you get answers to your planned questions, while semi and unstructured interviews are less rigid in their approach and typically lead to more spontaneous, user-centered insights.

When to use user interviews

Interviews are ideal when you want to gain an in-depth understanding of your users’ perspectives on your product or service, and why they feel a certain way.

Interviews can be used at any stage in the product design and development process, being particularly helpful during:

• The discovery phase: To better understand user needs, problems, and the context in which they use your product—revealing the best potential solutions
• The design phase: To get contextual feedback on mockups, wireframes, and prototypes, helping you pinpoint issues and the reasons behind them
• Post-launch: To assess if your product continues to meet users’ shifting expectations and understand why or why not

How to conduct user interviews: The basics

• Draft questions based on your research objectives
• Recruit relevant research participants and schedule interviews
• Conduct the interview and transcribe responses
• Analyze the interview responses to extract insights
• Use your findings to inform design, product, and business decisions

💡 A specialized user interview tool makes interviewing easier. With Maze Interview Studies , you can recruit, host, and analyze interviews all on one platform.

User interviews: A qualitative research example

Let’s say you’ve designed a recruitment platform, called Tech2Talent , that connects employers with tech talent. Before starting the design process, you want to clearly understand the pain points employers experience with existing recruitment tools'.

You draft a list of ten questions for a semi-structured interview for 15 different one-on-one interviews. As it’s semi-structured, you don’t expect to ask all the questions—the script serves as more of a guide.

One key question in your script is: “Have tech recruitment platforms helped you find the talent you need in the past?”

Most respondents answer with a resounding and passionate ‘no’ with one of them expanding:

“For our company, it’s been pretty hit or miss honestly. They let just about anyone make a profile and call themselves tech talent. It’s so hard sifting through serious candidates. I can’t see any of their achievements until I invest time setting up an interview.”

You begin to notice a pattern in your responses: recruitment tools often lack easily accessible details on talent profiles.

You’ve gained contextual feedback on why other recruitment platforms fail to solve user needs.

2. Focus groups

A focus group is a research method that involves gathering a small group of people—around five to ten users—to discuss a specific topic, such as their’ experience with your new product feature. Unlike user interviews, focus groups aim to capture the collective opinion of a wider market segment and encourage discussion among the group.

When to use focus groups

You should use focus groups when you need a deeper understanding of your users’ collective opinions. The dynamic discussion among participants can spark in-depth insights that might not emerge from regular interviews.

Focus groups can be used before, during, and after a product launch. They’re ideal:

• Throughout the problem discovery phase: To understand your user segment’s pain points and expectations, and generate product ideas
• Post-launch: To evaluate and understand the collective opinion of your product’s user experience
• When conducting market research: To grasp usage patterns, consumer perceptions, and market opportunities for your product

How to conduct focus group studies: The basics

• Draft prompts to spark conversation, or a series of questions based on your UX research objectives
• Find a group of five to ten users who are representative of your target audience (or a specific user segment) and schedule your focus group session
• Conduct the focus group by talking and listening to users, then transcribe responses
• Analyze focus group responses and extract insights
• Use your findings to inform design decisions

The number of participants can make it difficult to take notes or do manual transcriptions. We recommend using a transcription or a specialized UX research tool , such as Maze, that can automatically create ready-to-share reports and highlight key user insights.

Focus groups: A qualitative research example

You’re a UX researcher at FitMe , a fitness app that creates customized daily workouts for gym-goers. Unlike many other apps, FitMe takes into account the previous day’s workout and aims to create one that allows users to effectively rest different muscles.

However, FitMe has an issue. Users are generating workouts but not completing them. They’re accessing the app, taking the necessary steps to get a workout for the day, but quitting at the last hurdle.

Time to talk to users.

You organize a focus group to get to the root of the drop-off issue. You invite five existing users, all of whom have dropped off at the exact point you’re investigating, and ask them questions to uncover why.

A dialog develops:

Participant 1: “Sometimes I’ll get a workout that I just don’t want to do. Sure, it’s a good workout—but I just don’t want to physically do it. I just do my own thing when that happens.”

Participant 2: “Same here, some of them are so boring. I go to the gym because I love it. It’s an escape.”

Participant 3: “Right?! I get that the app generates the best one for me on that specific day, but I wish I could get a couple of options.”

Participant 4: “I’m the same, there are some exercises I just refuse to do. I’m not coming to the gym to do things I dislike.”

Conducting the focus groups and reviewing the transcripts, you realize that users want options. A workout that works for one gym-goer doesn’t necessarily work for the next.

A possible solution? Adding the option to generate a new workout (that still considers previous workouts)and the ability to blacklist certain exercises, like burpees.

3. Ethnographic research

Ethnographic research is a research method that involves observing and interacting with users in a real-life environment. By studying users in their natural habitat, you can understand how your product fits into their daily lives.

Ethnographic research can be active or passive. Active ethnographic research entails engaging with users in their natural environment and then following up with methods like interviews. Passive ethnographic research involves letting the user interact with the product while you note your observations.

When to use ethnographic research

Ethnographic research is best suited when you want rich insights into the context and environment in which users interact with your product. Keep in mind that you can conduct ethnographic research throughout the entire product design and development process —from problem discovery to post-launch. However, it’s mostly done early in the process:

• Early concept development: To gain an understanding of your user's day-to-day environment. Observe how they complete tasks and the pain points they encounter. The unique demands of their everyday lives will inform how to design your product.
• Initial design phase: Even if you have a firm grasp of the user’s environment, you still need to put your solution to the test. Conducting ethnographic research with your users interacting with your prototype puts theory into practice.

How to conduct ethnographic research:

• Recruit users who are reflective of your audience
• Meet with them in their natural environment, and tell them to behave as they usually would
• Take down field notes as they interact with your product
• Engage with your users, ask questions, or host an in-depth interview if you’re doing an active ethnographic study
• Collect all your data and analyze it for insights

While ethnographic studies provide a comprehensive view of what potential users actually do, they are resource-intensive and logistically difficult. A common alternative is diary studies. Like ethnographic research, diary studies examine how users interact with your product in their day-to-day, but the data is self-reported by participants.

⚙️ Recruiting participants proving tough and time-consuming? Maze Panel makes it easy, with 400+ filters to find your ideal participants from a pool of 3 million participants.

Ethnographic research: A qualitative research example

You're a UX researcher for a project management platform called ProFlow , and you’re conducting an ethnographic study of the project creation process with key users, including a startup’s COO.

The first thing you notice is that the COO is rushing while navigating the platform. You also take note of the 46 tabs and Zoom calls opened on their monitor. Their attention is divided, and they let out an exasperated sigh as they repeatedly hit “refresh” on your website’s onboarding interface.

You conclude the session with an interview and ask, “How easy or difficult did you find using ProFlow to coordinate a project?”

The COO answers: “Look, the whole reason we turn to project platforms is because we need to be quick on our feet. I’m doing a million things so I need the process to be fast and simple. The actual project management is good, but creating projects and setting up tables is way too complicated.”

You realize that ProFlow ’s project creation process takes way too much time for professionals working in fast-paced, dynamic environments. To solve the issue, propose a quick-create option that enables them to move ahead with the basics instead of requiring in-depth project details.

4. Qualitative observation

Qualitative observation is a similar method to ethnographic research, though not as deep. It involves observing your users in a natural or controlled environment and taking notes as they interact with a product. However, be sure not to interrupt them, as this compromises the integrity of the study and turns it into active ethnographic research.

When to qualitative observation

Qualitative observation is best when you want to record how users interact with your product without anyone interfering. Much like ethnographic research, observation is best done during:

• Early concept development: To help you understand your users' daily lives, how they complete tasks, and the problems they deal with. The observations you collect in these instances will help you define a concept for your product.
• Initial design phase: Observing how users deal with your prototype helps you test if they can easily interact with it in their daily environments

How to conduct qualitative observation:

• Recruit users who regularly use your product
• Meet with users in either their natural environment, such as their office, or within a controlled environment, such as a lab
• Observe them and take down field notes based on what you notice

Qualitative observation: An qualitative research example

You’re conducting UX research for Stackbuilder , an app that connects businesses with tools ideal for their needs and budgets. To determine if your app is easy to use for industry professionals, you decide to conduct an observation study.

Sitting in with the participant, you notice they breeze past the onboarding process, quickly creating an account for their company. Yet, after specifying their company’s budget, they suddenly slow down. They open links to each tool’s individual page, confusingly switching from one tab to another. They let out a sigh as they read through each website.

Conducting your observation study, you realize that users find it difficult to extract information from each tool’s website. Based on your field notes, you suggest including a bullet-point summary of each tool directly on your platform.

5. Case study research

Case studies are a UX research method that provides comprehensive and contextual insights into a real-world case over a long period of time. They typically include a range of other qualitative research methods, like interviews, observations, and ethnographic research. A case study allows you to form an in-depth analysis of how people use your product, helping you uncover nuanced differences between your users.

When to use case studies

Case studies are best when your product involves complex interactions that need to be tracked over a longer period or through in-depth analysis. You can also use case studies when your product is innovative, and there’s little existing data on how users interact with it.

As for specific phases in the product design and development process:

• Initial design phase: Case studies can help you rigorously test for product issues and the reasons behind them, giving you in-depth feedback on everything between user motivations, friction points, and usability issues
• Post-launch phase: Continuing with case studies after launch can give you ongoing feedback on how users interact with the product in their day-to-day lives. These insights ensure you can meet shifting user expectations with product updates and future iterations

How to conduct case studies:

• Outline an objective for your case study such as examining specific user tasks or the overall user journey
• Select qualitative research methods such as interviews, ethnographic studies, or observations
• Collect and analyze your data for comprehensive insights
• Include your findings in a report with proposed solutions

Case study research: A qualitative research example

Your team has recently launched Pulse , a platform that analyzes social media posts to identify rising digital marketing trends. Pulse has been on the market for a year, and you want to better understand how it helps small businesses create successful campaigns.

To conduct your case study, you begin with a series of interviews to understand user expectations, ethnographic research sessions, and focus groups. After sorting responses and observations into common themes you notice a main recurring pattern. Users have trouble interpreting the data from their dashboards, making it difficult to identify which trends to follow.

With your synthesized insights, you create a report with detailed narratives of individual user experiences, common themes and issues, and recommendations for addressing user friction points.

Some of your proposed solutions include creating intuitive graphs and summaries for each trend study. This makes it easier for users to understand trends and implement strategic changes in their campaigns.

6. Secondary research

Secondary research is a research method that involves collecting and analyzing documents, records, and reviews that provide you with contextual data on your topic. You’re not connecting with participants directly, but rather accessing pre-existing available data. For example, you can pull out insights from your UX research repository to reexamine how they apply to your new UX research objective.

Strictly speaking, it can be both qualitative and quantitative—but today we focus on its qualitative application.

When to use secondary research

Record keeping is particularly useful when you need supplemental insights to complement, validate, or compare current research findings. It helps you analyze shifting trends amongst your users across a specific period. Some other scenarios where you need record keeping include:

• Initial discovery or exploration phase: Secondary research can help you quickly gather background information and data to understand the broader context of a market
• Design and development phase: See what solutions are working in other contexts for an idea of how to build yours

Secondary research is especially valuable when your team faces budget constraints, tight deadlines, or limited resources. Through review mining and collecting older findings, you can uncover useful insights that drive decision-making throughout the product design and development process.

How to conduct secondary research:

• Outline your UX research objective
• Identify potential data sources for information on your product, market, or target audience. Some of these sources can include: a. Review websites like Capterra and G2 b. Social media channels c. Customer service logs and disputes d. Website reviews e. Reports and insights from previous research studies f. Industry trends g. Information on competitors
• Analyze your data by identifying recurring patterns and themes for insights

Secondary research: A qualitative research example

SafeSurf is a cybersecurity platform that offers threat detection, security audits, and real-time reports. After conducting multiple rounds of testing, you need a quick and easy way to identify remaining usability issues. Instead of conducting another resource-intensive method, you opt for social listening and data mining for your secondary research.

Browsing through your company’s X, you identify a recurring theme: many users without a background in tech find SafeSurf ’s reports too technical and difficult to read. Users struggle with understanding what to do if their networks are breached.

After checking your other social media channels and review sites, the issue pops up again.

With your gathered insights, your team settles on introducing a simplified version of reports, including clear summaries, takeaways, and step-by-step protocols for ensuring security.

By conducting secondary research, you’ve uncovered a major usability issue—all without spending large amounts of time and resources to connect with your users.

7. Open-ended surveys

Open-ended surveys are a type of unmoderated UX research method that involves asking users to answer a list of qualitative research questions designed to uncover their attitudes, expectations, and needs regarding your service or product. Open-ended surveys allow users to give in-depth, nuanced, and contextual responses.

When to use open-ended surveys

User surveys are an effective qualitative research method for reaching a large number of users. You can use them at any stage of the design and product development process, but they’re particularly useful:

• When you’re conducting generative research : Open-ended surveys allow you to reach a wide range of users, making them especially useful during initial research phases when you need broad insights into user experiences
• When you need to understand customer satisfaction: Open-ended customer satisfaction surveys help you uncover why your users might be dissatisfied with your product, helping you find the root cause of their negative experiences
• In combination with close-ended surveys: Get a combination of numerical, statistical insights and rich descriptive feedback. You’ll know what a specific percentage of your users think and why they think it.

How to conduct open-ended surveys:

• Design your survey and draft out a list of survey questions
• Distribute your surveys to respondents
• Analyze survey participant responses for key themes and patterns
• Use your findings to inform your design process

Open-ended surveys: A qualitative research example

You're a UX researcher for RouteReader , a comprehensive logistics platform that allows users to conduct shipment tracking and route planning. Recently, you’ve launched a new predictive analytics feature that allows users to quickly identify and prepare for supply chain disruptions.

To better understand if users find the new feature helpful, you create an open-ended, in-app survey.

The questions you ask your users:

• “What has been your experience with our new predictive analytics feature?"
• “Do you find it easy or difficult to rework your routes based on our predictive suggestions?”
• “Does the predictive analytics feature make planning routes easier? Why or why not?”

Most of the responses are positive. Users report using the predictive analytics feature to make last-minute adjustments to their route plans, and some even rely on it regularly. However, a few users find the feature hard to notice, making it difficult to adjust their routes on time.

To ensure users have supply chain insights on time, you integrate the new feature into each interface so users can easily spot important information and adjust their routes accordingly.

💡 Surveys are a lot easier with a quality survey tool. Maze’s Feedback Surveys solution has all you need to ensure your surveys get the insights you need—including AI-powered follow-up and automated reports.

Qualitative research vs. quantitative research: What’s the difference?

Alongside qualitative research approaches, UX teams also use quantitative research methods. Despite the similar names, the two are very different.

Here are some of the key differences between qualitative research and quantitative research .

Before selecting either qualitative or quantitative methods, first identify what you want to achieve with your UX research project. As a general rule of thumb, think qualitative data collection for in-depth understanding and quantitative studies for measurement and validation.

Conduct qualitative research with Maze

You’ll often find that knowing the what is pointless without understanding the accompanying why . Qualitative research helps you uncover your why.

So, what about how —how do you identify your 'what' and your 'why'?

The answer is with a user research tool like Maze.

Maze is the leading user research platform that lets you organize, conduct, and analyze both qualitative and quantitative research studies—all from one place. Its wide variety of UX research methods and advanced AI capabilities help you get the insights you need to build the right products and experiences faster.

Frequently asked questions about qualitative research examples

What is qualitative research?

Qualitative research is a research method that aims to provide contextual, descriptive, and non-numerical insights on a specific issue. Qualitative research methods like interviews, case studies, and ethnographic studies allow you to uncover the reasoning behind your user’s attitudes and opinions.

Can a study be both qualitative and quantitative?

Absolutely! You can use mixed methods in your research design, which combines qualitative and quantitative approaches to gain both descriptive and statistical insights.

For example, user surveys can have both close-ended and open-ended questions, providing comprehensive data like percentages of user views and descriptive reasoning behind their answers.

Is qualitative or quantitative research better?

The choice between qualitative and quantitative research depends upon your research goals and objectives.

Qualitative research methods are better suited when you want to understand the complexities of your user’s problems and uncover the underlying motives beneath their thoughts, feelings, and behaviors. Quantitative research excels in giving you numerical data, helping you gain a statistical view of your user's attitudes, identifying trends, and making predictions.

What are some approaches to qualitative research?

There are many approaches to qualitative studies. An approach is the underlying theory behind a method, and a method is a way of implementing the approach. Here are some approaches to qualitative research:

• Grounded theory: Researchers study a topic and develop theories inductively
• Phenomenological research: Researchers study a phenomenon through the lived experiences of those involved
• Ethnography: Researchers immerse themselves in organizations to understand how they operate

Educational resources and simple solutions for your research journey

Qualitative vs Quantitative Research: Differences, Examples, and Methods

There are two broad kinds of research approaches: qualitative and quantitative research that are used to study and analyze phenomena in various fields such as natural sciences, social sciences, and humanities. Whether you have realized it or not, your research must have followed either or both research types. In this article we will discuss what qualitative vs quantitative research is, their applications, pros and cons, and when to use qualitative vs quantitative research . Before we get into the details, it is important to understand the differences between the qualitative and quantitative research.     

Table of Contents

Qualitative v s Quantitative Research  

Quantitative research deals with quantity, hence, this research type is concerned with numbers and statistics to prove or disapprove theories or hypothesis. In contrast, qualitative research is all about quality – characteristics, unquantifiable features, and meanings to seek deeper understanding of behavior and phenomenon. These two methodologies serve complementary roles in the research process, each offering unique insights and methods suited to different research questions and objectives.    

Qualitative and quantitative research approaches have their own unique characteristics, drawbacks, advantages, and uses. Where quantitative research is mostly employed to validate theories or assumptions with the goal of generalizing facts to the larger population, qualitative research is used to study concepts, thoughts, or experiences for the purpose of gaining the underlying reasons, motivations, and meanings behind human behavior .   

What Are the Differences Between Qualitative and Quantitative Research  

Qualitative and quantitative research differs in terms of the methods they employ to conduct, collect, and analyze data. For example, qualitative research usually relies on interviews, observations, and textual analysis to explore subjective experiences and diverse perspectives. While quantitative data collection methods include surveys, experiments, and statistical analysis to gather and analyze numerical data. The differences between the two research approaches across various aspects are listed in the table below.    

Data Collection Methods  

There are differences between qualitative and quantitative research when it comes to data collection as they deal with different types of data. Qualitative research is concerned with personal or descriptive accounts to understand human behavior within society. Quantitative research deals with numerical or measurable data to delineate relations among variables. Hence, the qualitative data collection methods differ significantly from quantitative data collection methods due to the nature of data being collected and the research objectives. Below is the list of data collection methods for each research approach:    

Qualitative Research Data Collection  

• Interviews  
• Focus g roups  
• Content a nalysis  
• Literature review  
• Observation  
• Ethnography  

Qualitative research data collection can involve one-on-one group interviews to capture in-depth perspectives of participants using open-ended questions. These interviews could be structured, semi-structured or unstructured depending upon the nature of the study. Focus groups can be used to explore specific topics and generate rich data through discussions among participants. Another qualitative data collection method is content analysis, which involves systematically analyzing text documents, audio, and video files or visual content to uncover patterns, themes, and meanings. This can be done through coding and categorization of raw data to draw meaningful insights. Data can be collected through observation studies where the goal is to simply observe and document behaviors, interaction, and phenomena in natural settings without interference. Lastly, ethnography allows one to immerse themselves in the culture or environment under study for a prolonged period to gain a deep understanding of the social phenomena.   

Quantitative Research Data Collection  

• Surveys/ q uestionnaires  
• Experiments
• Secondary data analysis  
• Structured o bservations  
• Case studies   
• Tests and a ssessments  

Quantitative research data collection approaches comprise of fundamental methods for generating numerical data that can be analyzed using statistical or mathematical tools. The most common quantitative data collection approach is the usage of structured surveys with close-ended questions to collect quantifiable data from a large sample of participants. These can be conducted online, over the phone, or in person.   

Performing experiments is another important data collection approach, in which variables are manipulated under controlled conditions to observe their effects on dependent variables. This often involves random assignment of participants to different conditions or groups. Such experimental settings are employed to gauge cause-and-effect relationships and understand a complex phenomenon. At times, instead of acquiring original data, researchers may deal with secondary data, which is the dataset curated by others, such as government agencies, research organizations, or academic institute. With structured observations, subjects in a natural environment can be studied by controlling the variables which aids in understanding the relationship among various variables. The secondary data is then analyzed to identify patterns and relationships among variables. Observational studies provide a means to systematically observe and record behaviors or phenomena as they occur in controlled environments. Case studies form an interesting study methodology in which a researcher studies a single entity or a small number of entities (individuals or organizations) in detail to understand complex phenomena within a specific context.   

Qualitative vs Quantitative Research Outcomes  

Qualitative research and quantitative research lead to varied research outcomes, each with its own strengths and limitations. For example, qualitative research outcomes provide deep descriptive accounts of human experiences, motivations, and perspectives that allow us to identify themes or narratives and context in which behavior, attitudes, or phenomena occurs.  Quantitative research outcomes on the other hand produce numerical data that is analyzed statistically to establish patterns and relationships objectively, to form generalizations about the larger population and make predictions. This numerical data can be presented in the form of graphs, tables, or charts. Both approaches offer valuable perspectives on complex phenomena, with qualitative research focusing on depth and interpretation, while quantitative research emphasizes numerical analysis and objectivity.  

When to Use Qualitative vs Quantitative Research Approach  

The decision to choose between qualitative and quantitative research depends on various factors, such as the research question, objectives, whether you are taking an inductive or deductive approach, available resources, practical considerations such as time and money, and the nature of the phenomenon under investigation. To simplify, quantitative research can be used if the aim of the research is to prove or test a hypothesis, while qualitative research should be used if the research question is more exploratory and an in-depth understanding of the concepts, behavior, or experiences is needed.     

Qualitative research approach  

Qualitative research approach is used under following scenarios:   

• To study complex phenomena: When the research requires understanding the depth, complexity, and context of a phenomenon.  
• Collecting participant perspectives: When the goal is to understand the why behind a certain behavior, and a need to capture subjective experiences and perceptions of participants.  
• Generating hypotheses or theories: When generating hypotheses, theories, or conceptual frameworks based on exploratory research.  

Example: If you have a research question “What obstacles do expatriate students encounter when acquiring a new language in their host country?”  

This research question can be addressed using the qualitative research approach by conducting in-depth interviews with 15-25 expatriate university students. Ask open-ended questions such as “What are the major challenges you face while attempting to learn the new language?”, “Do you find it difficult to learn the language as an adult?”, and “Do you feel practicing with a native friend or colleague helps the learning process”?  

Based on the findings of these answers, a follow-up questionnaire can be planned to clarify things. Next step will be to transcribe all interviews using transcription software and identify themes and patterns.   

Quantitative research approach  

Quantitative research approach is used under following scenarios:   

• Testing hypotheses or proving theories: When aiming to test hypotheses, establish relationships, or examine cause-and-effect relationships.   
• Generalizability: When needing findings that can be generalized to broader populations using large, representative samples.  
• Statistical analysis: When requiring rigorous statistical analysis to quantify relationships, patterns, or trends in data.   

Example : Considering the above example, you can conduct a survey of 200-300 expatriate university students and ask them specific questions such as: “On a scale of 1-10 how difficult is it to learn a new language?”  

Next, statistical analysis can be performed on the responses to draw conclusions like, on an average expatriate students rated the difficulty of learning a language 6.5 on the scale of 10.    

Mixed methods approach  

In many cases, researchers may opt for a mixed methods approach , combining qualitative and quantitative methods to leverage the strengths of both approaches. Researchers may use qualitative data to explore phenomena in-depth and generate hypotheses, while quantitative data can be used to test these hypotheses and generalize findings to broader populations.  

Example: Both qualitative and quantitative research methods can be used in combination to address the above research question. Through open-ended questions you can gain insights about different perspectives and experiences while quantitative research allows you to test that knowledge and prove/disprove your hypothesis.   

How to Analyze Qualitative and Quantitative Data  

When it comes to analyzing qualitative and quantitative data, the focus is on identifying patterns in the data to highlight the relationship between elements. The best research method for any given study should be chosen based on the study aim. A few methods to analyze qualitative and quantitative data are listed below.  

Analyzing qualitative data  

Qualitative data analysis is challenging as it is not expressed in numbers and consists majorly of texts, images, or videos. Hence, care must be taken while using any analytical approach. Some common approaches to analyze qualitative data include:  

• Organization: The first step is data (transcripts or notes) organization into different categories with similar concepts, themes, and patterns to find inter-relationships.  
• Coding: Data can be arranged in categories based on themes/concepts using coding.  
• Theme development: Utilize higher-level organization to group related codes into broader themes.  
• Interpretation: Explore the meaning behind different emerging themes to understand connections. Use different perspectives like culture, environment, and status to evaluate emerging themes.  
• Reporting: Present findings with quotes or excerpts to illustrate key themes.   

Analyzing quantitative data  

Quantitative data analysis is more direct compared to qualitative data as it primarily deals with numbers. Data can be evaluated using simple math or advanced statistics (descriptive or inferential). Some common approaches to analyze quantitative data include:  

• Processing raw data: Check missing values, outliers, or inconsistencies in raw data.  
• Descriptive statistics: Summarize data with means, standard deviations, or standard error using programs such as Excel, SPSS, or R language.  
• Exploratory data analysis: Usage of visuals to deduce patterns and trends.  
• Hypothesis testing: Apply statistical tests to find significance and test hypothesis (Student’s t-test or ANOVA).  
• Interpretation: Analyze results considering significance and practical implications.  
• Validation: Data validation through replication or literature review.  
• Reporting: Present findings by means of tables, figures, or graphs.   

Benefits and limitations of qualitative vs quantitative research  

There are significant differences between qualitative and quantitative research; we have listed the benefits and limitations of both methods below:  

Benefits of qualitative research  

• Rich insights: As qualitative research often produces information-rich data, it aids in gaining in-depth insights into complex phenomena, allowing researchers to explore nuances and meanings of the topic of study.  
• Flexibility: One of the most important benefits of qualitative research is flexibility in acquiring and analyzing data that allows researchers to adapt to the context and explore more unconventional aspects.  
• Contextual understanding: With descriptive and comprehensive data, understanding the context in which behaviors or phenomena occur becomes accessible.   
• Capturing different perspectives: Qualitative research allows for capturing different participant perspectives with open-ended question formats that further enrich data.   
• Hypothesis/theory generation: Qualitative research is often the first step in generating theory/hypothesis, which leads to future investigation thereby contributing to the field of research.

Limitations of qualitative research  

• Subjectivity: It is difficult to have objective interpretation with qualitative research, as research findings might be influenced by the expertise of researchers. The risk of researcher bias or interpretations affects the reliability and validity of the results.   
• Limited generalizability: Due to the presence of small, non-representative samples, the qualitative data cannot be used to make generalizations to a broader population.  
• Cost and time intensive: Qualitative data collection can be time-consuming and resource-intensive, therefore, it requires strategic planning and commitment.   
• Complex analysis: Analyzing qualitative data needs specialized skills and techniques, hence, it’s challenging for researchers without sufficient training or experience.   
• Potential misinterpretation: There is a risk of sampling bias and misinterpretation in data collection and analysis if researchers lack cultural or contextual understanding.   

Benefits of quantitative research  

• Objectivity: A key benefit of quantitative research approach, this objectivity reduces researcher bias and subjectivity, enhancing the reliability and validity of findings.   
• Generalizability: For quantitative research, the sample size must be large and representative enough to allow for generalization to broader populations.   
• Statistical analysis: Quantitative research enables rigorous statistical analysis (increasing power of the analysis), aiding hypothesis testing and finding patterns or relationship among variables.   
• Efficiency: Quantitative data collection and analysis is usually more efficient compared to the qualitative methods, especially when dealing with large datasets.   
• Clarity and Precision: The findings are usually clear and precise, making it easier to present them as graphs, tables, and figures to convey them to a larger audience.  

Limitations of quantitative research  

• Lacks depth and details: Due to its objective nature, quantitative research might lack the depth and richness of qualitative approaches, potentially overlooking important contextual factors or nuances.   
• Limited exploration: By not considering the subjective experiences of participants in depth , there’s a limited chance to study complex phenomenon in detail.   
• Potential oversimplification: Quantitative research may oversimplify complex phenomena by boiling them down to numbers, which might ignore key nuances.   
• Inflexibility: Quantitative research deals with predecided varibales and measures , which limits the ability of researchers to explore unexpected findings or adjust the research design as new findings become available .  
• Ethical consideration: Quantitative research may raise ethical concerns especially regarding privacy, informed consent, and the potential for harm, when dealing with sensitive topics or vulnerable populations.   

Frequently asked questions  

• What is the difference between qualitative and quantitative research? 

Quantitative methods use numerical data and statistical analysis for objective measurement and hypothesis testing, emphasizing generalizability. Qualitative methods gather non-numerical data to explore subjective experiences and contexts, providing rich, nuanced insights.  

• What are the types of qualitative research? 

Qualitative research methods include interviews, observations, focus groups, and case studies. They provide rich insights into participants’ perspectives and behaviors within their contexts, enabling exploration of complex phenomena.  

• What are the types of quantitative research? 

Quantitative research methods include surveys, experiments, observations, correlational studies, and longitudinal research. They gather numerical data for statistical analysis, aiming for objectivity and generalizability.  

• Can you give me examples for qualitative and quantitative research? 

Qualitative Research Example: 

Research Question: What are the experiences of parents with autistic children in accessing support services?  

Method: Conducting in-depth interviews with parents to explore their perspectives, challenges, and needs.  

Quantitative Research Example: 

Research Question: What is the correlation between sleep duration and academic performance in college students?  

Method: Distributing surveys to a large sample of college students to collect data on their sleep habits and academic performance, then analyzing the data statistically to determine any correlations.  

Editage All Access is a subscription-based platform that unifies the best AI tools and services designed to speed up, simplify, and streamline every step of a researcher’s journey. The Editage All Access Pack is a one-of-a-kind subscription that unlocks full access to an AI writing assistant, literature recommender, journal finder, scientific illustration tool, and exclusive discounts on professional publication services from Editage.  

Based on 22+ years of experience in academia, Editage All Access empowers researchers to put their best research forward and move closer to success. Explore our top AI Tools pack, AI Tools + Publication Services pack, or Build Your Own Plan. Find everything a researcher needs to succeed, all in one place –  Get All Access now starting at just $14 a month !    

Related Posts

Back to School – Lock-in All Access Pack for a Year at the Best Price

Research Paper Appendix: Format and Examples

• MS in the Learning Sciences
• Tuition & Financial Aid

Qualitative vs. quantitative data analysis: How do they differ?

Learning analytics have become the cornerstone for personalizing student experiences and enhancing learning outcomes. In this data-informed approach to education there are two distinct methodologies: qualitative and quantitative analytics. These methods, which are typical to data analytics in general, are crucial to the interpretation of learning behaviors and outcomes. This blog will explore the nuances that distinguish qualitative and quantitative research, while uncovering their shared roles in learning analytics, program design and instruction.

What is qualitative data?

Qualitative data is descriptive and includes information that is non numerical. Qualitative research is used to gather in-depth insights that can't be easily measured on a scale like opinions, anecdotes and emotions. In learning analytics qualitative data could include in depth interviews, text responses to a prompt, or a video of a class period. 1

What is quantitative data?

Quantitative data is information that has a numerical value. Quantitative research is conducted to gather measurable data used in statistical analysis. Researchers can use quantitative studies to identify patterns and trends. In learning analytics quantitative data could include test scores, student demographics, or amount of time spent in a lesson. 2

Key difference between qualitative and quantitative data

It's important to understand the differences between qualitative and quantitative data to both determine the appropriate research methods for studies and to gain insights that you can be confident in sharing.

Data Types and Nature

Examples of qualitative data types in learning analytics:

• Observational data of human behavior from classroom settings such as student engagement, teacher-student interactions, and classroom dynamics
• Textual data from open-ended survey responses, reflective journals, and written assignments
• Feedback and discussions from focus groups or interviews
• Content analysis from various media

Examples of quantitative data types:

• Standardized test, assessment, and quiz scores
• Grades and grade point averages
• Attendance records
• Time spent on learning tasks
• Data gathered from learning management systems (LMS), including login frequency, online participation, and completion rates of assignments

Methods of Collection

Qualitative and quantitative research methods for data collection can occasionally seem similar so it's important to note the differences to make sure you're creating a consistent data set and will be able to reliably draw conclusions from your data.

Qualitative research methods

Because of the nature of qualitative data (complex, detailed information), the research methods used to collect it are more involved. Qualitative researchers might do the following to collect data:

• Conduct interviews to learn about subjective experiences
• Host focus groups to gather feedback and personal accounts
• Observe in-person or use audio or video recordings to record nuances of human behavior in a natural setting
• Distribute surveys with open-ended questions

Quantitative research methods

Quantitative data collection methods are more diverse and more likely to be automated because of the objective nature of the data. A quantitative researcher could employ methods such as:

• Surveys with close-ended questions that gather numerical data like birthdates or preferences
• Observational research and record measurable information like the number of students in a classroom
• Automated numerical data collection like information collected on the backend of a computer system like button clicks and page views

Analysis techniques

Qualitative and quantitative data can both be very informative. However, research studies require critical thinking for productive analysis.

Qualitative data analysis methods

Analyzing qualitative data takes a number of steps. When you first get all your data in one place you can do a review and take notes of trends you think you're seeing or your initial reactions. Next, you'll want to organize all the qualitative data you've collected by assigning it categories. Your central research question will guide your data categorization whether it's by date, location, type of collection method (interview vs focus group, etc), the specific question asked or something else. Next, you'll code your data. Whereas categorizing data is focused on the method of collection, coding is the process of identifying and labeling themes within the data collected to get closer to answering your research questions. Finally comes data interpretation. To interpret the data you'll take a look at the information gathered including your coding labels and see what results are occurring frequently or what other conclusions you can make. 3

Quantitative analysis techniques

The process to analyze quantitative data can be time-consuming due to the large volume of data possible to collect. When approaching a quantitative data set, start by focusing in on the purpose of your evaluation. Without making a conclusion, determine how you will use the information gained from analysis; for example: The answers of this survey about study habits will help determine what type of exam review session will be most useful to a class. 4

Next, you need to decide who is analyzing the data and set parameters for analysis. For example, if two different researchers are evaluating survey responses that rank preferences on a scale from 1 to 5, they need to be operating with the same understanding of the rankings. You wouldn't want one researcher to classify the value of 3 to be a positive preference while the other considers it a negative preference. It's also ideal to have some type of data management system to store and organize your data, such as a spreadsheet or database. Within the database, or via an export to data analysis software, the collected data needs to be cleaned of things like responses left blank, duplicate answers from respondents, and questions that are no longer considered relevant. Finally, you can use statistical software to analyze data (or complete a manual analysis) to find patterns and summarize your findings. 4

Qualitative and quantitative research tools

From the nuanced, thematic exploration enabled by tools like NVivo and ATLAS.ti, to the statistical precision of SPSS and R for quantitative analysis, each suite of data analysis tools offers tailored functionalities that cater to the distinct natures of different data types.

Qualitative research software:

NVivo: NVivo is qualitative data analysis software that can do everything from transcribe recordings to create word clouds and evaluate uploads for different sentiments and themes. NVivo is just one tool from the company Lumivero, which offers whole suites of data processing software. 5

ATLAS.ti: Similar to NVivo, ATLAS.ti allows researchers to upload and import data from a variety of sources to be tagged and refined using machine learning and presented with visualizations and ready for insert into reports. 6

SPSS: SPSS is a statistical analysis tool for quantitative research, appreciated for its user-friendly interface and comprehensive statistical tests, which makes it ideal for educators and researchers. With SPSS researchers can manage and analyze large quantitative data sets, use advanced statistical procedures and modeling techniques, predict customer behaviors, forecast market trends and more. 7

R: R is a versatile and dynamic open-source tool for quantitative analysis. With a vast repository of packages tailored to specific statistical methods, researchers can perform anything from basic descriptive statistics to complex predictive modeling. R is especially useful for its ability to handle large datasets, making it ideal for educational institutions that generate substantial amounts of data. The programming language offers flexibility in customizing analysis and creating publication-quality visualizations to effectively communicate results. 8

Applications in Educational Research

Both quantitative and qualitative data can be employed in learning analytics to drive informed decision-making and pedagogical enhancements. In the classroom, quantitative data like standardized test scores and online course analytics create a foundation for assessing and benchmarking student performance and engagement. Qualitative insights gathered from surveys, focus group discussions, and reflective student journals offer a more nuanced understanding of learners' experiences and contextual factors influencing their education. Additionally feedback and practical engagement metrics blend these data types, providing a holistic view that informs curriculum development, instructional strategies, and personalized learning pathways. Through these varied data sets and uses, educators can piece together a more complete narrative of student success and the impacts of educational interventions.

Master Data Analysis with an M.S. in Learning Sciences From SMU

Whether it is the detailed narratives unearthed through qualitative data or the informative patterns derived from quantitative analysis, both qualitative and quantitative data can provide crucial information for educators and researchers to better understand and improve learning. Dive deeper into the art and science of learning analytics with SMU's online Master of Science in the Learning Sciences program . At SMU, innovation and inquiry converge to empower the next generation of educators and researchers. Choose the Learning Analytics Specialization to learn how to harness the power of data science to illuminate learning trends, devise impactful strategies, and drive educational innovation. You could also find out how advanced technologies like augmented reality (AR), virtual reality (VR), and artificial intelligence (AI) can revolutionize education, and develop the insight to apply embodied cognition principles to enhance learning experiences in the Learning and Technology Design Specialization , or choose your own electives to build a specialization unique to your interests and career goals.

For more information on our curriculum and to become part of a community where data drives discovery, visit SMU's MSLS program website or schedule a call with our admissions outreach advisors for any queries or further discussion. Take the first step towards transforming education with data today.

• Retrieved on August 8, 2024, from nnlm.gov/guides/data-glossary/qualitative-data
• Retrieved on August 8, 2024, from nnlm.gov/guides/data-glossary/quantitative-data
• Retrieved on August 8, 2024, from cdc.gov/healthyyouth/evaluation/pdf/brief19.pdf
• Retrieved on August 8, 2024, from cdc.gov/healthyyouth/evaluation/pdf/brief20.pdf
• Retrieved on August 8, 2024, from lumivero.com/solutions/
• Retrieved on August 8, 2024, from atlasti.com/
• Retrieved on August 8, 2024, from ibm.com/products/spss-statistics
• Retrieved on August 8, 2024, from cran.r-project.org/doc/manuals/r-release/R-intro.html#Introduction-and-preliminaries

Return to SMU Online Learning Sciences Blog

Southern Methodist University has engaged Everspring , a leading provider of education and technology services, to support select aspects of program delivery.

This will only take a moment

• Usability testing

Run remote usability tests on any digital product to deep dive into your key user flows

• Product analytics

Learn how users are behaving on your website in real time and uncover points of frustration

• Research repository

A tool for collaborative analysis of qualitative data and for building your research repository and database.

• Trymata Blog

How-to articles, expert tips, and the latest news in user testing & user experience

• Knowledge Hub

Detailed explainers of Trymata’s features & plans, and UX research terms & topics

• Plans & Pricing

Get paid to test

• User Experience (UX) testing
• User Interface (UI) testing
• Ecommerce testing
• Remote usability testing
• Plans & Pricing
• Customer Stories

How do you want to use Trymata?

Conduct user testing, desktop usability video.

You’re on a business trip in Oakland, CA. You've been working late in downtown and now you're looking for a place nearby to grab a late dinner. You decided to check Zomato to try and find somewhere to eat. (Don't begin searching yet).

• Look around on the home page. Does anything seem interesting to you?
• How would you go about finding a place to eat near you in Downtown Oakland? You want something kind of quick, open late, not too expensive, and with a good rating.
• What do the reviews say about the restaurant you've chosen?
• What was the most important factor for you in choosing this spot?
• You're currently close to the 19th St Bart station, and it's 9PM. How would you get to this restaurant? Do you think you'll be able to make it before closing time?
• Your friend recommended you to check out a place called Belly while you're in Oakland. Try to find where it is, when it's open, and what kind of food options they have.
• Now go to any restaurant's page and try to leave a review (don't actually submit it).

What was the worst thing about your experience?

It was hard to find the bart station. The collections not being able to be sorted was a bit of a bummer

What other aspects of the experience could be improved?

Feedback from the owners would be nice

What did you like about the website?

The flow was good, lots of bright photos

What other comments do you have for the owner of the website?

I like that you can sort by what you are looking for and i like the idea of collections

You're going on a vacation to Italy next month, and you want to learn some basic Italian for getting around while there. You decided to try Duolingo.

• Please begin by downloading the app to your device.
• Choose Italian and get started with the first lesson (stop once you reach the first question).
• Now go all the way through the rest of the first lesson, describing your thoughts as you go.
• Get your profile set up, then view your account page. What information and options are there? Do you feel that these are useful? Why or why not?
• After a week in Italy, you're going to spend a few days in Austria. How would you take German lessons on Duolingo?
• What other languages does the app offer? Do any of them interest you?

I felt like there could have been a little more of an instructional component to the lesson.

It would be cool if there were some feature that could allow two learners studying the same language to take lessons together. I imagine that their screens would be synced and they could go through lessons together and chat along the way.

Overall, the app was very intuitive to use and visually appealing. I also liked the option to connect with others.

Overall, the app seemed very helpful and easy to use. I feel like it makes learning a new language fun and almost like a game. It would be nice, however, if it contained more of an instructional portion.

All accounts, tests, and data have been migrated to our new & improved system!

Use the same email and password to log in:

Legacy login: Our legacy system is still available in view-only mode, login here >

What’s the new system about? Read more about our transition & what it-->

What is Qualitative Research Design? Definition, Types, Examples and Best Practices

What is Qualitative Research Design?

Qualitative research design is defined as a systematic and flexible approach to conducting research that focuses on understanding and interpreting the complexity of human phenomena. 

Unlike quantitative research, which seeks to measure and quantify variables, qualitative research is concerned with exploring the underlying meanings, patterns, and perspectives that shape individuals’ experiences and behaviors. This type of research design is particularly useful when studying social and cultural phenomena, as it allows researchers to delve deeply into the context and nuances of a particular subject.

In qualitative research, data is often collected through methods such as interviews, focus groups, participant observation, and document analysis. These methods aim to gather rich, detailed information that can provide insights into the subjective experiences of individuals or groups. 

Researchers employing qualitative design are often interested in exploring social processes, cultural norms, and the lived experiences of participants. The emphasis is on understanding the depth and context of the phenomena under investigation, rather than generating statistical generalizations.

One key characteristic of qualitative research design is its iterative nature. The research process is dynamic and may evolve as new insights emerge. Researchers continually engage with the data, refining their questions and methods based on ongoing analysis. 

This flexibility allows for a more organic and responsive exploration of the research topic, making it well-suited for complex and multifaceted inquiries.

Qualitative research design also involves careful consideration of ethical concerns, as researchers often work closely with participants to gather personal and sensitive information. 

Establishing trust, maintaining confidentiality, and ensuring participants’ autonomy are critical aspects of ethical practice in qualitative research. In summary, qualitative research design is a holistic and interpretive approach that prioritizes understanding the intricacies of human experience, offering depth and context to our comprehension of social and cultural phenomena.

Key Characteristics of Qualitative Research Design

Qualitative research design is characterized by several key features that distinguish it from quantitative approaches. Here are some of the essential characteristics:

• Open-ended Nature: Qualitative research is open-ended and flexible, allowing for the exploration of complex social phenomena without preconceived hypotheses. Researchers often start with broad questions and adapt their focus based on emerging insights.
• Rich Descriptions: Qualitative research emphasizes rich and detailed descriptions of the subject under investigation. This depth helps capture the context, nuances, and subtleties of human experiences, behaviors, and social phenomena.
• Subjective Understanding: Qualitative researchers acknowledge the role of the researcher in shaping the study. The subjective interpretations and perspectives of both researchers and participants are considered valuable for understanding the phenomena being studied.
• Interpretive Approach: Rather than seeking universal laws or generalizations, qualitative research aims to interpret and make sense of the meanings and patterns inherent in the data. Interpretation is often context-dependent and involves understanding the social and cultural context in which the study takes place.
• Non-probability Sampling: Qualitative studies typically use non-probability sampling methods, such as purposeful or snowball sampling, to select participants deliberately chosen for their relevance to the research question. Sample sizes are often small but information-rich, allowing for a deep understanding of the selected cases.
• Inductive Reasoning: Qualitative data analysis is often inductive, meaning that it involves identifying patterns, themes, and categories that emerge from the data itself. Researchers let the data shape the analysis, rather than fitting it into preconceived categories.
• Coding and Categorization: Researchers use coding techniques to systematically organize and categorize data. This involves assigning labels or codes to segments of data based on recurring themes or patterns.
• Flexible Design: Qualitative research design is adaptable and allows for changes in research questions, methods, and strategies as the study progresses. This flexibility accommodates the evolving nature of the research process.
• Iterative Nature: Researchers engage in an iterative process of data collection, analysis, and refinement. As new insights emerge, researchers may revisit previous stages of the research, leading to a deeper and more nuanced understanding of the subject.

By embracing these key characteristics, qualitative research design offers a holistic and contextualized approach to studying the complexities of human behavior, culture, and social phenomena.

Key Components of Qualitative Research Design

Qualitative research design involves several key components that shape the overall framework and methodology of the study. These components help guide researchers in conducting in-depth investigations into the complexities of human experiences, behaviors, and social phenomena. Here are the key components of qualitative research design:

• Central Inquiry: Qualitative research begins with a well-defined central research question or objective. This question guides the entire study and determines the focus of data collection and analysis. The question is often broad and open-ended to allow for exploration and discovery.
• Rationale: Researchers provide a clear rationale for why the study is being conducted, outlining its significance and relevance. This may involve identifying gaps in existing literature, addressing practical problems, or contributing to theoretical debates.
• Theoretical Framework: Qualitative studies often draw on existing theories or conceptual frameworks to guide their inquiry. The theoretical lens helps shape the research design and provides a basis for interpreting findings.
• Study Design: Researchers decide on the overall approach to the study, whether it’s a case study, ethnography, grounded theory, phenomenology, or another qualitative design. The choice depends on the research question and the nature of the phenomenon under investigation.
• Sampling Strategy: Qualitative research employs purposeful or theoretical sampling to select participants who can provide rich and relevant information related to the research question. Sampling decisions are made to ensure diversity and depth in the data.
• Interviews: In-depth interviews are a common method in qualitative research. These interviews are typically semi-structured, allowing for flexibility while ensuring key topics are covered.
• Observation: Researchers may engage in direct observation of participants in natural settings. This can involve participant observation, where the researcher becomes part of the environment, or non-participant observation, where the researcher remains separate.
• Document Analysis: Researchers analyze existing documents, artifacts, or texts relevant to the study, such as diaries, letters, organizational records, or media content.
• Thematic Analysis: Researchers identify and analyze recurring themes or patterns in the data. This involves coding and categorizing data to uncover underlying meanings and concepts.
• Constant Comparative Analysis: Common in grounded theory, this method involves comparing data as it is collected, allowing researchers to refine categories and theories iteratively.
• Narrative Analysis: Focuses on the stories people tell, examining the structure and content of narratives to understand the meaning-making process.
• Informed Consent: Researchers obtain informed consent from participants, explaining the purpose of the study, potential risks, and ensuring participants have the right to withdraw at any time.
• Confidentiality and Anonymity: Researchers take measures to protect the privacy of participants by ensuring that their identities and personal information are kept confidential or anonymized.
• Credibility: Establishing credibility involves demonstrating that the study accurately represents participants’ perspectives. Techniques such as member checking, peer debriefing, and prolonged engagement contribute to credibility.
• Transferability: Researchers aim to make the study findings applicable to similar contexts. Detailed descriptions and thick descriptions enhance the transferability of qualitative research.
• Dependability and Confirmability: Ensuring dependability involves maintaining consistency in data collection and analysis, while confirmability ensures that findings are rooted in the data rather than researcher bias.
• Reflexivity: Researchers acknowledge their role in shaping the study and consider how their background, experiences, and biases may influence the research process and interpretation of findings. Reflexivity enhances transparency and the researcher’s self-awareness.

By carefully considering and integrating these key components, qualitative researchers can design studies that yield rich, contextually grounded insights into the social phenomena they aim to explore.

Types of Qualitative Research Design

Qualitative research design encompasses various approaches, each suited to different research questions and objectives. Here are some common types of qualitative research designs:

• Focus: Ethnography involves immersing the researcher in the natural environment of the participants to observe and understand their behaviors, practices, and cultural context.
• Data Collection: Researchers often use participant observation, interviews, and document analysis to gather data.
• Example: An anthropologist immersed in a remote tribe might live with the community for an extended period, participating in their daily activities, conducting interviews, and documenting observations. By doing so, the researcher gains a deep understanding of the tribe’s cultural practices, social relationships, and the significance of rituals in their way of life.
• Focus: Phenomenology explores the lived experiences of individuals to uncover the essence of a phenomenon.
• Data Collection: In-depth interviews and sometimes participant observation are common methods.
• Purpose: It seeks to understand the subjective meaning individuals attribute to an experience.
• In a study on the lived experiences of cancer survivors, researchers might conduct in-depth interviews to explore the subjective meaning individuals attach to their diagnosis, treatment, and recovery. Phenomenology seeks to uncover the essence of these experiences, capturing the emotional, psychological, and social dimensions that shape survivors’ perspectives on their journey through cancer.
• Focus: Grounded theory aims to develop a theory grounded in the data, allowing patterns and concepts to emerge organically.
• Data Collection: It involves constant comparative analysis of interviews or observations, with coding and categorization.
• Purpose: This approach is used when researchers want to generate theories or concepts based on the data itself.
• Research on retirement transitions using grounded theory might involve interviewing retirees from various backgrounds. Through constant comparison and iterative analysis, researchers may identify emerging themes and categories, ultimately developing a theory that explains the commonalities and variations in retirees’ experiences as they navigate this life stage.
• Focus: Case studies delve deeply into a specific case or context to understand it in detail.
• Data Collection: Multiple sources of data, such as interviews, observations, and documents, are used to provide a comprehensive view.
• Purpose: Case studies are useful for exploring complex phenomena within their real-life context.
• A case study on a company’s crisis response could involve a detailed examination of communication strategies, decision-making processes, and the organizational dynamics during a specific crisis. By analyzing the case in-depth, researchers gain insights into how the company’s actions and decisions influenced the outcome of the crisis and what lessons can be learned for future situations.
• Focus: Narrative research examines the stories people tell to understand how individuals construct meaning and identity.
• Data Collection: It involves collecting and analyzing narratives through interviews, personal accounts, or written documents.
• Purpose: Narrative research is often used to explore personal or cultural stories and their implications.
• Examining the life stories of refugees may involve collecting and analyzing personal narratives through interviews or written accounts. Researchers explore how displacement has shaped the refugees’ identities, relationships, and perceptions of home, providing a nuanced understanding of their experiences through the lens of storytelling.
• Focus: Action research involves collaboration between researchers and participants to identify and solve practical problems.
• Data Collection: Researchers collect data through cycles of planning, acting, observing, and reflecting.
• Purpose: It is geared towards facilitating positive change in a particular context or community.
• In an educational setting, action research might involve teachers and researchers collaborating to address a specific classroom challenge. Through cycles of planning, implementing interventions, and reflecting, the aim is to improve teaching practices and student learning outcomes, with the findings contributing to both practical solutions and the broader understanding of effective pedagogy.
• Focus: Content analysis examines the content of written, visual, or audio materials to identify patterns or themes.
• Data Collection: Researchers systematically analyze texts, images, or media content using coding and categorization.
• Purpose: It is often used to study communication, media, or cultural artifacts.
• A content analysis of news articles covering a specific social issue, such as climate change, could involve systematically coding and categorizing language and themes. This approach allows researchers to identify patterns in media discourse, explore public perceptions, and understand how the issue is framed in the media.
• Focus: Critical ethnography combines ethnographic methods with a critical perspective to examine power structures and social inequalities.
• Data Collection: Researchers engage in participant observation, interviews, and document analysis with a focus on social justice issues.
• Purpose: This approach aims to explore and challenge existing power dynamics and social structures.
• A critical ethnography examining gender dynamics in a workplace might involve observing daily interactions, conducting interviews, and analyzing policies. Researchers, guided by a critical perspective, aim to uncover power imbalances, stereotypes, and systemic inequalities within the organizational culture, contributing to a deeper understanding of gender dynamics in the workplace.
• Focus: Similar to grounded theory, constructivist grounded theory acknowledges the role of the researcher in shaping interpretations.
• Data Collection: It involves a flexible approach to data collection, including interviews, observations, or documents.
• Purpose: This approach recognizes the co-construction of meaning between researchers and participants.
• In a study on the experiences of individuals with chronic illness, researchers employing constructivist grounded theory might engage in open-ended interviews and data collection. The focus is on co-constructing meanings with participants, acknowledging the dynamic relationship between the researcher and those being studied, ultimately leading to a theory that reflects the collaborative nature of knowledge creation.

These qualitative research designs offer diverse methods for exploring and understanding the complexities of human experiences, behaviors, and social phenomena. The choice of design depends on the research question, the context of the study, and the desired depth of understanding.

Best practices for Qualitative Research Design

Qualitative research design requires careful planning and execution to ensure the credibility, reliability, and richness of the findings. Here are some best practices to consider when designing qualitative research:

• Clearly articulate the research questions or objectives to guide the study. Ensure they are specific, open-ended, and aligned with the qualitative research approach.
• Select a qualitative research design that aligns with the research questions and objectives. Consider approaches such as ethnography, phenomenology, grounded theory, or case study based on the nature of the study.
• Conduct a comprehensive literature review to understand existing theories, concepts, and research related to the study. This helps situate the research within the broader scholarly context.
• Use purposeful or theoretical sampling to select participants who can provide rich information related to the research questions. Aim for diversity in participants to capture a range of perspectives.
• Clearly outline the data collection methods, such as interviews, observations, or document analysis. Develop detailed protocols, guides, or questionnaires to maintain consistency across data collection sessions.
• Prioritize building trust and rapport with participants. Clearly communicate the study’s purpose, obtain informed consent, and establish a comfortable environment for open and honest discussions.
• Adhere to ethical guidelines throughout the research process. Protect participant confidentiality, respect their autonomy, and obtain ethical approval from relevant review boards.
• Pilot the data collection instruments and procedures with a small sample to identify and address any ambiguities, refine questions, and enhance the overall quality of data collection.
• Use a systematic approach to analyze data, such as thematic analysis, constant comparison, or narrative analysis. Maintain transparency in the coding process, and consider inter-coder reliability if multiple researchers are involved.
• Acknowledge and document the researcher’s background, biases, and perspectives. Practice reflexivity by continually reflecting on how the researcher’s positionality may influence the study.
• Enhance the credibility of findings by using multiple data sources and methods. Triangulation helps validate results and provides a more comprehensive understanding of the research topic.
• Consider member checking, where researchers share preliminary findings with participants to validate interpretations. This process enhances the credibility and trustworthiness of the study.
• Keep a detailed journal documenting decisions, reflections, and insights throughout the research process. This journal helps provide transparency and can contribute to the rigor of the study.
• Aim for data saturation, the point at which new data no longer provide additional insights. Saturation ensures thorough exploration of the research questions and increases the robustness of the findings.
• Clearly document the research process, from design to findings. Provide a detailed and transparent account of the study methodology, facilitating the reproducibility and evaluation of the research.

By incorporating these best practices, qualitative researchers can enhance the rigor, credibility, and relevance of their studies, ultimately contributing valuable insights to the field.

Interested in learning more about the fields of product, research, and design? Search our articles here for helpful information spanning a wide range of topics!

15 User Acceptance Testing Tools for Quality & Satisfaction

Mobile usability test: from preparation to execution & more, ux research process: a step-by-step guide for you, usability testing questions for improving user’s experience.

Understanding Qualitative and Quantitative Data

• 7 minute read
• August 22, 2024

Written by:

Smith Alex is a committed data enthusiast and an aspiring leader in the domain of data analytics. With a foundation in engineering and practical experience in the field of data science

Summary: This article delves into qualitative and quantitative data, defining each type and highlighting their key differences. It discusses when to use each data type, the benefits of integrating both, and the challenges researchers face. Understanding these concepts is crucial for effective research design and achieving comprehensive insights.

Introduction

In the realm of research and Data Analysis , two fundamental types of data play pivotal roles: qualitative and quantitative data. Understanding the distinctions between these two categories is essential for researchers, analysts, and decision-makers alike, as each type serves different purposes and is suited to various contexts.

This article will explore the definitions, characteristics, uses, and challenges associated with both qualitative and quantitative data, providing a comprehensive overview for anyone looking to enhance their understanding of data collection and analysis.

Read More:   Exploring 5 Statistical Data Analysis Techniques with Real-World Examples

Defining Qualitative Data

Qualitative data is non-numerical in nature and is primarily concerned with understanding the qualities, characteristics, and attributes of a subject.

This type of data is descriptive and often involves collecting information through methods such as interviews, focus groups, observations, and open-ended survey questions. The goal of qualitative data is to gain insights into the underlying motivations, opinions, and experiences of individuals or groups.

Characteristics of Qualitative Data

• Descriptive : Qualitative data provides rich, detailed descriptions of phenomena, allowing researchers to capture the complexity of human experiences.
• Subjective : The interpretation of qualitative data can vary based on the researcher’s perspective, making it inherently subjective.
• Contextual : This type of data is often context-dependent, meaning that the insights gained can be influenced by the environment or situation in which the data was collected.
• Exploratory : Qualitative data is typically used in exploratory research to generate hypotheses or to understand phenomena that are not well understood.

Examples of Qualitative Data

• Interview transcripts that capture participants’ thoughts and feelings.
• Observational notes from field studies.
• Responses to open-ended questions in surveys.
• Personal narratives or case studies that illustrate individual experiences.

Defining Quantitative Data

Quantitative data, in contrast, is numerical and can be measured or counted. This type of data is often used to quantify variables and analyse relationships between them. Quantitative research typically employs statistical methods to test hypotheses, identify patterns, and make predictions based on numerical data.

Characteristics of Quantitative Data

• Objective : Quantitative data is generally considered more objective than qualitative data, as it relies on measurable values that can be statistically analysed.
• Structured : This type of data is often collected using structured methods such as surveys with closed-ended questions, experiments, or observational checklists.
• Generalizable : Because quantitative data is based on numerical values, findings can often be generalised to larger populations if the sample is representative.
• Statistical Analysis : Quantitative data lends itself to various statistical analyses , allowing researchers to draw conclusions based on numerical evidence.

Examples of Quantitative Data

• Age, height, and weight measurements.
• Survey results with numerical ratings (e.g., satisfaction scores).
• Test scores or academic performance metrics.
• Financial data such as income, expenses, and profit margins.

Key Differences Between Qualitative and Quantitative Data

Understanding the differences between qualitative and quantitative data is crucial for selecting the appropriate research methods and analysis techniques. Here are some key distinctions:

When to Use Qualitative Data

Qualitative data is particularly useful in situations where the research aims to explore complex phenomena, understand human behaviour, or generate new theories. Here are some scenarios where qualitative data is the preferred choice:

Exploratory Research

When investigating a new area of study where little is known, qualitative methods can help uncover insights and generate hypotheses.

Understanding Context

Qualitative data is valuable for capturing the context surrounding a particular phenomenon, providing depth to the analysis.

Gaining Insights into Attitudes and Behaviours

When the goal is to understand why individuals think or behave in a certain way, qualitative methods such as interviews can provide rich, nuanced insights.

Developing Theories

Qualitative research can help in the development of theories by exploring relationships and patterns that quantitative methods may overlook.

When to Use Quantitative Data

Quantitative data is best suited for research that requires measurement, comparison, and statistical analysis. Here are some situations where quantitative data is the preferred choice:

Testing Hypotheses

When researchers have specific hypotheses to test , quantitative methods allow for rigorous statistical analysis to confirm or reject these hypotheses.

Measuring Variables

Quantitative data is ideal for measuring variables and establishing relationships between them, making it useful for experiments and surveys.

Generalising Findings

When the goal is to generalise findings to a larger population, quantitative research provides the necessary data to support such conclusions.

Identifying Patterns and Trends

Quantitative analysis can reveal patterns and trends in data that can inform decision-making and policy development.

Integrating Qualitative and Quantitative Data

While qualitative and quantitative data are distinct, they can be effectively integrated to provide a more comprehensive understanding of a research question. This mixed-methods approach combines the strengths of both types of data, allowing researchers to triangulate findings and gain deeper insights.

Benefits of Integration

Integrating qualitative and quantitative data enhances research by combining numerical analysis with rich, descriptive insights. This mixed-methods approach allows for a comprehensive understanding of complex phenomena, validating findings and providing a more nuanced perspective on research questions.

• Enhanced Validity: By using both qualitative and quantitative data, researchers can validate their findings through multiple sources of evidence.
• Rich Insights : Qualitative data can provide context and depth to quantitative findings, helping to explain the “why” behind numerical trends.
• Comprehensive Understanding: Integrating both types of data allows for a more holistic understanding of complex phenomena, leading to more informed conclusions and recommendations.

Examples of Integration

• Surveys with Open-Ended Questions: Combining closed-ended questions (quantitative) with open-ended questions (qualitative) in surveys can provide both measurable data and rich descriptive insights.
• Case Studies with Statistical Analysis: Researchers can conduct case studies (qualitative) while also collecting quantitative data to support their findings, offering a more robust analysis.
• Focus Groups with Follow-Up Surveys: After conducting focus groups (qualitative), researchers can administer surveys (quantitative) to a larger population to validate the insights gained.

Challenges and Considerations

While qualitative and quantitative data offer distinct advantages, researchers must also be aware of the challenges and considerations associated with each type:

Challenges of Qualitative Data

The challenges of qualitative data are multifaceted and can significantly impact the research process. Here are some of the primary challenges faced by researchers when working with qualitative data:

Subjectivity and Bias

One of the most significant challenges in qualitative research is the inherent subjectivity involved in data collection and analysis. Researchers’ personal beliefs, assumptions, and experiences can influence their interpretation of data.

Data Overload

Qualitative research often generates large volumes of data, which can be overwhelming. This data overload can make it challenging to identify key themes and insights. Researchers may struggle to manage and analyse vast amounts of qualitative data, leading to potential insights being overlooked.

Lack of Structure

Qualitative data is often unstructured, making it difficult to analyse systematically. The absence of a predefined format can lead to challenges in drawing meaningful conclusions from the data.

Time-Consuming Nature

Qualitative analysis can be extremely time-consuming, especially when dealing with extensive data sets. The process of collecting, transcribing, and analysing qualitative data often requires significant time and resources, which can be a barrier for researchers.

Challenges of Quantitative Data

Quantitative data provides objective, measurable evidence, it also faces challenges in capturing the full complexity of human experiences, maintaining data accuracy, and avoiding misinterpretation of statistical results. Integrating qualitative data can help overcome some of these limitations.

Limits in Capturing Complexity

Quantitative data, by its nature, can oversimplify complex phenomena and miss important nuances that qualitative data can capture. The focus on numerical measurements may not fully reflect the depth and richness of human experiences and behaviours.

Chances for Misinterpretation

Numbers can be twisted or misinterpreted if not analysed properly. Researchers must be cautious in interpreting statistical results, as correlation does not imply causation. Poor knowledge of statistical analysis can negatively impact the analysis and interpretation of quantitative data.

Influence of Measurement Errors

Due to the numerical nature of quantitative data, even small measurement errors can skew the entire dataset. Inaccuracies in data collection methods can lead to drawing incorrect conclusions from the analysis.

Lack of Context

Quantitative experiments often do not take place in natural settings. The data may lack the context and nuance that qualitative data can provide to fully explain the phenomena being studied.

Sample Size Limitations

Small sample sizes in quantitative studies can reduce the reliability of the data. Large sample sizes are needed for more accurate statistical analysis. This also affects the ability to generalise findings to wider populations.

Confirmation Bias

Researchers may miss observing important phenomena due to their focus on testing pre-determined hypotheses rather than generating new theories. The confirmation bias inherent in hypothesis testing can limit the discovery of unexpected insights.

In conclusion, understanding the distinctions between qualitative and quantitative data is essential for effective research and Data Analysis . Each type of data serves unique purposes and is suited to different contexts, making it crucial for researchers to select the appropriate methods based on their research objectives.

By integrating both qualitative and quantitative data, researchers can gain a more comprehensive understanding of complex phenomena, leading to richer insights and more informed decision-making.

As the landscape of research continues to evolve, the ability to effectively utilise and integrate both types of data will remain a valuable skill for researchers and analysts alike.

Frequently Asked Questions

What is the primary difference between qualitative and quantitative data.

The primary difference is that qualitative data is descriptive and non-numerical, focusing on understanding qualities and experiences, while quantitative data is numerical and measurable, focusing on quantifying variables and testing hypotheses.

When Should I Use Qualitative Data in My Research?

Qualitative data is best used when exploring new topics, understanding complex behaviours, or generating hypotheses, particularly when context and depth are important.

Can Qualitative and Quantitative Data Be Used Together?

Yes, integrating qualitative and quantitative data can provide a more comprehensive understanding of a research question, allowing researchers to validate findings and gain richer insights.

Reviewed by:

Post written by: Smith Alex

Artificial Intelligence Web Scraping vs. Web Crawling: Understanding the Differences

Python Metaprogramming: Unlocking the Power of Code Manipulation

You may also like.

• Data Analysts

Top Data Analytics Projects in 2024 for Beginners to Experienced

• July 20, 2023

Explaining Four Types of Analytics With Examples

• Shlok Kamat
• July 24, 2023
• 15 minute read

Qualitative vs Quantitative Research Methods & Data Analysis

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

The main difference between quantitative and qualitative research is the type of data they collect and analyze.

Quantitative data is information about quantities, and therefore numbers, and qualitative data is descriptive, and regards phenomenon which can be observed but not measured, such as language.

• Quantitative research collects numerical data and analyzes it using statistical methods. The aim is to produce objective, empirical data that can be measured and expressed numerically. Quantitative research is often used to test hypotheses, identify patterns, and make predictions.
• Qualitative research gathers non-numerical data (words, images, sounds) to explore subjective experiences and attitudes, often via observation and interviews. It aims to produce detailed descriptions and uncover new insights about the studied phenomenon.

On This Page:

What Is Qualitative Research?

Qualitative research is the process of collecting, analyzing, and interpreting non-numerical data, such as language. Qualitative research can be used to understand how an individual subjectively perceives and gives meaning to their social reality.

Qualitative data is non-numerical data, such as text, video, photographs, or audio recordings. This type of data can be collected using diary accounts or in-depth interviews and analyzed using grounded theory or thematic analysis.

Qualitative research is multimethod in focus, involving an interpretive, naturalistic approach to its subject matter. This means that qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them. Denzin and Lincoln (1994, p. 2)

Interest in qualitative data came about as the result of the dissatisfaction of some psychologists (e.g., Carl Rogers) with the scientific study of psychologists such as behaviorists (e.g., Skinner ).

Since psychologists study people, the traditional approach to science is not seen as an appropriate way of carrying out research since it fails to capture the totality of human experience and the essence of being human.  Exploring participants’ experiences is known as a phenomenological approach (re: Humanism ).

Qualitative research is primarily concerned with meaning, subjectivity, and lived experience. The goal is to understand the quality and texture of people’s experiences, how they make sense of them, and the implications for their lives.

Qualitative research aims to understand the social reality of individuals, groups, and cultures as nearly as possible as participants feel or live it. Thus, people and groups are studied in their natural setting.

Some examples of qualitative research questions are provided, such as what an experience feels like, how people talk about something, how they make sense of an experience, and how events unfold for people.

Research following a qualitative approach is exploratory and seeks to explain ‘how’ and ‘why’ a particular phenomenon, or behavior, operates as it does in a particular context. It can be used to generate hypotheses and theories from the data.

Qualitative Methods

There are different types of qualitative research methods, including diary accounts, in-depth interviews , documents, focus groups , case study research , and ethnography .

The results of qualitative methods provide a deep understanding of how people perceive their social realities and in consequence, how they act within the social world.

The researcher has several methods for collecting empirical materials, ranging from the interview to direct observation, to the analysis of artifacts, documents, and cultural records, to the use of visual materials or personal experience. Denzin and Lincoln (1994, p. 14)

Here are some examples of qualitative data:

Interview transcripts : Verbatim records of what participants said during an interview or focus group. They allow researchers to identify common themes and patterns, and draw conclusions based on the data. Interview transcripts can also be useful in providing direct quotes and examples to support research findings.

Observations : The researcher typically takes detailed notes on what they observe, including any contextual information, nonverbal cues, or other relevant details. The resulting observational data can be analyzed to gain insights into social phenomena, such as human behavior, social interactions, and cultural practices.

Unstructured interviews : generate qualitative data through the use of open questions.  This allows the respondent to talk in some depth, choosing their own words.  This helps the researcher develop a real sense of a person’s understanding of a situation.

Diaries or journals : Written accounts of personal experiences or reflections.

Notice that qualitative data could be much more than just words or text. Photographs, videos, sound recordings, and so on, can be considered qualitative data. Visual data can be used to understand behaviors, environments, and social interactions.

Qualitative Data Analysis

Qualitative research is endlessly creative and interpretive. The researcher does not just leave the field with mountains of empirical data and then easily write up his or her findings.

Qualitative interpretations are constructed, and various techniques can be used to make sense of the data, such as content analysis, grounded theory (Glaser & Strauss, 1967), thematic analysis (Braun & Clarke, 2006), or discourse analysis .

For example, thematic analysis is a qualitative approach that involves identifying implicit or explicit ideas within the data. Themes will often emerge once the data has been coded .

Key Features

• Events can be understood adequately only if they are seen in context. Therefore, a qualitative researcher immerses her/himself in the field, in natural surroundings. The contexts of inquiry are not contrived; they are natural. Nothing is predefined or taken for granted.
• Qualitative researchers want those who are studied to speak for themselves, to provide their perspectives in words and other actions. Therefore, qualitative research is an interactive process in which the persons studied teach the researcher about their lives.
• The qualitative researcher is an integral part of the data; without the active participation of the researcher, no data exists.
• The study’s design evolves during the research and can be adjusted or changed as it progresses. For the qualitative researcher, there is no single reality. It is subjective and exists only in reference to the observer.
• The theory is data-driven and emerges as part of the research process, evolving from the data as they are collected.

Limitations of Qualitative Research

• Because of the time and costs involved, qualitative designs do not generally draw samples from large-scale data sets.
• The problem of adequate validity or reliability is a major criticism. Because of the subjective nature of qualitative data and its origin in single contexts, it is difficult to apply conventional standards of reliability and validity. For example, because of the central role played by the researcher in the generation of data, it is not possible to replicate qualitative studies.
• Also, contexts, situations, events, conditions, and interactions cannot be replicated to any extent, nor can generalizations be made to a wider context than the one studied with confidence.
• The time required for data collection, analysis, and interpretation is lengthy. Analysis of qualitative data is difficult, and expert knowledge of an area is necessary to interpret qualitative data. Great care must be taken when doing so, for example, looking for mental illness symptoms.

Advantages of Qualitative Research

• Because of close researcher involvement, the researcher gains an insider’s view of the field. This allows the researcher to find issues that are often missed (such as subtleties and complexities) by the scientific, more positivistic inquiries.
• Qualitative descriptions can be important in suggesting possible relationships, causes, effects, and dynamic processes.
• Qualitative analysis allows for ambiguities/contradictions in the data, which reflect social reality (Denscombe, 2010).
• Qualitative research uses a descriptive, narrative style; this research might be of particular benefit to the practitioner as she or he could turn to qualitative reports to examine forms of knowledge that might otherwise be unavailable, thereby gaining new insight.

What Is Quantitative Research?

Quantitative research involves the process of objectively collecting and analyzing numerical data to describe, predict, or control variables of interest.

The goals of quantitative research are to test causal relationships between variables , make predictions, and generalize results to wider populations.

Quantitative researchers aim to establish general laws of behavior and phenomenon across different settings/contexts. Research is used to test a theory and ultimately support or reject it.

Quantitative Methods

Experiments typically yield quantitative data, as they are concerned with measuring things.  However, other research methods, such as controlled observations and questionnaires , can produce both quantitative information.

For example, a rating scale or closed questions on a questionnaire would generate quantitative data as these produce either numerical data or data that can be put into categories (e.g., “yes,” “no” answers).

Experimental methods limit how research participants react to and express appropriate social behavior.

Findings are, therefore, likely to be context-bound and simply a reflection of the assumptions that the researcher brings to the investigation.

There are numerous examples of quantitative data in psychological research, including mental health. Here are a few examples:

Another example is the Experience in Close Relationships Scale (ECR), a self-report questionnaire widely used to assess adult attachment styles .

The ECR provides quantitative data that can be used to assess attachment styles and predict relationship outcomes.

Neuroimaging data : Neuroimaging techniques, such as MRI and fMRI, provide quantitative data on brain structure and function.

This data can be analyzed to identify brain regions involved in specific mental processes or disorders.

For example, the Beck Depression Inventory (BDI) is a clinician-administered questionnaire widely used to assess the severity of depressive symptoms in individuals.

The BDI consists of 21 questions, each scored on a scale of 0 to 3, with higher scores indicating more severe depressive symptoms. 

Quantitative Data Analysis

Statistics help us turn quantitative data into useful information to help with decision-making. We can use statistics to summarize our data, describing patterns, relationships, and connections. Statistics can be descriptive or inferential.

Descriptive statistics help us to summarize our data. In contrast, inferential statistics are used to identify statistically significant differences between groups of data (such as intervention and control groups in a randomized control study).

• Quantitative researchers try to control extraneous variables by conducting their studies in the lab.
• The research aims for objectivity (i.e., without bias) and is separated from the data.
• The design of the study is determined before it begins.
• For the quantitative researcher, the reality is objective, exists separately from the researcher, and can be seen by anyone.
• Research is used to test a theory and ultimately support or reject it.

Limitations of Quantitative Research

• Context: Quantitative experiments do not take place in natural settings. In addition, they do not allow participants to explain their choices or the meaning of the questions they may have for those participants (Carr, 1994).
• Researcher expertise: Poor knowledge of the application of statistical analysis may negatively affect analysis and subsequent interpretation (Black, 1999).
• Variability of data quantity: Large sample sizes are needed for more accurate analysis. Small-scale quantitative studies may be less reliable because of the low quantity of data (Denscombe, 2010). This also affects the ability to generalize study findings to wider populations.
• Confirmation bias: The researcher might miss observing phenomena because of focus on theory or hypothesis testing rather than on the theory of hypothesis generation.

Advantages of Quantitative Research

• Scientific objectivity: Quantitative data can be interpreted with statistical analysis, and since statistics are based on the principles of mathematics, the quantitative approach is viewed as scientifically objective and rational (Carr, 1994; Denscombe, 2010).
• Useful for testing and validating already constructed theories.
• Rapid analysis: Sophisticated software removes much of the need for prolonged data analysis, especially with large volumes of data involved (Antonius, 2003).
• Replication: Quantitative data is based on measured values and can be checked by others because numerical data is less open to ambiguities of interpretation.
• Hypotheses can also be tested because of statistical analysis (Antonius, 2003).

Antonius, R. (2003). Interpreting quantitative data with SPSS . Sage.

Black, T. R. (1999). Doing quantitative research in the social sciences: An integrated approach to research design, measurement and statistics . Sage.

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology . Qualitative Research in Psychology , 3, 77–101.

Carr, L. T. (1994). The strengths and weaknesses of quantitative and qualitative research : what method for nursing? Journal of advanced nursing, 20(4) , 716-721.

Denscombe, M. (2010). The Good Research Guide: for small-scale social research. McGraw Hill.

Denzin, N., & Lincoln. Y. (1994). Handbook of Qualitative Research. Thousand Oaks, CA, US: Sage Publications Inc.

Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968). The discovery of grounded theory; strategies for qualitative research. Nursing research, 17(4) , 364.

Minichiello, V. (1990). In-Depth Interviewing: Researching People. Longman Cheshire.

Punch, K. (1998). Introduction to Social Research: Quantitative and Qualitative Approaches. London: Sage

Further Information

• Mixed methods research
• Designing qualitative research
• Methods of data collection and analysis
• Introduction to quantitative and qualitative research
• Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
• Qualitative research in health care: Analysing qualitative data
• Qualitative data analysis: the framework approach
• Using the framework method for the analysis of
• Qualitative data in multi-disciplinary health research
• Content Analysis
• Grounded Theory
• Thematic Analysis

23 Advantages and Disadvantages of Qualitative Research

Investigating methodologies. Taking a closer look at ethnographic, anthropological, or naturalistic techniques. Data mining through observer recordings. This is what the world of qualitative research is all about. It is the comprehensive and complete data that is collected by having the courage to ask an open-ended question.

Print media has used the principles of qualitative research for generations. Now more industries are seeing the advantages that come from the extra data that is received by asking more than a “yes” or “no” question.

The advantages and disadvantages of qualitative research are quite unique. On one hand, you have the perspective of the data that is being collected. On the other hand, you have the techniques of the data collector and their own unique observations that can alter the information in subtle ways.

That’s why these key points are so important to consider.

What Are the Advantages of Qualitative Research?

1. Subject materials can be evaluated with greater detail. There are many time restrictions that are placed on research methods. The goal of a time restriction is to create a measurable outcome so that metrics can be in place. Qualitative research focuses less on the metrics of the data that is being collected and more on the subtleties of what can be found in that information. This allows for the data to have an enhanced level of detail to it, which can provide more opportunities to glean insights from it during examination.

2. Research frameworks can be fluid and based on incoming or available data. Many research opportunities must follow a specific pattern of questioning, data collection, and information reporting. Qualitative research offers a different approach. It can adapt to the quality of information that is being gathered. If the available data does not seem to be providing any results, the research can immediately shift gears and seek to gather data in a new direction. This offers more opportunities to gather important clues about any subject instead of being confined to a limited and often self-fulfilling perspective.

3. Qualitative research data is based on human experiences and observations. Humans have two very different operating systems. One is a subconscious method of operation, which is the fast and instinctual observations that are made when data is present. The other operating system is slower and more methodical, wanting to evaluate all sources of data before deciding. Many forms of research rely on the second operating system while ignoring the instinctual nature of the human mind. Qualitative research doesn’t ignore the gut instinct. It embraces it and the data that can be collected is often better for it.

4. Gathered data has a predictive quality to it. One of the common mistakes that occurs with qualitative research is an assumption that a personal perspective can be extrapolated into a group perspective. This is only possible when individuals grow up in similar circumstances, have similar perspectives about the world, and operate with similar goals. When these groups can be identified, however, the gathered individualistic data can have a predictive quality for those who are in a like-minded group. At the very least, the data has a predictive quality for the individual from whom it was gathered.

5. Qualitative research operates within structures that are fluid. Because the data being gathered through this type of research is based on observations and experiences, an experienced researcher can follow-up interesting answers with additional questions. Unlike other forms of research that require a specific framework with zero deviation, researchers can follow any data tangent which makes itself known and enhance the overall database of information that is being collected.

6. Data complexities can be incorporated into generated conclusions. Although our modern world tends to prefer statistics and verifiable facts, we cannot simply remove the human experience from the equation. Different people will have remarkably different perceptions about any statistic, fact, or event. This is because our unique experiences generate a different perspective of the data that we see. These complexities, when gathered into a singular database, can generate conclusions with more depth and accuracy, which benefits everyone.

7. Qualitative research is an open-ended process. When a researcher is properly prepared, the open-ended structures of qualitative research make it possible to get underneath superficial responses and rational thoughts to gather information from an individual’s emotional response. This is critically important to this form of researcher because it is an emotional response which often drives a person’s decisions or influences their behavior.

8. Creativity becomes a desirable quality within qualitative research. It can be difficult to analyze data that is obtained from individual sources because many people subconsciously answer in a way that they think someone wants. This desire to “please” another reduces the accuracy of the data and suppresses individual creativity. By embracing the qualitative research method, it becomes possible to encourage respondent creativity, allowing people to express themselves with authenticity. In return, the data collected becomes more accurate and can lead to predictable outcomes.

9. Qualitative research can create industry-specific insights. Brands and businesses today need to build relationships with their core demographics to survive. The terminology, vocabulary, and jargon that consumers use when looking at products or services is just as important as the reputation of the brand that is offering them. If consumers are receiving one context, but the intention of the brand is a different context, then the miscommunication can artificially restrict sales opportunities. Qualitative research gives brands access to these insights so they can accurately communicate their value propositions.

10. Smaller sample sizes are used in qualitative research, which can save on costs. Many qualitative research projects can be completed quickly and on a limited budget because they typically use smaller sample sizes that other research methods. This allows for faster results to be obtained so that projects can move forward with confidence that only good data is able to provide.

11. Qualitative research provides more content for creatives and marketing teams. When your job involves marketing, or creating new campaigns that target a specific demographic, then knowing what makes those people can be quite challenging. By going through the qualitative research approach, it becomes possible to congregate authentic ideas that can be used for marketing and other creative purposes. This makes communication between the two parties to be handled with more accuracy, leading to greater level of happiness for all parties involved.

12. Attitude explanations become possible with qualitative research. Consumer patterns can change on a dime sometimes, leaving a brand out in the cold as to what just happened. Qualitative research allows for a greater understanding of consumer attitudes, providing an explanation for events that occur outside of the predictive matrix that was developed through previous research. This allows the optimal brand/consumer relationship to be maintained.

What Are the Disadvantages of Qualitative Research?

1. The quality of the data gathered in qualitative research is highly subjective. This is where the personal nature of data gathering in qualitative research can also be a negative component of the process. What one researcher might feel is important and necessary to gather can be data that another researcher feels is pointless and won’t spend time pursuing it. Having individual perspectives and including instinctual decisions can lead to incredibly detailed data. It can also lead to data that is generalized or even inaccurate because of its reliance on researcher subjectivisms.

2. Data rigidity is more difficult to assess and demonstrate. Because individual perspectives are often the foundation of the data that is gathered in qualitative research, it is more difficult to prove that there is rigidity in the information that is collective. The human mind tends to remember things in the way it wants to remember them. That is why memories are often looked at fondly, even if the actual events that occurred may have been somewhat disturbing at the time. This innate desire to look at the good in things makes it difficult for researchers to demonstrate data validity.

3. Mining data gathered by qualitative research can be time consuming. The number of details that are often collected while performing qualitative research are often overwhelming. Sorting through that data to pull out the key points can be a time-consuming effort. It is also a subjective effort because what one researcher feels is important may not be pulled out by another researcher. Unless there are some standards in place that cannot be overridden, data mining through a massive number of details can almost be more trouble than it is worth in some instances.

4. Qualitative research creates findings that are valuable, but difficult to present. Presenting the findings which come out of qualitative research is a bit like listening to an interview on CNN. The interviewer will ask a question to the interviewee, but the goal is to receive an answer that will help present a database which presents a specific outcome to the viewer. The goal might be to have a viewer watch an interview and think, “That’s terrible. We need to pass a law to change that.” The subjective nature of the information, however, can cause the viewer to think, “That’s wonderful. Let’s keep things the way they are right now.” That is why findings from qualitative research are difficult to present. What a research gleans from the data can be very different from what an outside observer gleans from the data.

5. Data created through qualitative research is not always accepted. Because of the subjective nature of the data that is collected in qualitative research, findings are not always accepted by the scientific community. A second independent qualitative research effort which can produce similar findings is often necessary to begin the process of community acceptance.

6. Researcher influence can have a negative effect on the collected data. The quality of the data that is collected through qualitative research is highly dependent on the skills and observation of the researcher. If a researcher has a biased point of view, then their perspective will be included with the data collected and influence the outcome. There must be controls in place to help remove the potential for bias so the data collected can be reviewed with integrity. Otherwise, it would be possible for a researcher to make any claim and then use their bias through qualitative research to prove their point.

7. Replicating results can be very difficult with qualitative research. The scientific community wants to see results that can be verified and duplicated to accept research as factual. In the world of qualitative research, this can be very difficult to accomplish. Not only do you have the variability of researcher bias for which to account within the data, but there is also the informational bias that is built into the data itself from the provider. This means the scope of data gathering can be extremely limited, even if the structure of gathering information is fluid, because of each unique perspective.

8. Difficult decisions may require repetitive qualitative research periods. The smaller sample sizes of qualitative research may be an advantage, but they can also be a disadvantage for brands and businesses which are facing a difficult or potentially controversial decision. A small sample is not always representative of a larger population demographic, even if there are deep similarities with the individuals involve. This means a follow-up with a larger quantitative sample may be necessary so that data points can be tracked with more accuracy, allowing for a better overall decision to be made.

9. Unseen data can disappear during the qualitative research process. The amount of trust that is placed on the researcher to gather, and then draw together, the unseen data that is offered by a provider is enormous. The research is dependent upon the skill of the researcher being able to connect all the dots. If the researcher can do this, then the data can be meaningful and help brands and progress forward with their mission. If not, there is no way to alter course until after the first results are received. Then a new qualitative process must begin.

10. Researchers must have industry-related expertise. You can have an excellent researcher on-board for a project, but if they are not familiar with the subject matter, they will have a difficult time gathering accurate data. For qualitative research to be accurate, the interviewer involved must have specific skills, experiences, and expertise in the subject matter being studied. They must also be familiar with the material being evaluated and have the knowledge to interpret responses that are received. If any piece of this skill set is missing, the quality of the data being gathered can be open to interpretation.

11. Qualitative research is not statistically representative. The one disadvantage of qualitative research which is always present is its lack of statistical representation. It is a perspective-based method of research only, which means the responses given are not measured. Comparisons can be made and this can lead toward the duplication which may be required, but for the most part, quantitative data is required for circumstances which need statistical representation and that is not part of the qualitative research process.

The advantages and disadvantages of qualitative research make it possible to gather and analyze individualistic data on deeper levels. This makes it possible to gain new insights into consumer thoughts, demographic behavioral patterns, and emotional reasoning processes. When a research can connect the dots of each information point that is gathered, the information can lead to personalized experiences, better value in products and services, and ongoing brand development.

• Open access
• Published: 26 August 2024

Evaluating panel discussions in ESP classes: an exploration of international medical students’ and ESP instructors’ perspectives through qualitative research

• Elham Nasiri   ORCID: orcid.org/0000-0002-0644-1646 1 &
• Laleh Khojasteh   ORCID: orcid.org/0000-0002-6393-2759 1  

BMC Medical Education volume  24 , Article number:  925 ( 2024 ) Cite this article

65 Accesses

Metrics details

This study investigates the effectiveness of panel discussions, a specific interactive teaching technique where a group of students leads a pre-planned, topic-focused discussion with audience participation, in English for Specific Purposes (ESP) courses for international medical students. This approach aims to simulate professional conference discussions, preparing students for future academic and clinical environments where such skills are crucial. While traditional group presentations foster critical thinking and communication, a gap exists in understanding how medical students perceive the complexities of preparing for and participating in panel discussions within an ESP setting. This qualitative study investigates the perceived advantages and disadvantages of these discussions from the perspectives of both panelists (medical students) and the audience (peers). Additionally, the study explores potential improvements based on insights from ESP instructors. Utilizing a two-phase design involving reflection papers and focus group discussions, data were collected from 46 medical students and three ESP instructors. Thematic analysis revealed that panel discussions offer unique benefits compared to traditional presentations, including enhanced engagement and more dynamic skill development for both panelists and the audience. Panelists reported gains in personal and professional development, including honing critical thinking, communication, and presentation skills. The audience perceived these discussions as engaging learning experiences that fostered critical analysis and information synthesis. However, challenges such as academic workload and concerns about discussion quality were also identified. The study concludes that panel discussions, when implemented effectively, can be a valuable tool for enhancing critical thinking, communication skills, and subject matter knowledge in ESP courses for medical students. These skills are transferable and can benefit students in various academic and professional settings, including future participation in medical conferences. This research provides valuable insights for ESP instructors seeking to integrate panel discussions into their curriculum, ultimately improving student learning outcomes and preparing them for future success in professional communication.

Peer Review reports

Introduction

In the field of medical education, the acquisition and application of effective communication skills are crucial for medical students in today’s global healthcare environment [ 1 ]. This necessitates not only strong English language proficiency but also the ability to present complex medical information clearly and concisely to diverse audiences.

Language courses, especially English for Specific Purposes (ESP) courses for medical students, are highly relevant in today’s globalized healthcare environment [ 2 ]. In non-English speaking countries like Iran, these courses are particularly important as they go beyond mere language instruction to include the development of critical thinking, cultural competence, and professional communication skills [ 3 ]. Proficiency in English is crucial for accessing up-to-date research, participating in international conferences, and communicating with patients and colleagues from diverse backgrounds [ 4 ]. Additionally, ESP courses help medical students understand and use medical terminologies accurately, which is essential for reading technical articles, listening to audio presentations, and giving spoken presentations [ 5 ]. In countries where English is not the primary language, ESP courses ensure that medical professionals can stay current with global advancements and collaborate effectively on an international scale [ 6 ]. Furthermore, these courses support students who may seek to practice medicine abroad, enhancing their career opportunities and professional growth [ 7 ].

Moreover, ESP courses enable medical professionals to communicate effectively with international patients, which is crucial in multicultural societies and for medical tourism, ensuring that patient care is not compromised due to language barriers [ 8 ]. Many medical textbooks, journals, and online resources are available primarily in English, and ESP courses equip medical students with the necessary language skills to access and comprehend these resources, ensuring they are well-informed about the latest medical research and practices [ 9 ].

Additionally, many medical professionals from non-English speaking countries aim to take international certification exams, such as the USMLE or PLAB, which are conducted in English, and ESP courses prepare students for these exams by familiarizing them with the medical terminology and language used in these assessments [ 10 ]. ESP courses also contribute to the professional development of medical students by improving their ability to write research papers, case reports, and other academic documents in English, which is essential for publishing in international journals and contributing to global medical knowledge [ 11 ]. In the increasingly interdisciplinary field of healthcare, collaboration with professionals from other countries is common, and ESP courses facilitate effective communication and collaboration with international colleagues, fostering innovation and the exchange of ideas [ 12 ].

With the rise of telemedicine and online medical consultations, proficiency in English is essential for non-English speaking medical professionals to provide remote healthcare services to international patients, and ESP courses prepare students for these modern medical practices [ 13 ].

Finally, ESP courses often include training on cultural competence, which is crucial for understanding and respecting the cultural backgrounds of patients and colleagues, leading to more empathetic and effective patient care and professional interactions [ 14 ]. Many ESP programs for medical students incorporate group presentations as a vital component of their curriculum, recognizing the positive impact on developing these essential skills [ 15 ].

Group projects in language courses, particularly in ESP for medical students, are highly relevant for several reasons. They provide a collaborative environment that mimics real-world professional settings, where healthcare professionals often work in multidisciplinary teams [ 16 ]. These group activities foster not only language skills but also crucial soft skills such as teamwork, leadership, and interpersonal communication, which are essential in medical practice [ 17 ].

The benefits of group projects over individual projects in language learning are significant. Hartono, Mujiyanto [ 18 ] found that group presentation tasks in ESP courses led to higher self-efficacy development compared to individual tasks. Group projects encourage peer learning, where students can learn from each other’s strengths and compensate for individual weaknesses [ 19 ]. They also provide a supportive environment that can reduce anxiety and increase willingness to communicate in the target language [ 20 ]. However, it is important to note that group projects also come with challenges, such as social loafing and unequal contribution, which need to be managed effectively [ 21 ].

Traditional lecture-based teaching methods, while valuable for knowledge acquisition, may not effectively prepare medical students for the interactive and collaborative nature of real-world healthcare settings [ 22 ]. Panel discussions (hereafter PDs), an interactive teaching technique where a group of students leads a pre-planned, topic-focused discussion with audience participation, are particularly relevant in this context. They simulate professional conference discussions and interdisciplinary team meetings, preparing students for future academic and clinical environments where such skills are crucial [ 23 ].

PDs, also known as moderated discussions or moderated panels, are a specific type of interactive format where a group of experts or stakeholders engage in a facilitated conversation on a particular topic or issue [ 22 ]. In this format, a moderator guides the discussion, encourages active participation from all panelists, and fosters a collaborative environment that promotes constructive dialogue and critical thinking [ 24 ]. The goal is to encourage audience engagement and participation, which can be achieved through various strategies such as asking open-ended questions, encouraging counterpoints and counterarguments, and providing opportunities for audience members to pose questions or share their own experiences [ 25 ]. These discussions can take place in-person or online, and can be designed to accommodate diverse audiences and settings [ 26 ].

In this study, PD is considered a speaking activity where medical students are assigned specific roles to play during the simulation, such as a physician, quality improvement specialist, policymaker, or patient advocate. By taking on these roles, students can gain a better understanding of the diverse perspectives and considerations that come into play in real-world healthcare discussions [ 23 ]. Simulating PDs within ESP courses can be a powerful tool for enhancing medical students’ learning outcomes in multiple areas. This approach improves language proficiency, academic skills, and critical thinking abilities, while also enabling students to communicate effectively with diverse stakeholders in the medical field [ 27 , 28 ].

Theoretical framework

The panel discussions in our study are grounded in the concept of authentic assessment (outlined by Villarroel, Bloxham [ 29 ]), which involves designing tasks that mirror real-life situations and problems. In the context of medical education, this approach is particularly relevant as it prepares students for the complex, multidisciplinary nature of healthcare communication. Realism can be achieved through two means: providing a realistic context that describes and delivers a frame for the problem to be solved and creating tasks that are similar to those faced in real and/or professional life [ 30 ]. In our study, the PDs provide a realistic context by simulating scenarios where medical students are required to discuss and present complex medical topics in a professional setting, mirroring the types of interactions they will encounter in their future careers.

The task of participating in PDs also involves cognitive challenge, as students are required to think critically about complex medical topics, analyze information, and communicate their findings effectively. This type of task aims to generate processes of problem-solving, application of knowledge, and decision-making that correspond to the development of cognitive and metacognitive skills [ 23 ]. For medical students, these skills are crucial in developing clinical reasoning and effective patient communication. The PDs encourage students to go beyond the textual reproduction of fragmented and low-order content and move towards understanding, establishing relationships between new ideas and previous knowledge, linking theoretical concepts with everyday experience, deriving conclusions from the analysis of data, and examining both the logic of the arguments present in the theory and its practical scope [ 24 , 25 , 27 ].

Furthermore, the evaluative judgment aspect of our study is critical in helping students develop criteria and standards about what a good performance means in medical communication. This involves students judging their own performance and regulating their own learning [ 31 ]. In the context of panel discussions, students reflect on their own work, compare it with desired standards, and seek feedback from peers and instructors. By doing so, students can develop a sense of what constitutes good performance in medical communication and what areas need improvement [ 32 ]. Boud, Lawson and Thompson [ 33 ] argue that students need to build a precise judgment about the quality of their work and calibrate these judgments in the light of evidence. This skill is particularly important for future medical professionals who will need to continually assess and improve their communication skills throughout their careers.

The theoretical framework presented above highlights the importance of authentic learning experiences in medical education. By drawing on the benefits of group work and panel discussions, university instructor-researchers aimed to provide medical students with a unique opportunity to engage with complex cases and develop their communication and collaboration skills. As noted by Suryanarayana [ 34 ], authentic learning experiences can lead to deeper learning and improved retention. Considering the advantages of group work in promoting collaborative problem-solving and language development, the instructor-researchers designed a panel discussion task that simulates real-world scenarios, where students can work together to analyze complex cases, share knowledge, and present their findings to a simulated audience.

While previous studies have highlighted the benefits of interactive learning experiences and critical thinking skills in medical education, a research gap remains in understanding how medical students perceive the relevance of PDs in ESP courses. This study aims to address this gap by investigating medical students’ perceptions of PD tasks in ESP courses and how these perceptions relate to their language proficiency, critical thinking skills, and ability to communicate effectively with diverse stakeholders in the medical field. This understanding can inform best practices in medical education, contributing to the development of more effective communication skills for future healthcare professionals worldwide [ 23 ]. The research questions guiding this study are:

What are the perceived advantages of PDs from the perspectives of panelists and the audience?

What are the perceived disadvantages of PDs from the perspectives of panelists and the audience?

How can PDs be improved for panelists and the audience based on the insights of ESP instructors?

Methodology

Aim and design.

For this study, a two-phase qualitative design was employed to gain an understanding of the advantages and disadvantages of PDs from the perspectives of both student panelists and the audience (Phase 1) and to acquire an in-depth understanding of the suggested strategies provided by experts to enhance PPs for future students (Phase 2).

Participants and context of the study

This study was conducted in two phases (Fig.  1 ) at Shiraz University of Medical Sciences (SUMS), Shiraz, Iran.

Participants of the study in two phases

In the first phase, the student participants were 46 non-native speakers of English and international students who studied medicine at SUMS. Their demographic characteristics can be seen in Table  1 .

These students were purposefully selected because they were the only SUMS international students who had taken the ESP (English for Specific Purposes) course. The number of international students attending SUMS is indeed limited. Each year, a different batch of international students joins the university. They progress through a sequence of English courses, starting with General English 1 and 2, followed by the ESP course, and concluding with academic writing. At the time of data collection, the students included in the study were the only international students enrolled in the ESP course. This mandatory 3-unit course is designed to enhance their language and communication skills specifically tailored to their profession. As a part of the Medicine major curriculum, this course aims to improve their English language proficiency in areas relevant to medicine, such as understanding medical terminology, comprehending original medicine texts, discussing clinical cases, and communicating with patients, colleagues, and other healthcare professionals.

Throughout the course, students engage in various interactive activities, such as group discussions, role-playing exercises, and case studies, to develop their practical communication skills. In this course, medical students receive four marks out of 20 for their oral presentations, while the remaining marks are allocated to their written midterm and final exams. From the beginning of the course, they are briefed about PDs, and they are shown two YouTube-downloaded videos about PDs at medical conferences, a popular format for discussing and sharing knowledge, research findings, and expert opinions on various medical topics.

For the second phase of the study, a specific group of participants was purposefully selected. This group consisted of three faculty members from SUMS English department who had extensive experience attending numerous conferences at national and international levels, particularly in the medical field, as well as working as translators and interpreters in medical congresses. Over the course of ten years, they also gained considerable experience in PDs. They were invited to discuss strategies helpful for medical students with PDs.

Panel discussion activity design and implementation

When preparing for a PD session, medical students received comprehensive guidance on understanding the roles and responsibilities of each panel member. This guidance was aimed at ensuring that each participant was well-prepared and understood their specific role in the discussion.

Moderators should play a crucial role in steering the conversation. They are responsible for ensuring that all panelists have an opportunity to contribute and that the audience is engaged effectively. Specific tasks include preparing opening remarks, introducing panelists, and crafting transition questions to facilitate smooth topic transitions. The moderators should also manage the time to ensure balanced participation and encourage active audience involvement.

Panelists are expected to be subject matter experts who bring valuable insights and opinions to the discussion. They are advised to conduct thorough research on the topic and prepare concise talking points. Panelists are encouraged to draw from their medical knowledge and relevant experiences, share evidence-based information, and engage with other panelists’ points through active listening and thoughtful responses.

The audience plays an active role in the PDs. They are encouraged to participate by asking questions, sharing relevant experiences, and contributing to the dialogue. To facilitate this, students are advised to take notes during the discussion and think of questions or comments they can contribute during the Q&A segment.

For this special course, medical students were advised to choose topics either from their ESP textbook or consider current medical trends, emerging research, and pressing issues in their field. Examples included breast cancer, COVID-19, and controversies in gene therapy. The selection process involved brainstorming sessions and consultation with the course instructor to ensure relevance and appropriateness.

To accommodate the PD sessions within the course structure, students were allowed to start their PD sessions voluntarily from the second week. However, to maintain a balance between peer-led discussions and regular course content, only one PD was held weekly. This approach enabled the ESP lecturer to deliver comprehensive content while also allowing students to engage in these interactive sessions.

A basic time structure was suggested for each PD (Fig.  2 ):

Time allocation for panel discussion stages in minutes

To ensure the smooth running of the course and maintain momentum, students were informed that they could cancel their PD session only once. In such cases, they were required to notify the lecturer and other students via the class Telegram channel to facilitate rescheduling and minimize disruptions. This provision was essential in promoting a sense of community among students and maintaining the course’s continuity.

Research tools and data collection

The study utilized various tools to gather and analyze data from participants and experts, ensuring a comprehensive understanding of the research topic.

Reflection papers

In Phase 1 of the study, 46 medical students detailed their perceptions of the advantages and disadvantages of panel discussions from dual perspectives: as panelists (presenters) and as audience members (peers).

Participants were given clear instructions and a 45-minute time frame to complete the reflection task. With approximately 80% of the international language students being native English speakers and the rest fluent in English, the researchers deemed this time allocation reasonable. The questions and instructions were straightforward, facilitating quick comprehension. It was estimated that native English speakers would need about 30 min to complete the task, while non-native speakers might require an extra 15 min for clarity and expression. This time frame aimed to allow students to respond thoughtfully without feeling rushed. Additionally, students could request more time if needed.

Focus group discussion

In phase 2 of the study, a focus group discussion was conducted with three expert participants. The purpose of the focus group was to gather insights from expert participants, specifically ESP (English for Specific Purposes) instructors, on how presentation dynamics can be improved for both panelists and the audience.

According to Colton and Covert [ 35 ], focus groups are useful for obtaining detailed input from experts. The appropriate size of a focus group is determined by the study’s scope and available resources [ 36 ]. Morgan [ 37 ] suggests that small focus groups are suitable for complex topics where specialist participants might feel frustrated if not allowed to express themselves fully.

The choice of a focus group over individual interviews was based on several factors. First, the exploratory nature of the study made focus groups ideal for interactive discussions, generating new ideas and in-depth insights [ 36 ]. Second, while focus groups usually involve larger groups, they can effectively accommodate a limited number of experts with extensive knowledge [ 37 ]. Third, the focus group format fostered a more open environment for idea exchange, allowing participants to engage dynamically [ 36 ]. Lastly, conducting a focus group was more time- and resource-efficient than scheduling three separate interviews [ 36 ].

Data analysis

The first phase of the study involved a thorough examination of the data related to the research inquiries using thematic analysis. This method was chosen for its effectiveness in uncovering latent patterns from a bottom-up perspective, facilitating a comprehensive understanding of complex educational phenomena [ 38 ]. The researchers first familiarized themselves with the data by repeatedly reviewing the reflection papers written by the medical students. Next, an initial round of coding was independently conducted to identify significant data segments and generate preliminary codes that reflected the students’ perceptions of the advantages and disadvantages of presentation dynamics PDs from both the presenter and audience viewpoints [ 38 ].

The analysis of the reflection papers began with the two researchers coding a subset of five papers independently, adhering to a structured qualitative coding protocol [ 39 ]. They convened afterward to compare their initial codes and address any discrepancies. Through discussion, they reached an agreement on the codes, which were then analyzed, organized into categories and themes, and the frequency of each code was recorded [ 38 ].

After coding the initial five papers, the researchers continued to code the remaining 41 reflection paper transcripts in batches of ten, meeting after each batch to review their coding, resolve any inconsistencies, and refine the coding framework as needed. This iterative process, characterized by independent coding, joint reviews, and consensus-building, helped the researchers establish a robust and reliable coding approach consistently applied to the complete dataset [ 40 ]. Once all 46 reflection paper transcripts were coded, the researchers conducted a final review and discussion to ensure accurate analysis. They extracted relevant excerpts corresponding to the identified themes and sub-themes from the transcripts to provide detailed explanations and support for their findings [ 38 ]. This multi-step approach of separate initial coding, collaborative review, and frequency analysis enhanced the credibility and transparency of the qualitative data analysis.

To ensure the trustworthiness of the data collected in this study, the researchers adhered to the Guba and Lincoln standards of scientific accuracy in qualitative research, which encompass credibility, confirmability, dependability, and transferability [ 41 ] (Table  2 ).

The analysis of the focus group data obtained from experts followed the same rigorous procedure applied to the student participants’ data. Thematic analysis was employed to examine the experts’ perspectives, maintaining consistency in the analytical approach across both phases of the study. The researchers familiarized themselves with the focus group transcript, conducted independent preliminary coding, and then collaboratively refined the codes. These codes were subsequently organized into categories and themes, with the frequency of each code recorded. The researchers engaged in thorough discussions to ensure agreement on the final themes and sub-themes. Relevant excerpts from the focus group transcript were extracted to provide rich, detailed explanations of each theme, thereby ensuring a comprehensive and accurate analysis of the experts’ insights.

1. What are the advantages of PDs from the perspective of panelists and the audience?

The analysis of the advantages of PDs from the perspectives of both panelists and audience members revealed several key themes and categories. Tables  2 and 3 present the frequency and percentage of responses for each code within these categories.

From the panelists’ perspective (Table  3 ), the overarching theme was “Personal and Professional Development.” The most frequently reported advantage was knowledge sharing (93.5%), followed closely by increased confidence (91.3%) and the importance of interaction in presentations (91.3%).

Notably, all categories within this theme had at least one code mentioned by over 80% of participants, indicating a broad range of perceived benefits. The category of “Effective teamwork and communication” was particularly prominent, with collaboration (89.1%) and knowledge sharing (93.5%) being among the most frequently cited advantages. This suggests that PDs are perceived as valuable tools for fostering interpersonal skills and collective learning. In the “Language mastery” category, increased confidence (91.3%) and better retention of key concepts (87.0%) were highlighted, indicating that PDs are seen as effective for both language and content learning.

The audience perspective (Table  4 ), encapsulated under the theme “Enriching Learning Experience,” showed similarly high frequencies across all categories.

The most frequently mentioned advantage was exposure to diverse speakers (93.5%), closely followed by the range of topics covered (91.3%) and increased audience interest (91.3%). The “Broadening perspectives” category was particularly rich, with all codes mentioned by over 70% of participants. This suggests that audience members perceive PDs as valuable opportunities for expanding their knowledge and viewpoints. In the “Language practice” category, the opportunity to practice language skills (89.1%) was the most frequently cited advantage, indicating that even as audience members, students perceive significant language learning benefits.

Comparing the two perspectives reveals several interesting patterns:

High overall engagement: Both panelists and audience members reported high frequencies across all categories, suggesting that PDs are perceived as beneficial regardless of the role played.

Language benefits: While panelists emphasized increased confidence (91.3%) and better retention of concepts (87.0%), audience members highlighted opportunities for language practice (89.1%). This indicates that PDs offer complementary language learning benefits for both roles.

Interactive learning: The importance of interaction was highly rated by panelists (91.3%), while increased audience interest was similarly valued by the audience (91.3%). This suggests that PDs are perceived as an engaging, interactive learning method from both perspectives.

Professional development: Panelists uniquely emphasized professional growth aspects such as experiential learning (84.8%) and real-world application (80.4%). These were not directly mirrored in the audience perspective, suggesting that active participation in PDs may offer additional professional development benefits.

Broadening horizons: Both groups highly valued the diversity aspect of PDs. Panelists appreciated diversity and open-mindedness (80.4%), while audience members valued diverse speakers (93.5%) and a range of topics (91.3%).

2. What are the disadvantages of PDs from the perspective of panelists and the audience?

The analysis of the disadvantages of panel discussions (PDs) from the perspectives of both panelists and audience members revealed several key themes and categories. Tables  4 and 5 present the frequency and percentage of responses for each code within these categories.

From the panelists’ perspective (Table  5 ), the theme “Drawbacks of PDs” was divided into two main categories: “Academic Workload Challenges” and “Coordination Challenges.” The most frequently reported disadvantage was long preparation (87.0%), followed by significant practice needed (82.6%) and the time-consuming nature of PDs (80.4%). These findings suggest that the primary concern for panelists is the additional workload that PDs impose on their already demanding academic schedules. The “Coordination Challenges” category, while less prominent than workload issues, still presented significant concerns. Diverse panel skills (78.3%) and finding suitable panelists (73.9%) were the most frequently cited issues in this category, indicating that team dynamics and composition are notable challenges for panelists.

The audience perspective (Table  6 ), encapsulated under the theme “Drawbacks of PDs,” was divided into two main categories: “Time-related Issues” and “Interaction and Engagement Issues.” In the “Time-related Issues” category, the most frequently mentioned disadvantage was the inefficient use of time (65.2%), followed by the perception of PDs as too long and boring (60.9%). Notably, 56.5% of respondents found PDs stressful due to overwhelming workload from other studies, and 52.2% considered them not very useful during exam time. The “Interaction and Engagement Issues” category revealed more diverse concerns. The most frequently mentioned disadvantage was the repetitive format (82.6%), followed by limited engagement with the audience (78.3%) and the perception of PDs as boring (73.9%). The audience also noted issues related to the panelists’ preparation and coordination, such as “Not practiced and natural” (67.4%) and “Coordination and Interaction Issues” (71.7%), suggesting that the challenges faced by panelists directly impact the audience’s experience.

Workload concerns: Both panelists and audience members highlighted time-related issues. For panelists, this manifested as long preparation times (87.0%) and difficulty balancing with other studies (76.1%). For the audience, it appeared as perceptions of inefficient use of time (65.2%) and stress due to overwhelming workload from other studies (56.5%).

Engagement issues: While panelists focused on preparation and coordination challenges, the audience emphasized the quality of the discussion and engagement. This suggests a potential mismatch between the efforts of panelists and the expectations of the audience.

Boredom and repetition: The audience frequently mentioned boredom (73.9%) and repetitive format (82.6%) as issues, which weren’t directly mirrored in the panelists’ responses. This indicates that while panelists may be focused on content preparation, the audience is more concerned with the delivery and variety of the presentation format.

Coordination challenges: Both groups noted coordination issues, but from different perspectives. Panelists struggled with team dynamics and finding suitable co-presenters, while the audience observed these challenges manifesting as unnatural or unpracticed presentations.

Academic pressure: Both groups acknowledged the strain PDs put on their academic lives, with panelists viewing it as a burden (65.2%) and the audience finding it less useful during exam times (52.2%).

3. How can PDs be improved for panelists and the audience from the experts’ point of view?

The presentation of data for this research question differs from the previous two due to the unique nature of the information gathered. Unlike the quantifiable student responses in earlier questions, this data stems from expert opinions and a reflection discussion session, focusing on qualitative recommendations for improvement rather than frequency of responses (Braun & Clarke, 2006). The complexity and interconnectedness of expert suggestions, coupled with the integration of supporting literature, necessitate a more narrative approach (Creswell & Poth, 2018). This format allows for a richer exploration of the context behind each recommendation and its potential implications (Patton, 2015). Furthermore, the exploratory nature of this question, aimed at generating ideas for improvement rather than measuring prevalence of opinions, is better served by a detailed, descriptive presentation (Merriam & Tisdell, 2016). This approach enables a more nuanced understanding of how PDs can be enhanced, aligning closely with the “how” nature of the research question and providing valuable insights for potential implementation (Yin, 2018).

The experts provided several suggestions to address the challenges faced by students in panel discussions (PDs) and improve the experience for both panelists and the audience. Their recommendations focused on six key areas: time management and workload, preparation and skill development, engagement and interactivity, technological integration, collaboration and communication, and institutional support.

To address the issue of time management and heavy workload, one expert suggested teaching students to “ break down the task to tackle the time-consuming nature of panel discussions and balance it with other studies .” This approach aims to help students manage the extensive preparation time required for PDs without compromising their other academic responsibilities. Another expert emphasized “ enhancing medical students’ abilities to prioritize tasks , allocate resources efficiently , and optimize their workflow to achieve their goals effectively .” These skills were seen as crucial not only for PD preparation but also for overall academic success and future professional practice.

Recognizing the challenges of long preparation times and the perception of PDs being burdensome, an expert proposed “ the implementation of interactive training sessions for panelists .” These sessions were suggested to enhance coordination skills and improve the ability of group presenters to engage with the audience effectively. The expert emphasized that such training could help students view PDs as valuable learning experiences rather than additional burdens, potentially increasing their motivation and engagement in the process.

To combat issues of limited engagement and perceived boredom, experts recommended increasing engagement opportunities for the audience through interactive elements like audience participation and group discussions. They suggested that this could transform PDs from passive listening experiences to active learning opportunities. One expert suggested “ optimizing time management and restructuring the format of panel discussions ” to address inefficiency during sessions. This restructuring could involve shorter presentation segments interspersed with interactive elements to maintain audience attention and engagement.

An innovative solution proposed by one expert was “ using ChatGPT to prepare for PDs by streamlining scenario presentation preparation and role allocation. ” The experts collectively discussed the potential of AI to assist medical students in reducing their workload and saving time in preparing scenario presentations and allocating roles in panel discussions. They noted that AI could help generate initial content drafts, suggest role distributions based on individual strengths, and even provide practice questions for panelists, significantly reducing preparation time while maintaining quality.

Two experts emphasized the importance of enhancing collaboration and communication among panelists to address issues related to diverse panel skills and coordination challenges. They suggested establishing clear communication channels and guidelines to improve coordination and ensure a cohesive presentation. This could involve creating structured team roles, setting clear expectations for each panelist, and implementing regular check-ins during the preparation process to ensure all team members are aligned and progressing.

All experts were in agreement that improving PDs would not be possible “ if nothing is done by the university administration to reduce the ESP class size for international students .” They believed that large class sizes in ESP or EFL classes could negatively influence group oral presentations, hindering language development and leading to uneven participation. The experts suggested that smaller class sizes would allow for more individualized attention, increased speaking opportunities for each student, and more effective feedback mechanisms, all of which are crucial for developing strong presentation skills in a second language.

Research question 1: what are the advantages of PDs from the perspective of panelists and the audience?

The results of this study reveal significant advantages of PDs for both panelists and audience members in the context of medical education. These findings align with and expand upon previous research in the field of educational presentations and language learning.

Personal and professional development for panelists

The high frequency of reported benefits in the “Personal and Professional Development” theme for panelists aligns with several previous studies. The emphasis on language mastery, particularly increased confidence (91.3%) and better retention of key concepts (87.0%), supports the findings of Hartono, Mujiyanto [ 42 ], Gedamu and Gezahegn [ 15 ], Li [ 43 ], who all highlighted the importance of language practice in English oral presentations. However, our results show a more comprehensive range of benefits, including professional growth aspects like experiential learning (84.8%) and real-world application (80.4%), which were not as prominently featured in these earlier studies.

Interestingly, our findings partially contrast with Chou [ 44 ] study, which found that while group oral presentations had the greatest influence on improving students’ speaking ability, individual presentations led to more frequent use of metacognitive, retrieval, and rehearsal strategies. Our results suggest that PDs, despite being group activities, still provide significant benefits in these areas, possibly due to the collaborative nature of preparation and the individual responsibility each panelist bears. The high frequency of knowledge sharing (93.5%) and collaboration (89.1%) in our study supports Harris, Jones and Huffman [ 45 ] emphasis on the importance of group dynamics and varied perspectives in educational settings. However, our study provides more quantitative evidence for these benefits in the specific context of PDs.

Enriching learning experience for the audience

The audience perspective in our study reveals a rich learning experience, with high frequencies across all categories. This aligns with Agustina [ 46 ] findings in business English classes, where presentations led to improvements in all four language skills. However, our study extends these findings by demonstrating that even passive participation as an audience member can lead to significant perceived benefits in language practice (89.1%) and broadening perspectives (93.5% for diverse speakers). The high value placed on diverse speakers (93.5%) and range of topics (91.3%) by the audience supports the notion of PDs as a tool for expanding knowledge and viewpoints. This aligns with the concept of situated learning experiences leading to deeper understanding in EFL classes, as suggested by Li [ 43 ] and others [ 18 , 31 ]. However, our study provides more specific evidence for how this occurs in the context of PDs.

Interactive learning and engagement

Both panelists and audience members in our study highly valued the interactive aspects of PDs, with the importance of interaction rated at 91.3% by panelists and increased audience interest at 91.3% by the audience. This strong emphasis on interactivity aligns with Azizi and Farid Khafaga [ 19 ] study on the benefits of dynamic assessment and dialogic learning contexts. However, our study provides more detailed insights into how this interactivity is perceived and valued by both presenters and audience members in PDs.

Professional growth and real-world application

The emphasis on professional growth through PDs, particularly for panelists, supports Li’s [ 43 ] assertion about the power of oral presentations as situated learning experiences. Our findings provide more specific evidence for how PDs contribute to professional development, with high frequencies reported for experiential learning (84.8%) and real-world application (80.4%). This suggests that PDs may be particularly effective in bridging the gap between academic learning and professional practice in medical education.

Research question 2: what are the disadvantages of pds from the perspective of panelists and the audience?

Academic workload challenges for panelists.

The high frequency of reported challenges in the “Academic Workload Challenges” category for panelists aligns with several previous studies in medical education [ 47 , 48 , 49 ]. The emphasis on long preparation (87.0%), significant practice needed (82.6%), and the time-consuming nature of PDs (80.4%) supports the findings of Johnson et al. [ 24 ], who noted that while learners appreciate debate-style journal clubs in health professional education, they require additional time commitment. This is further corroborated by Nowak, Speed and Vuk [ 50 ], who found that intensive learning activities in medical education, while beneficial, can be time-consuming for students.

Perceived value of pds relative to time investment

While a significant portion of the audience (65.2%) perceived PDs as an inefficient use of time, the high frequency of engagement-related concerns (82.6% for repetitive format, 78.3% for limited engagement) suggests that the perceived lack of value may be more closely tied to the quality of the experience rather than just the time investment. This aligns with Dyhrberg O’Neill [ 27 ] findings on debate-based oral exams, where students perceived value despite the time-intensive nature of the activity. However, our results indicate a more pronounced concern about the return on time investment in PDs. This discrepancy might be addressed through innovative approaches to PD design and implementation, such as those proposed by Almazyad et al. [ 22 ], who suggested using AI tools to enhance expert panel discussions and potentially improve efficiency.

Coordination challenges for panelists

The challenges related to coordination in medical education, such as diverse panel skills (78.3%) and finding suitable panelists (73.9%), align with previous research on teamwork in higher education [ 21 ]. Our findings support the concept of the free-rider effect discussed by Hall and Buzwell [ 21 ], who explored reasons for non-contribution in group projects beyond social loafing. This is further elaborated by Mehmood, Memon and Ali [ 51 ], who proposed that individuals may not contribute their fair share due to various factors including poor communication skills or language barriers, which is particularly relevant in medical education where clear communication is crucial [ 52 ]. Comparing our results to other collaborative learning contexts in medical education, Rodríguez-Sedano, Conde and Fernández-Llamas [ 53 ] measured teamwork competence development in a multidisciplinary project-based learning environment. They found that while teamwork skills improved over time, initial coordination challenges were significant. This aligns with our findings on the difficulties of coordinating diverse panel skills and opinions in medical education settings.

Our results also resonate with Chou’s [ 44 ] study comparing group and individual oral presentations, which found that group presenters often had a limited understanding of the overall content. This is supported by Wilson, Ho and Brookes [ 54 ], who examined student perceptions of teamwork in undergraduate science degrees, highlighting the challenges and benefits of collaborative work, which are equally applicable in medical education [ 52 ].

Quality of discussions and perception for the audience

The audience perspective in our study reveals significant concerns about the quality and engagement of PDs in medical education. The high frequency of issues such as repetitive format (82.6%) and limited engagement with the audience (78.3%) aligns with Parmar and Bickmore [ 55 ] findings on the importance of addressing individual audience members and gathering feedback. This is further supported by Nurakhir et al. [ 25 ], who explored students’ views on classroom debates as a strategy to enhance critical thinking and oral communication skills in nursing education, which shares similarities with medical education. Comparing our results to other interactive learning methods in medical education, Jones et al. [ 26 ] reviewed the use of journal clubs and book clubs in pharmacy education. They found that while these methods enhanced engagement, they also faced challenges in maintaining student interest over time, similar to the boredom issues reported in our study of PDs in medical education. The perception of PDs as boring (73.9%) and not very useful during exam time (52.2%) supports previous research on the stress and pressure experienced by medical students [ 48 , 49 ]. Grieve et al. [ 20 ] specifically examined student fears of oral presentations and public speaking in higher education, which provides context for the anxiety and disengagement observed in our study of medical education. Interestingly, Bhuvaneshwari et al. [ 23 ] found positive impacts of panel discussions in educating medical students on specific modules. This contrasts with our findings and suggests that the effectiveness of PDs in medical education may vary depending on the specific context and implementation.

Comparative analysis and future directions

Our study provides a unique comparative analysis of the challenges faced by both panelists and audience members in medical education. The alignment of concerns around workload and time management between the two groups suggests that these are overarching issues in the implementation of PDs in medical curricula. This is consistent with the findings of Pasandín et al. [ 56 ], who examined cooperative oral presentations in higher education and their impact on both technical and soft skills, which are crucial in medical education [ 52 ]. The mismatch between panelist efforts and audience expectations revealed in our study is a novel finding that warrants further investigation in medical education. This disparity could be related to the self-efficacy beliefs of presenters, as explored by Gedamu and Gezahegn [ 15 ] in their study of TEFL trainees’ attitudes towards academic oral presentations, which may have parallels in medical education. Looking forward, innovative approaches could address some of the challenges identified in medical education. Almazyad et al. [ 22 ] proposed using AI tools like ChatGPT to enhance expert panel discussions in pediatric palliative care, which could potentially address some of the preparation and engagement issues identified in our study of medical education. Additionally, Ragupathi and Lee [ 57 ] discussed the role of rubrics in higher education, which could provide clearer expectations and feedback for both panelists and audience members in PDs within medical education.

Research question 3: how can PDs be improved for panelists and the audience from the experts’ point of view?

The expert suggestions for improving PDs address several key challenges identified in previous research on academic presentations and student workload management. These recommendations align with current trends in educational technology and pedagogical approaches, while also considering the unique needs of medical students.

The emphasis on time management and workload reduction strategies echoes findings from previous studies on medical student stress and academic performance. Nowak, Speed and Vuk [ 50 ] found that medical students often struggle with the fast-paced nature of their courses, which can lead to reduced motivation and superficial learning approaches. The experts’ suggestions for task breakdown and prioritization align with Rabbi and Islam [ 58 ] recommendations for reducing workload stress through effective assignment prioritization. Additionally, Popa et al. [ 59 ] highlight the importance of acceptance and planning in stress management for medical students, supporting the experts’ focus on these areas.

The proposed implementation of interactive training sessions for panelists addresses the need for enhanced presentation skills in professional contexts, a concern highlighted by several researchers [ 17 , 60 ]. This aligns with Grieve et al. [ 20 ] findings on student fears of oral presentations and public speaking in higher education, emphasizing the need for targeted training. The focus on interactive elements and audience engagement also reflects current trends in active learning pedagogies, as demonstrated by Pasandín et al. [ 56 ] in their study on cooperative oral presentations in engineering education.

The innovative suggestion to use AI tools like ChatGPT for PD preparation represents a novel approach to leveraging technology in education. This aligns with recent research on the potential of AI in scientific research, such as the study by Almazyad et al. [ 22 ], which highlighted the benefits of AI in supporting various educational tasks. However, it is important to consider potential ethical implications and ensure that AI use complements rather than replaces critical thinking and creativity.

The experts’ emphasis on enhancing collaboration and communication among panelists addresses issues identified in previous research on teamwork in higher education. Rodríguez-Sedano, Conde and Fernández-Llamas [ 53 ] noted the importance of measuring teamwork competence development in project-based learning environments. The suggested strategies for improving coordination align with best practices in collaborative learning, as demonstrated by Romero-Yesa et al. [ 61 ] in their qualitative assessment of challenge-based learning and teamwork in electronics programs.

The unanimous agreement on the need to reduce ESP class sizes for international students reflects ongoing concerns about the impact of large classes on language learning and student engagement. This aligns with research by Li [ 3 ] on issues in developing EFL learners’ oral English communication skills. Bosco et al. [ 62 ] further highlight the challenges of teaching and learning ESP in mixed classes, supporting the experts’ recommendation for smaller class sizes. Qiao, Xu and bin Ahmad [ 63 ] also emphasize the implementation challenges for ESP formative assessment in large classes, further justifying the need for reduced class sizes.

These expert recommendations provide a comprehensive approach to improving PDs, addressing not only the immediate challenges of preparation and delivery but also broader issues of student engagement, workload management, and institutional support. By implementing these suggestions, universities could potentially transform PDs from perceived burdens into valuable learning experiences that enhance both academic and professional skills. This aligns with Kho and Ting [ 64 ] systematic review on overcoming oral presentation anxiety among tertiary ESL/EFL students, which emphasizes the importance of addressing both challenges and strategies in improving presentation skills.

This study has shed light on the complex challenges associated with PDs in medical education, revealing a nuanced interplay between the experiences of panelists and audience members. The findings underscore the need for a holistic approach to implementing PDs that addresses both the academic workload concerns and the quality of engagement.

Our findings both support and extend previous research on the challenges of oral presentations and group work in medical education settings. The high frequencies of perceived challenges across multiple categories for both panelists and audience members suggest that while PDs may offer benefits, they also present significant obstacles that need to be addressed in medical education. These results highlight the need for careful consideration in the implementation of PDs in medical education, with particular attention to workload management, coordination strategies, and audience engagement techniques. Future research could focus on developing and testing interventions to mitigate these challenges while preserving the potential benefits of PDs in medical education.

Moving forward, medical educators should consider innovative approaches to mitigate these challenges. This may include:

Integrating time management and stress coping strategies into the PD preparation process [ 59 ].

Exploring the use of AI tools to streamline preparation and enhance engagement [ 22 ].

Developing clear rubrics and expectations for both panelists and audience members [ 57 ].

Incorporating interactive elements to maintain audience interest and participation [ 25 ].

Limitations and future research

One limitation of this study is that it focused on a specific population of medical students, which may limit the generalizability of the findings to other student populations. Additionally, the study relied on self-report data from panelists and audience members, which may introduce bias and affect the validity of the results. Future research could explore the effectiveness of PDs in different educational contexts and student populations to provide a more comprehensive understanding of the benefits and challenges of panel discussions.

Future research should focus on evaluating the effectiveness of these interventions and exploring how PDs can be tailored to the unique demands of medical education. By addressing the identified challenges, PDs have the potential to become a more valuable and engaging component of medical curricula, fostering both academic and professional development. Ultimately, the goal should be to transform PDs from perceived burdens into opportunities for meaningful learning and skill development, aligning with the evolving needs of medical education in the 21st century.

Future research could also examine the long-term impact of PDs on panelists’ language skills, teamwork, and communication abilities. Additionally, exploring the effectiveness of different training methods and tools, such as AI technology, in improving coordination skills and reducing workload stress for panelists could provide valuable insights for educators and administrators. Further research could also investigate the role of class size and audience engagement in enhancing the overall effectiveness of PDs in higher education settings. By addressing these gaps in the literature, future research can contribute to the ongoing development and improvement of PDs as a valuable learning tool for students in higher education.

However, it is important to note that implementing these changes may require significant institutional resources and a shift in pedagogical approaches. Future research could focus on piloting these recommendations and evaluating their effectiveness in improving student outcomes and experiences with PDs.

Data availability

We confirm that the data supporting the findings are available within this article. Raw data supporting this study’s findings are available from the corresponding author, upon request.

Abbreviations

Artificial Intelligence

English as a Foreign Language

English for Specific Purposes

Panel Discussion

Shiraz University of Medical Sciences

Harden RM, Laidlaw JM. Essential skills for a medical teacher: an introduction to teaching and learning in medicine. Elsevier Health Sciences; 2020.

Ibrahim Mohamed O, Al Jadaan DO. English for Specific purposes (Esp) Needs Analysis for Health Sciences students: a cross-sectional study at a University in the UAE. English for Specific purposes (Esp) Needs Analysis for Health Sciences Students: A Cross-Sectional Study at a University in the UAE.

Li Y, Heron M. English for general academic purposes or English for specific purposes? Language learning needs of medical students at a Chinese university. Theory Pract Lang Stud. 2021;11(6):621–31.

Article   Google Scholar  

Chan SMH, Mamat NH, Nadarajah VD. Mind your language: the importance of English language skills in an International Medical Programme (IMP). BMC Med Educ. 2022;22(1):405.

Cortez Faustino BS, Ticas de Córdova CK, de la Hernández DI. Teaching English for specific purposes: contents and methodologies that could be implemented in the English for Medical purposes (EMP) course for the doctor of Medicine Major at the University of El Salvador. Universidad de El Salvador; 2022.

BENYAMINA E-Z BOUKAHLAH. Enhancing Specialty Language learning through content-based instruction: students of Paramedical Institute of Tiaret as a case study. Université IBN KHALDOUN-Tiaret; 2023.

Prikazchikov M. Medical English course for russian-speaking dentists: a needs analysis study. Iowa State University; 2024.

Kim C, Lee SY, Park S-H. Is Korea Ready to be a key player in the Medical Tourism Industry? An English Education Perspective. Iran J Public Health. 2020;49(2):267–73.

Google Scholar  

Syakur A, Zainuddin H, Hasan MA. Needs analysis English for specific purposes (esp) for vocational pharmacy students. Budapest International Research and Critics in Linguistics and Education (BirLE). Journal. 2020;3(2):724–33.

Chan S, Taylor L. Comparing writing proficiency assessments used in professional medical registration: a methodology to inform policy and practice. Assess Writ. 2020;46:100493.

Hyland K, Jiang FK. Delivering relevance: the emergence of ESP as a discipline. Engl Specif Purp. 2021;64:13–25.

Maftuna B. The role of English in ESP. Am J Adv Sci Res. 2024;1(2):1–5.

LEON LI, HUMANIZING THE FOREIGN LANGUAGE. COURSE: NEW TEACHING METHODS FOR MEDICAL STUDENTS. Language, Culture and Change. 2022:243.

Dahm MR, Yates L. Rapport, empathy and professional identity: Some challenges for international medical graduates speaking English as a second or foreign language. Multilingual Healthcare: A Global View on Communicative Challenges. 2020:209 – 34.

Gedamu AD, Gezahegn TH. TEFL trainees’ attitude to and self-efficacy beliefs of academic oral presentation. Cogent Educ. 2023;10(1):2163087.

Saliu B, Hajrullai H. Best practices in the English for specific purpose classes at the language center. Procedia-Social Behav Sci. 2016;232:745–9.

Clokie TL, Fourie E. Graduate employability and communication competence: are undergraduates taught relevant skills? Bus Prof Communication Q. 2016;79(4):442–63.

Hartono H, Mujiyanto J, Fitriati SW, Sakhiyya Z, Lotfie MM, Maharani MM. English Presentation Self-Efficacy Development of Indonesian ESP students: the effects of Individual versus Group Presentation tasks. Int J Lang Educ. 2023;7(3):361–76.

Azizi Z, Farid Khafaga A. Scaffolding via Group-dynamic Assessment to positively affect motivation, learning anxiety, and willingness to Communicate: a Case Study of High School Students. J Psycholinguist Res. 2023;52(3):831–51.

Grieve R, Woodley J, Hunt SE, McKay A. Student fears of oral presentations and public speaking in higher education: a qualitative survey. J Furth High Educ. 2021;45(9):1281–93.

Hall D, Buzwell S. The problem of free-riding in group projects: looking beyond social loafing as reason for non-contribution. Act Learn High Educ. 2013;14(1):37–49.

Almazyad M, Aljofan F, Abouammoh NA, Muaygil R, Malki KH, Aljamaan F, et al. Enhancing Expert Panel discussions in Pediatric Palliative Care: innovative scenario development and summarization with ChatGPT-4. Cureus. 2023;15(4):e38249.

Bhuvaneshwari S, Rashmi R, Deepika K, Anirudh VM, Vijayamathy A, Rekha S, Kathiravan R. Impact of panel discussion in educating AETCOM First Module among Undergraduate Medical Students. Latin Am J Pharmacy: Life Sci J. 2023;42(6):407–12.

Johnson BR, Logan LD, Darley A, Stone RH, Smith SE, Osae SP, et al. A scoping review for Debate-Style Journal Clubs in Health Professional Education. Am J Pharm Educ. 2023;87(6):100064.

Nurakhir A, Palupi FN, Langeveld C, Nurmalia D. Students’ views of classroom debates as a strategy to enhance critical thinking and oral communication skills. 2020.

Jones EP, Nelson NR, Thorpe CT, Rodgers PT, Carlson RB. Use of journal clubs and book clubs in pharmacy education: a scoping review. Currents Pharm Teach Learn. 2022;14(1):110–9.

Dyhrberg O’Neill L. Assessment of student debates in support of active learning? Students’ perceptions of a debate-based oral final exam. Act Learn High Educ. 2024.

Dyment JE, O’Connell TS. Assessing the quality of reflection in student journals: a review of the research. Teach High Educ. 2011;16(1):81–97.

Villarroel V, Bloxham S, Bruna D, Bruna C, Herrera-Seda C. Authentic assessment: creating a blueprint for course design. Assess Evaluation High Educ. 2018;43(5):840–54.

Schultz M, Young K, Gunning K, Harvey T. Defining and measuring authentic assessment: a case study in the context of tertiary science. Assess Evaluation High Educ. 2022;47(1):77–94.

Sundrarajun C, Kiely R. The oral presentation as a context for learning and assessment. Innov Lang Learn Teach. 2010;4(2):101–17.

Wyatt-Smith C, Adie L. The development of students’ evaluative expertise: enabling conditions for integrating criteria into pedagogic practice. J Curriculum Stud. 2021;53(4):399–419.

Boud D, Lawson R, Thompson DG. The calibration of student judgement through self-assessment: disruptive effects of assessment patterns. High Educ Res Dev. 2015;34(1):45–59.

A. S. Enhancing Meaningful Learning experiences through Comprehension and Retention by students. Twentyfirst Century Publications Patiala. 2023;49.

Colton D, Covert RW. Designing and constructing instruments for social research and evaluation. Wiley; 2007.

Krueger RA, Casey MA. Focus group interviewing. Handbook of practical program evaluation. 2015:506 – 34.

Morgan DL. Handbook of interview research: Context and method. Oaks, CA, USA: Sage Publications Thousand; 2002.

Braun V, Clarke V. Conceptual and design thinking for thematic analysis. Qualitative Psychol. 2022;9(1):3.

Elliott V. Thinking about the coding process in qualitative data analysis. Qualitative Rep. 2018;23(11).

Syed M, Nelson SC. Guidelines for establishing reliability when coding narrative data. Emerg Adulthood. 2015;3(6):375–87.

Lincoln Y. Naturalistic inquiry: Sage; 1985.

Hartono H, Mujiyanto J, Fitriati SW, Sakhiyya Z, Lotfie MM, Maharani MM. English presentation self-efficacy development of Indonesian ESP students: the effects of Individual versus Group Presentation tasks. Int J Lang Educ. 2023;7(3).

Li X. Teaching English oral presentations as a situated task in an EFL classroom: a quasi-experimental study of the effect of video-assisted self-reflection. Revista Signos. 2018;51(98):359–81.

Chou M-h. The influence of learner strategies on oral presentations: a comparison between group and individual performance. Engl Specif Purp. 2011;30(4):272–85.

Harris A, Jones M, Huffman J. Teachers leading educational reform. The power of. 2017.

Agustina L. Stimulating students to speak up through presentation in business English class. J Appl Stud Lang. 2019;3(1):21–8.

Babal JC, Abraham O, Webber S, Watterson T, Moua P, Chen J. Student pharmacist perspectives on factors that influence wellbeing during pharmacy school. Am J Pharm Educ. 2020;84(9):ajpe7831.

Moir F, Yielder J, Sanson J, Chen Y. Depression in medical students: current insights. Adv Med Educ Pract. 2018;323:33.

Pavlinac Dodig I, Lusic Kalcina L, Demirovic S, Pecotic R, Valic M, Dogas Z. Sleep and lifestyle habits of medical and non-medical students during the COVID-19 lockdown. Behav Sci. 2023;13(5):407.

Nowak G, Speed O, Vuk J. Microlearning activities improve student comprehension of difficult concepts and performance in a biochemistry course. Currents Pharm Teach Learn. 2023;15(1):69–78.

Mehmood K, Memon S, Ali F. Language barriers to Effective Communication in speaking English: a phenomenological study of Pakistan International cricketers. Pakistan Lang Humanit Rev. 2024;8(1):107–14.

Buelow JR, Downs D, Jorgensen K, Karges JR, Nelson D. Building interdisciplinary teamwork among allied health students through live clinical case simulations. J Allied Health. 2008;37(2):e109–23.

Rodríguez-Sedano FJ, Conde M, Fernández-Llamas C, editors. Measuring teamwork competence development in a multidisciplinary project based learning environment. Learning and Collaboration Technologies Design, Development and Technological Innovation: 5th International Conference, LCT 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15–20, 2018, Proceedings, Part I 5; 2018: Springer.

Wilson L, Ho S, Brookes RH. Student perceptions of teamwork within assessment tasks in undergraduate science degrees. Assess Evaluation High Educ. 2018;43(5):786–99.

Parmar D, Bickmore T. Making it personal: addressing individual audience members in oral presentations using augmented reality. Proc ACM Interact Mob Wearable Ubiquitous Technol. 2020;4(2):1–22.

Pasandín AMR, Pérez IP, Iglesias PO, Díaz JJG. Cooperative oral presentations in higher education to enhance technical and soft skills in engineering students. Int J Continuing Eng Educ Life Long Learn. 2023;33(6):592–607.

Ragupathi K, Lee A. Beyond fairness and consistency in grading: The role of rubrics in higher education. Diversity and inclusion in global higher education: Lessons from across Asia. 2020:73–95.

Rabbi MF, Islam MS. The effect of academic stress and Mental anxiety among the students of Khulna University. Edukasiana: Jurnal Inovasi Pendidikan. 2024;3(3):280–99.

Popa CO, Schenk A, Rus A, Szasz S, Suciu N, Szabo DA, Cojocaru C. The role of acceptance and planning in stress management for medical students. Acta Marisiensis-Seria Med. 2020;66(3):101–5.

Christianson M, Payne S. Helping students develop skills for better presentations: Using the 20x20 format for presentation training. 語学研究. 2012;26:1–15.

Romero-Yesa S, Fonseca D, Aláez M, Amo-Filva D. Qualitative assessment of a challenge-based learning and teamwork applied in electronics program. Heliyon. 2023;9(12).

Bosco TJ, Gabriel B, Florence M, Gilbert N. Towards effective teaching and learning ESP in mixed classes: students’ interest, challenges and remedies. Int J Engl Literature Social Sci. 2020;5(2):506–16.

Qiao L, Xu Y, bin Ahmad N, An Analysis Of Implementation Challenges For English, For Specific Purposes (Esp) Formative Assessment Via Blended Learning Mode At Chinese Vocational Polytechnics. Journal Of Digital Education, Communication, And Arts (DECA). 2023;6(02):64–76.

Kho MG-W, Ting S-H. Overcoming oral presentation anxiety: a systematic review of Tertiary ESL/EFL Students’ challenges and strategies. Qeios. 2023.

Download references

We confirm that no funding was received for this work.

Author information

Authors and affiliations.

Department of English Language, School of Paramedical Sciences, Shiraz University of Medical Sciences, Shiraz, Iran

Elham Nasiri & Laleh Khojasteh

You can also search for this author in PubMed   Google Scholar

Contributions

L.KH was involved in writing the proposal, reviewing the text, analyzing the data, and writing the manuscript. E. N was involvedin designing the research and collecting and analyzing the data. Both authors have reviewed and approved the final version of the manuscript.

Corresponding author

Correspondence to Laleh Khojasteh .

Ethics declarations

Ethics approval and consent to participate.

Our study, entitled “Evaluating Panel Discussions in ESP Classes: An Exploration of International Medical Students’ and ESP Instructors’ Perspectives through Qualitative Research,” was reviewed by the Institutional Review Board (IRB) of the School of Paramedical Sciences, Shiraz University of Medical Sciences (SUMS). The IRB reviewed the study on August 14th, 2024, and determined that formal ethics approval or a reference number was not required. This decision was based on the fact that the research posed minimal risk to participants and focused solely on their educational experiences without involving any intervention or the collection of sensitive personal data.

Consent for publication

Not Applicable.

Competing interests

We confirm that there are no known conflicts of interest associated with this publication and that this work did not receive any financial support.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Nasiri, E., Khojasteh, L. Evaluating panel discussions in ESP classes: an exploration of international medical students’ and ESP instructors’ perspectives through qualitative research. BMC Med Educ 24 , 925 (2024). https://doi.org/10.1186/s12909-024-05911-3

Download citation

Received : 08 May 2024

Accepted : 14 August 2024

Published : 26 August 2024

DOI : https://doi.org/10.1186/s12909-024-05911-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Group oral presentations
• Medical students
• Panel discussions
• ESP courses

BMC Medical Education

ISSN: 1472-6920

• General enquiries: [email protected]

A Common Student Question: Which Is Easier, Quantitative or Qualitative Research?

Leading a peer mentoring scheme means that I get a constant stream of messages and emails from first- and second-year psychology students. It is that time of the year where second-year students are choosing their final-year units and planning what they want to do for their final year research project (their dissertation).

The most common question I receive is: ‘Which is easier, quantitative or qualitative research?’ Of course, some researchers will have some biased views on this – probably based on what they are involved with themselves. But any good researcher will know that there is no straightforward answer to this question.

I remind students that they need to consider their research question. I conceptualise it between two questions: ‘w hat’ and ‘w hy’

The ‘what’ questions typically relate to a research question that requires a quantitative analysis to get a view of what variables influence other variables, or even how and to what extent one variable influences other variables (note that I use influence here, but such a question may also seek to establish causality).

The ‘why’ question, in my mind, would typically require a qualitative analysis . Why are students not receptive to feedback? Why is there a spike in teenage STD contraction? These questions will require asking samples from the population you’re interested in.

Of course, as with most things, there are some exceptions to this rule. For example, a ‘what’ question may require a qualitative analysis. Such as: ‘How does stress at work relate to quality of life in people working night shifts?’ This inevitably means seeking out a sample of people working nights shift.

Alternatively, a ‘why’ question may require a quantitative analysis. But researchers tend to form these ‘why’ questions in the way of a hypothesis. They may have an initial ‘why’ question, but then reflect this in an experimental hypothesis. For example: Why a consumer behaves in a certain why or how they’d feel if a certain situation were to take place.

A lot of students are also concerned about the time consumption of research for a final-year dissertation project. It is important to recognise that one approach (quantitative versus qualitative) is not necessarily faster than the other.

I conceptualise the time consumption of the methods as following, and find this helps students (for quantitative, then qualitative respectively):

data collection > data analysis data collection < data analysis

I have also noticed something peculiar, and I believe I may have experienced this myself before getting more involved in research: statistics anxiety .

Many students are coming to me asking how hard statistics is and whether they will get lots of support from their supervisors on their ‘independent’ projects.

I know many current final-year students, and second-year students, who are opting for a qualitative research project just to avoid running statistical analyses. It is apparent that this reasoning for choosing a qualitative project is a wrong one, especially the aforementioned discussion on choosing a method based on your research question.

A ‘why’ question may require a quantitative analysis.

This raises an important question: Are universities failing to engage students in Research Methods and Statistics? Unfortunately, in my own opinion (as a student), the answer to this question is yes, yes they are.

However, there is a way to fix this. Universities need to realise that the current way of teaching Research Methods and Statistics is failing. I have had countless lectures on different statistical tests, which are important, but I have had to retain knowledge on different pieces of logic and philosophy, which is impractical. At the end of the day, the real world of research does not require this knowledge. It requires you to:

• Formulate a research question;
• Read the literature;
• Design an experiment (or qualitative alternative);
• Collect data;
• Analyse that data;
• Interpret your results.

In my second-year I had a multiple choice question section of my examination, which I strongly believe was pointless on many levels. I failed this section of the examination. The second half of the examination required me to read some SPSS outputs, interpret them, and write up design, results, and discussion (first paragraph) sections of a laboratory report.

I excelled this section of the examination. This, of course, is far more representative of real-life research. I also wonder why students are not being assessed on their quantitative knowledge via using software such as SPSS – this is one of the most common statistics software that researchers use in the real world .

Universities have a duty to teach students to decide for themselves which is most important.

The concern that I have here is that Research Methods and Statistics is not being taught, nor examined, in a practical or realistic way. Another concern I have is that universities are giving the limelight to quantitative methodology, and not giving enough to qualitative methodology. In my first- and second-year, I had six lab classes that were quantitative-based and only two that were qualitative-based – both of which based on thematic analysis and nothing else.

This will lead students to believing that qualitative methodology is secondary to quantitative methodology. I cannot help but find the irony in this. Psychologists, with a wealth of knowledge on behaviour and attitudes, are still yet to develop curricula that will make the researchers of tomorrow. Universities have a duty to teach students to decide for themselves which is most important. In the case of those lab classes I mentioned above, surely this should be a 50/50 split. 

I think academia needs to reflect about the current way in which Research Methods and Statistics is taught. The discipline really must pay attention to the apparent trade-off between quantitative and qualitative methodology and the impression that it makes on students.

Callum Mogridge is an undergraduate psychology student at the University of Manchester. He leads the peer support on the degree programme. 

VIEW AUTHOR’S PROFILE

Related articles, city and st george’s universities merge to form major london institution, the rising importance of mental health advocacy in protecting vulnerable populations, is the stress and cost of advanced graduate degrees worth the career boost, mit’s black and latino enrolment plummets after supreme court ruling on affirmative action, how to turn disappointment into opportunity after unexpected a-level or t level results, new immersive technology suites giving students a hands-on learning experience, university of winchester celebrates achievements of health professionals completing practice education programme, exam results: how to support your child on results day, “pure power curriculum” improves students’ well-being and academic performance, reveals new study, the perfect teacher gift – say thank you with a causebook notebook, are you saying these words wrong literacy expert reveals commonly mispronounced words in english, the impact of online tutoring on student mental health, what are some online tools that can help you improve your korean pronunciation, navigating university clearing: how parents can offer proactive support, university of sunderland’s first graduating medics set to take up north-east nhs roles, teacher movement and space use are crucial for responsive teaching, according to new study, taking a look at the intensity and mental health toll of advanced education, benefits of partnering with medical education firms for professional growth, robert gordon university sets out manifesto ahead of general election 2024, full speed ahead for university clearing: application gateway, top tips for conquering public speaking anxiety, tutors see potential in blended learning but face tech challenges, finds new study, plagiarism-free zone: how to foster a culture of academic integrity, how ai can help you write a good book summary.

Psychreg is a digital media company and not a clinical company. Our content does not constitute a medical or psychological consultation. See a certified medical or mental health professional for diagnosis.

• Privacy Policy

© Copyright 2014–2034 Psychreg Ltd

• PSYCHREG JOURNAL
• MEET OUR WRITERS
• MEET THE TEAM

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Where are the costs using an economic analysis of educational interventions approach to improve the evaluation of a regional school improvement programme.

1. Introduction

2. formative assessment in policy and practice.

• Making observations: The teacher needs to explore what the learner does or does not know, and this is typically achieved by listening to learners’ responses, observing the learner on tasks, and/or assessing class or homework tasks.
• Interpretation: The teacher interprets the skill, knowledge, or attitudes of the learners.
• Judgement: Once evidence has been gathered through observation and interpretation, the teacher then makes a judgement on the next course of action to move the learner forward.
• Sharing Learning Expectations: Ensuring the learner knows what they are going to learn and the success criteria to achieve this goal.
• Questioning: Using effective questioning to facilitate learning.
• Feedback: Providing feedback that enhances learning within the moment.
• Self-assessment: Allowing learners to take ownership of, and reflect on, their learning.
• Peer assessment: Providing opportunities for learners to discuss their work with, and to instruct, others.

3.1. Trial Design

3.2. recruitment, 3.3. study population, 3.4. outcomes, 3.5. analysis, 3.6. interviews, 3.7. focus group, 3.8. procedure, 3.9. observations.

• Is it clear what the teacher intends the students to learn?
• Does the teacher identify student learning needs?
• Do students understand what criteria will make their work successful?
• Are students chosen at random to answer questions?
• Does the teacher ask questions that make students think?
• Does the teacher give students time to think after asking a question?
• Does the teacher allow time for students to elaborate their responses?
• Is a whole-class response system used?
• Is teaching adjusted after gathering feedback from pupils (data collection)?
• Is there more student talk than teacher talk?
• Are most students involved in answering questions?
• Are students supporting each other’s learning?
• Is there evidence that various forms of teacher feedback advance student learning?
• Do students take responsibility for their own learning?
• Does the teacher provide oral formative feedback?
• Does the teacher find out what the students have learned before they leave the room?

4. Intervention

5. economic analysis of educational interventions (eaei), 5.1. cost-consequence analysis, 5.2. rationale for cca, 5.3. cost collecting methodology, 5.4. collating costs, 5.5. sensitivity analysis, 6.1. learner outcomes, 6.2. classroom observations, 6.3. the opinions of teachers and learners, 6.4. interviews with teachers.

“…helps me feel I get a greater understanding of my children. And I don’t go home at the end of the week thinking, I don’t think I’ve said five words to that child.” Teacher 6

“And sometimes it can be a little bit of idleness of picking up a pen but sometimes it’s their belief in themselves a lot of the time. And it is, it’s them thinking, “actually, I can do it”. “I think a lot of it is the confidence they have…” Teacher 4

“So, it’s easier. I think the quality of work is easier to mark…I do feel I’ve got extra time.” Teacher 2

“I would say predominantly it’s that the lower achievers it’s had the bigger impact on.” Teacher 5

6.5. Focus Group Interviews with Learners

“You kind of get to know them more, cos like…you just like…you don’t really play with them, cos you like different things, but if you’re discussion partners, you might have to try and get to know them…You might think better of them.” Learner, school L

“It makes the work a bit more straight forward, Cos when you look at the success criteria when you’re working, then it like gives you more to think about it and then more to think about the work.” Learner, school P

“…the teacher will take you out of the lessons and things just to like go over your piece of work and if you’ve done something well, he’ll tell you what you’ve done well and he’ll like highlight it on the success criteria, which is a list of things that you have to do and he’ll highlight it pink and then if you need to do something better, he’ll highlight it green and then he’ll tell you to re-do it and he’ll tell you what to re-do and stuff.” Learner, school O

The teachers discussed how FAIP supported them to help learners better understand the nature of successful outcomes and understand the expectations of quality standards in their work. Additionally, learners described how success criteria helped them complete tasks more successfully.

Teachers identified how using the strategies contained within the FAIP training helped them better understand where the learners were in their learning and provided them with useful, additional information to plan next steps. Teachers also indicated that they were more able to identify which learners needed support and adapt teaching in real time to provide next steps advice and support to learners.

Teachers discussed how feedback strategies supported them to better understand learner progress. Some teachers discussed being able to give immediate feedback to support learners to improve the outcomes they achieve.

Learners identified how the use of a range of self-assessment strategies impacted positively on the learning process and how it helped them engage with, and complete, tasks more successfully.

Teachers identified improved opportunities for learners to discuss their own work to enhance understanding and knowledge. Learners understood what a talk partner was, and how it helped them with their learning. It also enabled them to provide support for other learners. Learners also identified how it improved social relationships in school.

6.6. The Full Economic Cost of FAIP for Tier 2 Teachers

6.7. sensitivity analysis, 6.7.1. sensitivity analysis 1, 6.7.2. sensitivity analysis 2, 6.7.3. sensitivity analysis 3, 6.7.4. sensitivity analysis 4, 7. discussion, 8. limitations, 9. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

• Gorard, S.; See, B.H.; Siddiqui, N. What is the evidence on the best way to get evidence into use in education? Rev. Educ. 2020 , 8 , 570–610. [ Google Scholar ] [ CrossRef ]
• OECD. Value for Money in School Education: Smart Investments, Quality Outcomes, Equal Opportunities ; OECD Publishing: Paris, France, 2022. [ Google Scholar ] [ CrossRef ]
• Pegram, J.; Watkins, R.C.; Hoerger, M.; Hughes, J.C. Assessing the range and evidence-base of interventions in a cluster of schools. Rev. Educ. 2022 , 10 , e3336. [ Google Scholar ] [ CrossRef ]
• Slavin, R.E. How evidence-based reform will transform research and practice in education. Educ. Psychol. 2020 , 55 , 21–31. [ Google Scholar ] [ CrossRef ]
• Levin, H. Waiting for Godot: Cost-effectiveness analysis in education. New Dir. Eval. 2001 , 2001 , 55–68. [ Google Scholar ] [ CrossRef ]
• Owen, K.L.; Watkins, R.C.; Hughes, J.C. From evidence-informed to evidence-based: An evidence building framework for education. Rev. Educ. 2022 , 10 , e3342. [ Google Scholar ] [ CrossRef ]
• Hummel-Rossi, B.; Ashdown, J. The state of cost-benefit and cost-effectiveness analyses in education. Rev. Educ. Res. 2002 , 72 , 1–30. [ Google Scholar ] [ CrossRef ]
• Machin, S.; Mcnally, S.; Wyness, G. Educational Research Educational attainment across the UK nations: Performance, inequality and evidence Educational attainment across the UK nations: Performance, inequality and evidence. Educ. Res. 2013 , 55 , 139–164. [ Google Scholar ] [ CrossRef ]
• Kraft, M.A. Interpreting Effect Sizes of Education Interventions. Educ. Res. 2020 , 49 , 241–253. [ Google Scholar ] [ CrossRef ]
• EEF. Cost Evaluation Guidance for EEF Evaluations. 2019. Available online: https://d2tic4wvo1iusb.cloudfront.net/documents/evaluation/evaluation-design/Cost_Evaluation_Guidance_2019.12.11.pdf (accessed on 9 January 2024).
• CBCSE. Centre for Benefit Cost Studies in Education. 2015. Available online: https://www.cbcse.org/costout (accessed on 8 January 2024).
• Hinde, S.; Walker, S.M.; Lortie-Forgues, H. Applying the Three Core Concepts of Economic Evaluation in Health to Education in the UK ; Centre for Health Economics, University of York: York, UK, 2019. [ Google Scholar ]
• Quinn, B.; Van Mondfrans, A.; Worthen, B.R. Cost-Effectiveness of Two Math Programs as Moderated by Pupil SES. Educ. Eval. Policy Anal. 1984 , 6 , 39–52. [ Google Scholar ] [ CrossRef ]
• Hollands, F.; Bowden, A.B.; Belfield, C.; Levin, H.M.; Cheng, H.; Shand, R.; Pan, Y.; Hanisch-Cerda, B. Cost-Effectiveness Analysis in Practice: Interventions to Improve High School Completion. Educ. Eval. Policy Anal. 2014 , 36 , 307–326. [ Google Scholar ] [ CrossRef ]
• Buxton, M.J. Economic evaluation and decision making in the UK. Pharmacoeconomics 2006 , 24 , 1133–1142. [ Google Scholar ] [ CrossRef ]
• Corbacho, B.; Pinto-Prades, J.L. Health economic decision-making: A comparison between UK and Spain. Br. Med. Bull. 2012 , 103 , 5–20. [ Google Scholar ] [ CrossRef ]
• Black, P.; Wiliam, D. Assessment and classroom learning. Assess. Educ. Princ. Policy Pract. 1998 , 5 , 7–74. [ Google Scholar ] [ CrossRef ]
• Black, P.; Wiliam, D. Classroom assessment and pedagogy. Assess. Educ. Princ. Policy Pract. 2018 , 25 , 551–575. [ Google Scholar ] [ CrossRef ]
• Clarke, S. Outstanding Formative Assessment: Culture and Practice ; Hachette: London, UK, 2014. [ Google Scholar ]
• EEF. Teacher Feedback to Improve Pupil Learning. 2021. Available online: https://d2tic4wvo1iusb.cloudfront.net/production/eef-guidance-reports/feedback/Teacher_Feedback_to_Improve_Pupil_Learning.pdf?v=1713861365 (accessed on 10 January 2024).
• Bennett, R.E. Formative assessment: A critical review. Assess. Educ. Princ. Policy Pract. 2011 , 18 , 5–25. [ Google Scholar ] [ CrossRef ]
• Black, P.; Wiliam, D. Developing the theory of formative assessment. Educ. Assess., Eval. Account. (Former. J. Pers. Eval. Educ.) 2009 , 21 , 5–31. [ Google Scholar ] [ CrossRef ]
• OECD; CERI. Formative Assessment: Improving Learning in Secondary Classrooms ; CERI/OECD: Paris, France, 2005. [ Google Scholar ]
• Wiliam, D. Embedded Formative Assessment: Strategies for Classroom Assessment That Drives Student Engagement and Learning ; Solution Tree: Bloomington, IN, USA, 2017. [ Google Scholar ]
• James, M. Embedding formative assessment in classroom practice. In Life in Schools and Classrooms ; Springer: Singapore, 2017; pp. 509–525. [ Google Scholar ]
• Wiliam, D. Research into practice. In Getting Evidence into Education: Evaluating the Routes to Policy and Practice ; Routledge: London, UK, 2020. [ Google Scholar ]
• Donaldson, G. Successful Futures Independent Review of Curriculum and Assessment Arrangements in Wales Cardiff: Welsh Government. 2015. Available online: https://gov.wales/sites/default/files/publications/2018-03/successful-futures.pdf (accessed on 9 January 2024).
• Allal, L. Involving primary school students in the co-construction of formative assessment in support of writing. Assess. Educ. Princ. Policy Pract. 2021 , 28 , 584–601. [ Google Scholar ] [ CrossRef ]
• Anders, J.; Foliano, F.; Bursnall, M.; Dorsett, R.; Hudson, N.; Runge, J.; Speckesser, S. The effect of embedding formative assessment on pupil attainment. J. Res. Educ. Eff. 2022 , 15 , 748–779. [ Google Scholar ] [ CrossRef ]
• Cosi, A.; Voltas, N.; Lázaro-Cantabrana, J.L.; Morales, P.; Calvo, M.; Molina, S.; Quiroga, M.Á. Formative assessment at university through digital technology tools. Profr. Rev. Curric. Y Form. Profr. 2020 , 24 , 164–183. [ Google Scholar ] [ CrossRef ]
• Cisterna, D.; Gotwals, A.W. Enactment of ongoing formative assessment: Challenges and opportunities for professional development and practice. J. Sci. Teach. Educ. 2018 , 29 , 200–222. [ Google Scholar ] [ CrossRef ]
• Barana, A.; Marchisio Conte, M. Promoting socioeconomic equity through automatic formative assessment. J. Math. Educ. 2023 , 15 , 227–252. [ Google Scholar ] [ CrossRef ]
• Kingston, N.; Nash, B. Formative assessment: A meta-analysis and a call for research. Educ. Meas. Issues Pract. 2011 , 30 , 28–37. [ Google Scholar ] [ CrossRef ]
• James, M. Assessment for learning: Research and policy in the (dis) United Kingdom. In Assessment Reform in Education ; Springer: Dordrecht, The Netherlands, 2011; pp. 15–32. [ Google Scholar ]
• Leahy, S.; Wiliam, D. Embedding Formative Assessment: Practical Techniques for K-12 Classrooms ; Hawker Brownlow Education: Cheltenham, UK, 2015. [ Google Scholar ]
• Campbell, D.T.; Stanley, J.C. Experimental and Quasi-Experimental Designs for Research ; Ravenio Books: Charleston, SC, USA, 2015. [ Google Scholar ]
• Gray, E.; McCambridge, J.; Strang, J. The effectiveness of motivational interviewing delivered by youth workers in reducing drinking, cigarette and cannabis smoking among young people: Quasi-experimental pilot study. Alcohol Alcohol. 2005 , 40 , 535–539. [ Google Scholar ] [ CrossRef ]
• Hakim, C. Research Design: Successful Designs for Social Economics Research ; Routledge: London, UK, 2012. [ Google Scholar ]
• Handley, M.A.; Lyles, C.R.; McCulloch, C.; Cattamanchi, A. Selecting and improving quasi-experimental designs in effectiveness and implementation research. Annu. Rev. Public Health 2018 , 39 , 5–25. [ Google Scholar ] [ CrossRef ]
• Palinkas, L.A.; Horwitz, S.M.; Green, C.A.; Wisdom, J.P.; Duan, N.; Hoagwood, K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm. Policy Ment. Health Ment. Health Serv. Res. 2015 , 42 , 533–544. [ Google Scholar ] [ CrossRef ]
• Welsh Government. National Reading and Numeracy Personalised Assessments: Administration Handbook. 2019. Available online: https://hwb.gov.wales/curriculum-for-wales/reading-and-numeracy-assessments/personalised-assessments/national-reading-and-numeracy-personalised-assessments-administration-handbook#specific-requirements-for-numeracy-(procedural)-and-numeracy-(reasoning)-personalised-assessments (accessed on 8 February 2024).
• Dunphy, E. Assessing early learning through formative assessment: Key issues and considerations. Ir. Educ. Stud. 2010 , 29 , 41–56. [ Google Scholar ] [ CrossRef ]
• Stevens, K. Valuation of the child health utility 9D index. Pharmacoeconomics 2012 , 30 , 729–747. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Boyer, N.; Miller, S.; Connolly, P.; McIntosh, E. Population Health Economic Evaluation of the School-Based Roots of Empathy Program in Northern Ireland. In Proceedings of the 2018 Annual Research Meeting, Washington, DC, USA, 24–26 June 2018; AcademyHealth: Washington, DC, USA, 2018. [ Google Scholar ]
• White, J.; Connelly, G.; Thompson, L.; Wilson, P. Assessing children’s social and emotional wellbeing at school entry using the strengths and difficulties questionnaire: Professional perspectives. Educ. Res. 2013 , 55 , 87–98. [ Google Scholar ] [ CrossRef ]
• Weintraub, N.; Bar-Haim Erez, A. Quality of life in school (QoLS) questionnaire: Development and validity. Am. J. Occup. Ther. 2009 , 63 , 724–731. [ Google Scholar ] [ CrossRef ]
• Ghotra, S.; McIsaac, J.L.D.; Kirk, S.F.; Kuhle, S. Validation of the “Quality of Life in School” instrument in Canadian elementary school students. PeerJ 2016 , 4 , e1567. [ Google Scholar ] [ CrossRef ]
• Gorard, S.A.C. Rethinking ‘quantitative’methods and the development of new researchers. Rev. Educ. 2015 , 3 , 72–96. [ Google Scholar ] [ CrossRef ]
• Gorard, S.; Gorard, J. What to do instead of significance testing? Calculating the ‘number of counterfactual cases needed to disturb a finding’. Int. J. Soc. Res. Methodol. 2016 , 19 , 481–490. [ Google Scholar ] [ CrossRef ]
• Bell, A. Designing and testing questionnaires for children. J. Res. Nurs. 2007 , 12 , 461–469. [ Google Scholar ] [ CrossRef ]
• Welsh Government. Curriculum for Wales ; Welsh Government: Cardiff, UK, 2024. Available online: https://hwb.gov.wales/curriculum-for-wales/ (accessed on 12 February 2024).
• Morris, S.; Devlin, N.; Parkin, D.; Spencer, A. Principles of economic evaluation in health care. In Economic Analysis in Health Care , 2nd ed.; John Wiley & Sons: Chichester, UK, 2012; pp. 232–250. [ Google Scholar ]
• Charles, J.; Edwards, R.T. A Guide to Health Economics for Those Working in Public Health: A Concise Desktop Handbook ; Centre for Health Economics and Medicines Evaluation: Bangor, UK, 2016. [ Google Scholar ]
• Charles, J.M.; Harrington, D.M.; Davies, M.J.; Edwardson, C.L.; Gorely, T.; Bodicoat, D.H.; Khunti, K.; Sherar, L.B.; Yates, T.; Edwards, R.T. Micro-costing and a cost-consequence analysis of the ‘Girls Active’programme: A cluster randomised controlled trial. PLoS ONE 2019 , 14 , e0221276. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Haghparast-Bidgoli, H.; Skordis, J.; Harris-Fry, H.; Krishnan, S.; O’Hearn, M.; Kumar, A.; Pradhan, R.; Mishra, N.K.; Upadhyay, A.; Pradhan, S.; et al. Protocol for the cost-consequence and equity impact analyses of a cluster randomised controlled trial comparing three variants of a nutrition-sensitive agricultural extension intervention to improve maternal and child dietary diversity and nutritional status in rural Odisha, India (UPAVAN trial). Trials 2019 , 20 , 287. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hartfiel, N.; Edwards, R.T. Cost–consequence analysis of public health interventions. In Applied Health Economics for Public Health Practice and Research ; Oxford University Press: Oxford, UK, 2019; pp. 233–247. [ Google Scholar ] [ CrossRef ]
• Coast, J. Is economic evaluation in touch with society’s health values? BMJ 2004 , 329 , 1233–1236. [ Google Scholar ] [ CrossRef ]
• Rogers, J.C.; Simmons, E.A.; Convery, I.; Weatherall, A. Public perceptions of opportunities for community-based renewable energy projects. Energy Policy 2008 , 36 , 4217–4226. [ Google Scholar ] [ CrossRef ]
• Glouberman, S.; Zimmerman, B. Complicated and Complex Systems: What Would Successful Reform of Medicare Look Like? Commission on the Future of Health Care in Canada: Ottawa, ON, Canada, 2002. [ Google Scholar ]
• Chapko, M.K.; Liu, C.F.; Perkins, M.; Li, Y.F.; Fortney, J.C.; Maciejewski, M.L. Equivalence of two healthcare costing methods: Bottom-up and top-down. Health Econ. 2009 , 18 , 1188–1201. [ Google Scholar ] [ CrossRef ]
• Scammacca, N.; Swanson, E.; Vaughn, S.; Roberts, G. Cost-Effectiveness of a Grade 8 Intensive Reading and Content Learning Intervention. Sch. Psychol. Rev. 2020 , 49 , 374–385. [ Google Scholar ] [ CrossRef ]
• Harden, J. The True Cost of £ducation. 2019. Available online: https://www.ascl.org.uk/ASCL/media/ASCL/Our%20view/Campaigns/The-True-Cost-of-Education.pdf (accessed on 10 January 2024).
• NASUWT. Directed Time (Wales). 2024. Available online: https://www.nasuwt.org.uk/advice/conditions-of-service/teachers-working-hours/directed-time-wales.html#TeachersWorkingTime (accessed on 25 January 2024).
• Limwattananon, S. Sensitivity analysis for handling uncertainty in an economic evaluation. J. Med. Assoc. Thail. Chotmaihet Thangphaet 2014 , 97 , S59–S64. [ Google Scholar ]
• Vaismoradi, M.; Jones, J.; Turunen, H.; Snelgrove, S. Theme development in qualitative content analysis and thematic analysis. J. Nurs. Educ. Pract. 2016 , 6 , 100–110. [ Google Scholar ] [ CrossRef ]
• Braun, V.; Clarke, V. Thematic Analysis ; American Psychological Association: Washington, DC, USA, 2012. [ Google Scholar ]
• Welsh Government. Draft Curriculum for Wales 2022: A Guide to Curriculum for Wales 2022 ; Welsh Government: Cardiff, UK, 2019. Available online: https://cyfarthfahigh.merthyr.sch.uk/wp-content/uploads/2019/05/A-Guide-to-the-Curriculum-for-Wales-2022.pdf (accessed on 12 February 2024).
• Levin, H.; McEwan, P.; Belfield, C.; Bowden, A. Economic Evaluation in Education: Cost-Effectiveness and Benefit-Cost Analysis ; SAGE Publications: London, UK, 2017. [ Google Scholar ]
• Welsh Government. School Census Results ; Welsh Government: Cardiff, UK, 2018. Available online: https://www.gov.wales/sites/default/files/statistics-and-research/2018-12/180725-school-census-results-2018-en.pdf (accessed on 12 August 2024).
• Boström, E.; Palm, T. The effect of a formative assessment practice on student achievement in mathematics. Front. Educ. 2023 , 8 , 1101192. [ Google Scholar ] [ CrossRef ]
• Martin, C.; Mraz, M.; Polly, D. Examining elementary school teachers’ perceptions of and use of formative assessment in mathematics. Int. Electron. J. Elem. Educ. 2022 , 14 , 417–425. [ Google Scholar ] [ CrossRef ]
• Beames, J.R.; Spanos, S.; Roberts, A.; McGillivray, L.; Li, S.; Newby, J.M.; O’Dea, B.; Werner-Seidler, A. Intervention programs targeting the mental health, professional burnout, and/or wellbeing of school teachers: Systematic review and meta-analyses. Educ. Psychol. Rev. 2023 , 35 , 26. [ Google Scholar ] [ CrossRef ]
• Gorard, S.; Siddiqui, N.; See, B.H. What works and what fails? Evidence from seven popular literacy ‘catch-up’schemes for the transition to secondary school in England. Res. Pap. Educ. 2017 , 32 , 626–648. [ Google Scholar ] [ CrossRef ]
• Speckesser, S.; Runge, J.; Foliano, F.; Bursnall, M.; Hudson-Sharp, N.; Rolfe, H.; Anders, J. Embedding Formative Assessment: Evaluation Report and Executive Summary ; Education Endowment Foundation: London, UK, 2018. [ Google Scholar ]
• Newell, R.G.; Pizer, W.A.; Prest, B.C. The Shadow Price of Capital: Accounting for Capital Displacement in Benefit-Cost Analysis. Environ. Energy Policy Econ. 2024 , 5 , 49–69. [ Google Scholar ] [ CrossRef ]
• Curtis, L.A. Unit Costs of Health and Social Care 2013 ; Personal Social Services Research Unit: Kent, UK, 2013. [ Google Scholar ]
• Shand, R.; Bowden, A.B. Empirical support for establishing common assumptions in cost research in education. J. Res. Educ. Eff. 2022 , 15 , 103–129. [ Google Scholar ] [ CrossRef ]
• MacKillop, E.; Sheard, S. Quantifying life: Understanding the history of quality-adjusted life-years (QALYs). Soc. Sci. Med. 2018 , 211 , 359–366. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hartz, S.; John, J. Contribution of economic evaluation to decision making in early phases of product development: A methodological and empirical review. Int. J. Technol. Assess. Health Care 2008 , 24 , 465–472. [ Google Scholar ] [ CrossRef ]
• Ramsey, S.D.; Willke, R.J.; Glick, H.; Reed, S.D.; Augustovski, F.; Jonsson, B.; Briggs, A.; Sullivan, S.D. Cost-effectiveness analysis alongside clinical trials II—An ISPOR Good Research Practices Task Force report. Value Health 2015 , 18 , 161–172. [ Google Scholar ] [ CrossRef ]
• Levin, H.M.; Belfield, C. Guiding the development and use of cost-effectiveness analysis in education. J. Res. Educ. Eff. 2015 , 8 , 400–418. [ Google Scholar ] [ CrossRef ]
• McMillan, J.H.; Venable, J.C.; Varier, D. Studies of the effect of formative assessment on student achievement: So much more is needed. Pract. Assess. Res. Eval. 2013 , 18 , 1–15. [ Google Scholar ] [ CrossRef ]
• Gorard, S. A proposal for judging the trustworthiness of research findings. Radic. Stat. 2014 , 110 , 47–60. [ Google Scholar ]
• Gorard, S. Judging the relative trustworthiness of research results: How to do it and why it matters. Rev. Educ. 2024 , 12 , e3448. [ Google Scholar ] [ CrossRef ]
• Greenberg, M.T.; Abenavoli, R. Universal interventions: Fully exploring their impacts and potential to produce population-level impacts. J. Res. Educ. Eff. 2017 , 10 , 40–67. [ Google Scholar ] [ CrossRef ]
• Charles, J.M.; Edwards, R.T.; Bywater, T.; Hutchings, J. Micro-costing in public health economics: Steps towards a standardized framework, using the incredible years toddler parenting program as a worked example. Prev. Sci. 2013 , 14 , 377–389. [ Google Scholar ] [ CrossRef ]
• Dolan, P.; Peasgood, T. Estimating the economic and social costs of the fear of crime. Br. J. Criminol. 2007 , 47 , 121–132. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Tiesteel, E.; Watkins, R.C.; Stringer, C.; Grigorie, A.; Sultana, F.; Hughes, J.C. Where Are the Costs? Using an Economic Analysis of Educational Interventions Approach to Improve the Evaluation of a Regional School Improvement Programme. Educ. Sci. 2024 , 14 , 957. https://doi.org/10.3390/educsci14090957

Tiesteel E, Watkins RC, Stringer C, Grigorie A, Sultana F, Hughes JC. Where Are the Costs? Using an Economic Analysis of Educational Interventions Approach to Improve the Evaluation of a Regional School Improvement Programme. Education Sciences . 2024; 14(9):957. https://doi.org/10.3390/educsci14090957

Tiesteel, Emma, Richard C. Watkins, Carys Stringer, Adina Grigorie, Fatema Sultana, and J. Carl Hughes. 2024. "Where Are the Costs? Using an Economic Analysis of Educational Interventions Approach to Improve the Evaluation of a Regional School Improvement Programme" Education Sciences 14, no. 9: 957. https://doi.org/10.3390/educsci14090957

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Get the Reddit app

A place to share and discuss articles/issues related to all fields of psychology. Discussions should be of an academic nature, avoiding ‘pop psychology.’ This is also a place to talk about your own psychology research, methods, and career in order to gain input from our vast psychology community. This subreddit is generally aimed at those in an intermediate to master level, mostly in/around graduate school, or for professionals; undergraduates, etc., are recommended for r/psychologystudents.

Why is qualitative research looked down upon in academia?

From what I have understood, qualitative research actually requires more skill by the researcher. So why are qualitative studies not valued?

By continuing, you agree to our User Agreement and acknowledge that you understand the Privacy Policy .

Enter the 6-digit code from your authenticator app

You’ve set up two-factor authentication for this account.

Enter a 6-digit backup code

Create your username and password.

Reddit is anonymous, so your username is what you’ll go by here. Choose wisely—because once you get a name, you can’t change it.

Reset your password

Enter your email address or username and we’ll send you a link to reset your password

Check your inbox

An email with a link to reset your password was sent to the email address associated with your account

Choose a Reddit account to continue

We Trust in Human Precision

20,000+ Professional Language Experts Ready to Help. Expertise in a variety of Niches.

API Solutions

• API Pricing
• Cost estimate
• Customer loyalty program
• Educational Discount
• Non-Profit Discount
• Green Initiative Discount1

Value-Driven Pricing

Unmatched expertise at affordable rates tailored for your needs. Our services empower you to boost your productivity.

• Special Discounts
• Enterprise transcription solutions
• Enterprise translation solutions
• Transcription/Caption API
• AI Transcription Proofreading API

Trusted by Global Leaders

GoTranscript is the chosen service for top media organizations, universities, and Fortune 50 companies.

GoTranscript

One of the Largest Online Transcription and Translation Agencies in the World. Founded in 2005.

Speaker 1: Hey guys, welcome to Grad Coach TV, where we demystify and simplify the oftentimes confusing world of academic research. My name's David, and today I'm chatting to one of our trusted coaches, Alexandra, about five common mistakes students make during their qualitative research analysis. This discussion is based on one of the many, many articles over at the Grad Coach blog. So, if you'd like to learn more about qualitative research analysis, head over to gradcoach.com forward slash blog. Also, if you're looking for a helping hand with your dissertation, thesis or research project, be sure to check out our one-on-one private coaching service, where we hold your hand throughout the research journey, step-by-step. For more information and to book a free consultation, head over to gradcoach.com. Hey, Alexandra, welcome back to the CoachCast. It's really great to have you back on board.

Speaker 2: Hey, David. always a pleasure to be here and happy to talk with you today. So today we are talking about

Speaker 1: five common mistakes students make about qualitative research analysis, and let us just dive into it. The first one that comes up quite frequently is a lack of alignment between the analysis and the golden thread. Alexandra, what am I getting at with this? Yes, so this idea

Speaker 2: of the golden thread, you will hear it in all walks of research, whether it is quantitative, mixed methods and qualitative so really what you want to do and consider for this golden thread are these three fundamental we'll call them puzzle pieces of the research aims the research objectives and the research questions so these are kind of the foundation of your qualitative research study and so how you consider these and you know what you're trying to do and answer and how you're going to do it will then help you determine what methodology you should choose that would be the most appropriate or suitable to answer those questions and this is not particularly easy because there are several different kinds of qualitative methodologies out there but it can have some some positive outcomes or some negative consequences depending on which methodology you choose to answer those aims objectives and questions of your golden thread so that's

Speaker 1: really helpful alexandra maybe you can give us an example or two of where there's alignment or a

Speaker 2: lack of alignment sure so two of the most common methodologies in qualitative research that we see at grad coach or elsewhere are case studies versus grounded theory and so the first thing to keep in mind with any study is that your the methodology that you choose should be the most suitable one to answer those golden thread notions of the aims objectives research questions not the other way around and so for example with the case study the case study should be used if in your golden thread ideas of the aims objectives and research questions you already have some sort of working knowledge of a group or an event and so you're using this case study methodology because it will appropriately answer those foundational aspects of the golden thread on the other hand let's say your research aims or objectives or questions are about something that you really have limited knowledge about or there's scarce research out there and you're wanting to kind of build up a framework or a theory in that case using a methodology like grounded theory would be more suitable. So you can see there with those two examples of case study versus grounded theory, these two methodology should be applied to answer different golden thread foundational aspects.

Speaker 1: That is really helpful, Alexandra. And I know it can seem a little bit overwhelming to think about getting this alignment right. In cases like this, do not necessarily just rely on your own judgment. It can be really helpful to get a friend or someone from your cohort just to take a look through and read of what you are working on. They will be able to help you identify where there is a lack of alignment. For instance, if you ask them to sort of give you the elevator pitch back of what you set out to do, and it is not lining up with your thinking, then maybe it is a good point to sort of identify where those lack of alignments are, and use that to help you sort of address that. But try and do this earlier rather than later. It's definitely going to make your life easier. So our second mistake is making use of a transcription program software without checking the transcripts. Alexandra, why is this such a problem? Yes. So first of all, you know, there

Speaker 2: are programs, an increasing number out there that are cost effective, mostly free, and for the most part accurate things like zoom transcription software otter ai atlas ti and these certainly have a lot of benefits for convenience sake and cost effectiveness however um that's not to say that these programs are perfect because with a lot of ai and other kinds of automated software it does lose that human element that can miss some of the more nuanced or minute pieces of information that are important. So for instance, in my own dissertation research, I had about 100 participants who all verbally reacted to a stimulus. And half of my participants were doing this in English and the other half in French. And each of these were about 30 minutes long, each participant 30 minutes now with qualitative research you know you have to have something to analyze and it's difficult to do that directly from the audio files so what you have to do is transcribe these from audio to text and so i was going through and i was doing these manually myself from about participant 80 i was beyond exhausted and so i decided to use one of these outside services or programs to kind of expedite this, kind of help me. And of course it was convenient. However, when I got the transcripts back, I noticed as I was going through the first few of them, some errors to content, to spelling, different words were showing up where other words had been said actually in the audio files. And as I was going along through the rest of them, I noticed that pretty much all 20 or so of these outside transcribed files had errors. So I ended up having to go back myself regardless and going through them again and fixing them. So this is all to kind of say that even though these programs can be very convenient and cost effective, there are some drawbacks. most of that has to do with kind of content, the words that they miss, spelling, punctuation, grammar, et cetera, et cetera. And you'll oftentimes definitely actually still have to go in and check these for quality and accuracy. This is why it's very important to kind of think about, even though these programs might be convenient, they're never going to replace kind of that human element of being able to really read and understand what's going on, make sure that it matches what was said in the audio files. And so one of the things that you can do if it's not yourself, you should check it yourself, but even go beyond that and ask someone else to check these transcripts for accuracy. Because either if you've used an outside service or program, or if you've done all the transcriptions by yourself, sometimes we miss things. Having someone else, an outside person, an actual person look at these and kind of make sure that they're accurate will not only help you catch potential errors, but in doing so, it kind of promotes the credibility of the transcripts because they're accurate, they're clear, they're actually what was said in the audio files and so sometimes what might be happen if you don't do this having that like human element it can diminish the credibility of the rest of your transcripts if they are accurate because the reader or your marker might say well this one was not accurate so maybe there's some flaws in the other ones as well but beyond that I mean other than the marking your transcriptions this is your this is really your raw data in qualitative analysis and so if you have errors or missing information in your transcripts that were there in the audio files this makes the coding and analysis flawed this puts things in misalignment and as such there's kind of a domino effect of repercussions that can happen if these things aren't transcribed

Speaker 1: accurately. I think that in the same way that in quantitative research your actual data is key to your analysis, it is the same for qualitative. So we really want to make sure we are doing due diligence to assess the quality of the work. That is not to say you cannot use services to help out. It will depend on your type of research as well. For instance, from a business perspective, you might be less interested in the specific nuance of how someone presented an idea compared to a language study. So in cases like that, there is a bit of a cost benefit to consider, but regardless of whether you are using a service or not, getting a second run through of it can be super helpful. And there are a range of services out there that you can use, both in terms of software or human run services. If you are interested in it, we even do it here at Grad Coach. So do take a look for the link down below. So our third mistake that frequently comes up is not specifying what type of coding you are doing in advance of actually jumping into the analysis. Alexandra, why do we need to be aware of what coding type we are using so early in the process?

Speaker 2: This goes back to the idea of making sure that all steps of your research align with the previous one and are justifiable in terms of it makes sense. There's a reason why you're doing what you're doing in the order that you're doing it. And coding is no exception to this. So the reason why coding is so important in qualitative research is that qualitative research is inherently kind of subjective. There is this inherent human interpretation that can happen. And so one of the reasons why it is so important to do coding appropriately is to kind of add the systematicity and the academic rigor to your research that is inherently not there. And so to kind of ensure this increased objectivity of something that is inherently subjective, doing this coding, you need to consider which kind of coding will be the most appropriate to answer your research goals that you've outlined prior, going back to that notion of the golden thread. And coding inherently kind of falls into two camps. There is inductive coding and deductive coding. So on the one hand, inductive coding is an approach where you are going into your data analysis and you are kind of, you're letting the themes and the codes emerge from the data. You don't have any preconceived notions, no existing ideas of what to expect. You're really letting the data, whether it comes from interviews or focus groups, you're letting the data from those transcripts emerge into these codes. And this is best for studies such as grounded theory approaches where you don't really have any idea of what to expect or anticipate. And you're really kind of trying to explore what is out there. You're letting these codes emerge directly from the data. On the other hand, deductive coding is another coding approach where you are actually, you have some ideas about what is out there, what you're looking for, what you hope your final findings to be. And for this coding approach, it's top down where prior to even collect the data, the interviews, focus groups what have you you have developed an initial set of codes into a code book whether you've put this in say Microsoft Excel or Microsoft Word or Google Sheets etc and you have kind of looked through the existing literature on your research topic and seen what what are the potential codes out there what are the themes you're looking for And then once you have collected your data and transcribed it, you're assigning pieces of that data to those codes that you've already created in advance. And you are not looking for new codes to emerge like you did in inductive. So all codes should go into something from your codebook.

Speaker 1: I think deductive coding is most commonly used where you have a theoretical framework that you're working within or a field that is really, really well researched. There, you're not going to be starting something new. Similarly, it's also become really popular to use a mixed approach of inductive and deductive. This is primarily starting deductively with a codebook and using that codebook to lead your coding and then develop further from that with an inductive approach. It is worth noting this is a fairly new way to go about coding, and so it is important that if you are choosing to go this way, that you can justify why it is appropriate and why it is useful relative to that golden thread, those research aims, objectives, and questions. Because you You don't want to be overcomplicating things or stepping too far out of your comfort zone just because it's novel. Rather, make sure it is what you need to do, where you need to do it.

Speaker 2: That's great advice, because sometimes as graduate students, we have this urge to do something novel or do it a different way. And that should not be your motivation or your justification to do something. So even though this this kind of new way is developing and coming and becoming increasingly popular, that doesn't mean that it's right for your study. So how you know it's right for your study is going back to that notion of the golden thread. And this idea extends even beyond inductive and deductive coding, because those are kind of your your starting idea of how you're going to code. Beyond that, there are additional specific approaches that you will use for your initial or your first set of coding versus your second set of coding. As an aside here, you should absolutely do more than one round of coding. Again, this will increase the systematicity, the rigor, and kind of the credibility, so to speak, of your data analysis. and so there are many different specific coding approaches but some of the the most common ones we'll name here are starting with your open coding and so for this one this kind of approach it's very loose it's very tentative as indicated from its name it's open and so this is more suitable when you're starting out other common approaches are things like in vivo coding and so with in vivo coding, this is actually using the participants own words in your analysis, not putting your interpretation of what they said or suggesting what they meant, but actually letting the participants own words do the talking, so to speak. And so this is typically most suitable to things where you're really interested in the perspectives or points of view or experiences of your participants and then the last one we'll mention but there are still plenty more is structural coding and so we use structural coding specifically well not specifically but commonly in cases where you say have conducted an interview or focus group discussion and you want to use those questions that you posed in the interview or the focus group kind of as headings all of the codes that go under one specific column for instance should be related to one specific question that was asked in the data collection and so this is really best if you are kind of looking for specific answers or codes or themes in response to one of your interview questions so or focus group questions so again there are still plenty more out there but these are some of the more common coding approaches.

Speaker 1: That's really helpful, Alexandra. And it can feel a little overwhelming that there are so many options to choose from. Don't worry, there are a ton of resources out there. Definitely take a look at any of your methodological textbooks from a qualitative perspective. You can take a look at methodology papers that have been published, YouTube tutorials, blog posts, you name it, it's out there. We even have some videos and some content about coding as well on the Grad Coach blog. Links to that will be down in the description below. But importantly, when you are considering these coding decisions, it is important to realize again what you are using them for. So look for that alignment, make sure it is on track, and then it will flow much smoother going forward as well. So our fourth common mistake is students downplay the importance of organization during both coding and analysis. How important is organization, Alexandra? It is so important. The reason why

Speaker 2: this is so important is that oftentimes we kind of assume that qualitative research and qualitative data cannot be structured. Of course, it's not as black and white or objective as quantitative research. And so what you need to do as a qualitative research is to kind of apply a framework that yourself that will promote this kind of objectivity, systematicity. And part of this relies on organization. And organization is important not only for the coding, but also the analysis. So part of the difficulty, but the importance of organizing is that sometimes the codes that you end up with after you've transcribed and done your, let's say, initial round of coding, you can end up with very high numbers of codes. For instance, I've seen some where it's upwards of 1000 codes. And so this number is very overwhelming, very large. and some of the ways to tackle this large amount of codes is one to make sure that you're organizing all of your codes in a spreadsheet of sorts whether it's excel or google sheets having them all in one place will then further facilitate you doing additional rounds of coding which we recommended previously and in doing so having these additional rounds of coding on your codes that are organized in one place, it will help you kind of whittle down these codes to the point where you have the codes that you need. There's none that are kind of superfluous or repeated, but it's very important to keep these organized in one place and to go through multiple rounds of coding. And this will make your life a whole lot easier and make sure that you have only the

Speaker 1: codes that you need and can justify. I think that's super helpful. It's also worth emphasizing that coding and organization it's a back and forth you're going to be moving from one to the next and back again and that's a good thing to do it enriches your analysis but it also allows your organization to inform your coding and your coding to inform your organizational structure and through that iterative process you're really going to develop the analysis so don't think I've coded it once, I'm done and dusted. Sorry to say it's a multiple approach. In terms of organization helping analysis, Alexandra, why is it also important to keep a track in that Google document

Speaker 2: or sheet of all your codes? Yeah, so this goes back to that notion we've repeated several times of the golden thread. So if you think of dominoes, for instance, you need to have your dominoes set up in such a way that if you knock one down, the rest go down. We can think of that, our qualitative research in such a way. And so if in the coding stage, everything has aligned with that golden thread and we move on to the analysis, the analysis will be further aligned with the coding, the transcription, the data collection, going back to the research questions, aims and objectives. And so having our codes organized in a sheet will then allow us to start to analyze our codes in a way that we can see themes and patterns emerging that are aligned with the codes, which will then add this rigor and systematicity of your study by having analysis that you know is based on very organized, solid foundations of your coding and your transcription. And so through this analysis, if we have our analysis organized, we can keep track of our patterns, our themes, and then going beyond that, actually, when we get to the point where we're writing our findings chapter, we have this set organization that will then kind of allow us to know how we're going to present these results because everything has been organized and justified up to that point.

Speaker 1: I think that's really helpful. It's also worth noting that having your codebook organized can be really helpful in sort of preventing you from getting stuck with your analysis or feeling like you're unsure of how to code because, you know, things are feeling uncertain. If you have an Excel sheet that you've developed before you start your coding process, you have it organized by the different rounds and you start bringing it from a large number of codes to the specific codes you are going to be using, that organization really helps make that process move forward. And it can be kind of cathartic to really work through that process, get it from a hundred transcripts of 30 minutes each down to some key findings. So our fifth and final mistake that we're covering today is not considering your researcher influence on your analysis. Alexandra, how do we affect our analysis and why is this something that we need to even think about?

Speaker 2: Yeah, so this kind of just goes back to the innate nature of qualitative research. It relies a lot on interpretation. It is subjective. It's not inherently black and white, such as quantitative research. And so the ways that this is kind of mitigated is through things like positionality and reflexivity. So these two concepts are becoming much more prominent and required in qualitative dissertations and theses. And so what these essentially mean is that you have your positionality, which are the underlying kind of beliefs, judgments, opinions, perceptions, all of those things that kind of make you you, the human elements. And so the way that you think about things might be different than the way someone else thinks about them. And so why we need to state our positionality in qualitative research is that it can impact our interpretation of the data, which then impacts the findings. And so, for example, in an example study where someone is exploring the perceptions of the tech industry of men versus women, a researcher who kind of identifies as a feminist versus one who identifies as more conservative or traditional, they might have underlying beliefs or assumptions about gender when it comes to the workplace or just in general. and so acknowledging that that you have these kind of underlying preferences or perspectives what have you it's important to acknowledge that because like i said it can have consequences for your analysis and your findings taking this a step further typically now we also have to to talk about our reflexivity in qualitative research and so essentially what this refers to is how our positionality affects our kind of interpretation so whereas positionality has to do more with the underlying assumptions reflexivity is taking those underlying assumptions and acknowledging how they might actually impact our interpretation and our findings and so the reason oftentimes why these are required now in qualitative studies is that this idea of you know validity and reliability we don't really use those in qualitative research we use more of these ideas of trustworthiness and that connects to our positionality and our reflexivity this reflexivity how it can impact you know it can impact the coding of your data the themes that you pull from the coding how you interpret it how you present it so in my example of the researcher who has more feminist underlying beliefs versus more traditional conservatives even if they're exploring the same phenomenon they can have vastly different interpretations and so acknowledging your positionality and indicating with your reflexivity how it might impact those steps of the research analysis can lend more credibility and more kind of trustworthiness to your your

Speaker 1: findings and ultimately your study. So that's really helpful to think about these aspects because we do need to consider how our positionality and our reflexivity might affect how we proceed with our analysis. There are potential opportunities for bias and if we're engaging in these behaviors we are able to a mitigate them during the analysis and in cases where you cannot mitigate it you can at least acknowledge it so other researchers can interpret that going forward but bias goes a little bit beyond just your positionality and reflexivity so Alexandra what other biases can come up because of research effect yes this idea of bias so going

Speaker 2: further beyond positionality and reflexivity it can be very easy to have biased interpretations and there are a few ways this can manifest so for instance spending too much time presenting the the findings from one particular participant in your study and neglecting those of the others and so one reason why this might happen is either you as the researcher totally agree personally with their perspective or even totally disagree and you want to to present that in um in some for some sort of reason um so it's very important to kind of mitigate that bias by presenting a balanced approach of all participants on the other hand there's also things like spending a lot of time presenting on one particular theme that emerged from your qualitative analysis and you know, kind of avoiding or neglecting the other ones. So this can happen where you found a theme that emerged from your analysis that was particularly interesting to you, whether it was novel, whether it confirmed what you thought, or even aligned with your personal beliefs. It's very important to make sure that you are giving enough attention to all the different themes that have emerged. And a third common bias that we see is that sometimes it can be easy to make claims or assumptions such as this means that or people should do this. So for instance, in my example of the tech industry and gender norms, making claims in your writing such as women in the tech industry felt that, or the way that the women in the tech industry talked means that, or the tech industry should do that. So making those kinds of grand sweeping claims that your qualitative findings mean some sort of big, big thing. We really have to try to avoid that in qualitative writing, despite it being tempting, especially if it aligns with our personal perspective. So, those are some common biases we see.

Speaker 1: I think that is super helpful to think through, particularly because biases are inherent to us. So, it is important to take that step back, to think about how you might interpret, interact with things, and then engage with that. One way to really go back to this is take a look at the data. We do not want to be making statements or assumptions that do not have support in the data. that is just gonna undermine your argument and your position as the researcher. So wherever possible, if you don't have data to support it, maybe consider not including it. If you do have data to support it, maybe just confirm with a second opinion, your supervisor or someone else, just to make sure that there's not bias coming in. But I think the most important part here is to think about the fact that we do have biases. And so as long as we're considering this, we're doing our due diligence as researchers.

Speaker 2: Yeah, and so one of the ways that you can also make sure that you are kind of following what you said you were going to do from the get-go is not to step out of your codes and your themes that you've established. The reason why this might be tempting to do, again, is going back to that fact that maybe you found something super interesting to you and you want to present it. What I would caution you towards is making sure that any findings that you're presenting fit or align with what your objectives, aims, and research questions were. Another reason why this might happen is because the dissertation or the thesis is such a long process, sometimes we can kind of get away from our original intent of our study. And so presenting these things that are outside of our codes or our themes, we think we can get away with but in reality this kind of minimizes the the rigor of of your findings and so even though you might find something very interesting like you said David be really careful make sure that you're still kind of staying within your codes within your themes and following that golden thread that you've been establishing throughout yeah you've

Speaker 1: probably heard it so much today but golden thread is key we want to make sure that we're maintaining alignment with our research. It is only going to improve the impact. So Alexandra, thank you so much for joining us today. It has been really great. There are some great insights here and thank you again for joining us on the CoachCasts. Always a pleasure, David. Thanks so much for

Speaker 2: having me and letting me kind of chat about these qualitative foibles.

Speaker 1: Alright, so that pretty much wraps up this episode of Grad Coach TV. Remember, if you are looking for more information about qualitative research analysis, be sure to check out our blog at gradcoach.com forward slash blog. There you can also get access to our free dissertation and thesis writing mini course that'll give you all the information you need to get started with your research journey. Also, if you're looking for a helping hand with your dissertation, thesis or research project, be sure to check out our one-on-one private coaching service where you can work with one of our friendly coaches, just like Alexandra. For more information and to book a free consultation, head over to gradcoach.com.

• Open access
• Published: 26 August 2024

Paramedics’ experiences and observations: work-related emotions and well-being resources during the initial months of the COVID-19 pandemic—a qualitative study

• Henna Myrskykari 1 , 2 &
• Hilla Nordquist 3  

BMC Emergency Medicine volume  24 , Article number:  152 ( 2024 ) Cite this article

91 Accesses

1 Altmetric

Metrics details

As first responders, paramedics are an extremely important part of the care chain. COVID-19 significantly impacted their working circumstances. We examined, according to the experiences and observations of paramedics, (1) what kinds of emotions the Emergency Medical Service (EMS) personnel experienced in their new working circumstances, and (2) what work-related factors became resources for the well-being of EMS personnel during the initial months of the COVID-19 pandemic.

This qualitative study utilized reflective essay material written by experienced, advanced-level Finnish paramedics ( n  = 30). The essays used in this study were written during the fall of 2020 and reflected the period when Finland had declared a state of emergency (on 17.3.2020) and the Emergency Powers Act was implemented. The data was analyzed using an inductive thematic analysis.

The emotions experienced by the EMS personnel in their new working circumstances formed three themes: (1) New concerns arose that were constantly present; (2) Surviving without proper guidance; and (3) Rapidly approaching breaking point. Three themes were formed from work-related factors that were identified as resources for the well-being of the EMS personnel. These were: (1) A high level of organizational efficiency was achieved; (2) Adaptable EMS operations; and (3) Encouraging atmosphere.

Conclusions

Crisis management practices should be more attentive to personnel needs, ensuring that managerial and psychological support is readily available in crisis situations. Preparedness that ensures effective organizational adaptation also supports personnel well-being during sudden changes in working circumstances.

Peer Review reports

At the onset of the COVID-19 pandemic, healthcare personnel across the globe faced unprecedented challenges. As initial responders in emergency healthcare, paramedics were quickly placed at the front lines of the pandemic, dealing with a range of emergencies in unpredictable conditions [ 1 ]. The pandemic greatly changed the everyday nature of work [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 ]. Those working on the front line were suddenly forced to adjust to personal protective equipment (PPE) requirements [ 9 , 10 ] and rapidly changing instructions that caused significant adjustments to their job description [ 11 , 12 ]. For instance, it has been reported that during the initial stages of the COVID-19 pandemic, Emergency Medical Services (EMS) personnel, including paramedics working in prehospital emergency care, experienced a significant increase in stress [ 10 , 13 ] due to several reasons, such as the lack of protection and support, increased demands, lack of personnel, fear of exposure to COVID-19 during missions, concerns of spreading the virus to family members, and frustration over quickly changing work policies [ 11 , 14 , 15 ].

With the unprecedented challenges posed by the COVID-19 pandemic, some research has been directed toward identifying available resources that help in coping with such situations. For example, Sangal et al. [ 15 ] underscored the association between effective communication and reduced work stress and burnout, and emphasized the critical need for two-way communication, consistent messaging, and the strategic consolidation of information prior to its dissemination. In parallel, Dickson et al. [ 16 ] highlight the pivotal role of leadership strategies in fostering a healthful work environment. These strategies include being relationally engaging, visibly present, open, and caring for oneself and others, while embodying core values such as compassion, empathy, courage, and authenticity. Moreover, Awais et al. [ 14 ] identify essential measures to reduce mental distress and support EMS personnel’s overall well-being in pandemic conditions, such as by providing accessible mental health and peer support, ensuring a transparent information flow, and the implementation of clear, best-practice protocols and guidelines. As a lesson learned from COVID-19, Kihlström et al. (2022) add that crisis communication, flexible working conditions, compensation, and allowing for mistakes should be part of crisis management. They also emphasize the importance of psychological support for employees. [ 12 ]

Overall, the COVID-19 pandemic had a multifaceted impact on EMS personnel, highlighting the necessity for comprehensive support and resilience strategies to safeguard their well-being [ 11 , 17 , 18 ] alongside organizational functions [ 12 , 19 ]. For example, in Finland, it has been noted in the aftermath of COVID-19 that the availability and well-being of healthcare workers are key vulnerabilities of the resilience of the Finnish health system [ 12 ]. Effective preparedness planning and organizational resilience benefit from learning from past events and gaining a deeper understanding of observations across different organizational levels [ 12 , 19 , 20 ]. For these reasons, it is important to study how the personnel experienced the changing working circumstances and to recognize the resources, even unexpected ones, that supported their well-being during the initial phase of the COVID-19 pandemic [ 12 , 19 ].

The aim of this study was to examine the emotions experienced and the resources identified as supportive of work well-being during the initial months of the COVID-19 pandemic, from the perspective of the paramedics. Our research questions were: According to the experiences and observations of paramedics, (1) what kinds of emotions did the EMS personnel experience in the new working circumstances, and (2) what work-related factors became resources for the well-being of EMS personnel during the initial months of the COVID-19 pandemic? In this study, emotions are understood as complex responses involving psychological, physiological, and behavioral components, triggered by significant events or situations [ 21 ]. Resources are understood as physical, psychological, social, or organizational aspects of the work that help achieve work goals, reduce demands and associated costs [ 22 ].

Materials and methods

This qualitative study utilized reflective essay material written in the fall of 2020 by experienced, advanced-level paramedics who worked in the Finnish EMS during the early phase of the pandemic, when Finland had declared (March 17, 2020 onward) a state of emergency and implemented the Emergency Powers Act. This allowed for new rules and guidelines from the government to ensure the security of healthcare resources. Some work rules for healthcare personnel changed, and non-urgent services were limited.

Data collection procedures

This study is part of a broader, non-project-based research initiative investigating the work well-being of paramedics from various perspectives, and the data was collected for research purposes from this standpoint. The data collection for this study was conducted at the South-Eastern Finland University of Applied Sciences as part of the Current Issues in EMS Management course. The course participants were experienced, advanced-level Finnish paramedics who were students of the master’s degree program in Development and Management of Emergency Medical Services. A similar data collection method has been utilized in other qualitative studies [for example, 23 , 24 ].

The South-Eastern Finland University of Applied Sciences granted research permission for the data collection on August 20, 2020. The learning platform “Learn” (an adapted version of Moodle [ 25 ]) was used to gather the data. A research notice, privacy statement, and essay writing instructions were published on the platform on August 21, 2020. The paramedics were asked to write about their own experiences and observations regarding how the state of emergency impacted the work well-being of EMS personnel. They were instructed not to use references but only their own reflections. Three guiding questions were asked: “What kind of workloads did EMS personnel experience during the state of emergency?” “How has this workload differed from normal conditions?” and “What effects did this workload have on the well-being of the EMS personnel?”. The assignment did not refer solely to paramedics because the EMS field community may also include individuals with other titles (such as EMS field supervisors or firefighters performing prehospital emergency care); hence the term “EMS personnel” was used.

The essay was part of the mandatory course assignments, but submitting it for research purposes was voluntary. The paramedics were informed that their participation in the study would not affect their course evaluations. They had the freedom to decline, remove parts of, or withdraw the essay before analysis. None of the paramedics exercised these options. They were also informed that the last author removes any identifying details (such as names, places, and organizational descriptions that could reveal their workplace) before sharing the data with other, at the time unnamed, researchers. The last author (female) is a senior researcher specializing in EMS and work well-being topics, a principal lecturer of the respective course, and the head of the respective master’s program, and familiar to all of them through their studies. The paramedics were aware that the essays were graded by the last author on a pass/fail scale as part of the course assessment. However, comprehensive and well-reasoned reflections positively influenced the course grade. The evaluation was not part of this study. The paramedics had the opportunity to ask further questions about the study directly from the last author during and after the essay writing process and the course.

The paramedics wrote the essays between August 23, 2020, and November 30, 2020. Thirty-two paramedics (out of 39) returned their essays using the Learn platform during this timeframe. Thus, seven of the course completions were delayed, and the essays written later were no longer appropriate to include in the data due to the time elapsed since the initial months of the COVID-19 pandemic.

All 32 gave their informed consent for their essays to be included in the study. Essays written by paramedics who had not actively participated in EMS field work during exceptional circumstances were excluded from the material ( n  = 2), because they wrote the essay from a different perspective, as they could not reflect on their own experiences and observations. Thus, a total of 30 essays were included in the study. The total material was 106 pages long and comprised 32,621 words in Finnish.

Study participants

Thirty advanced-level paramedics from Finland participated in this study. They all had a bachelor’s degree in emergency care or nursing with additional emergency care specialization. At the time of the study, they were pursuing their master’s studies. Thirteen of them were women, and seventeen were men. The average age of the participants was 33.5 years among women and 35.9 years among men. Women had an average of 8.7 years of work experience, and men had 8.8 years. All the participating paramedics worked in EMS in different areas across Finland (except northern Finland) during their studies and the early phase of the pandemic.

Data analysis

The data was analyzed with a thematic analysis following the process detailed by Braun & Clarke [ 26 ]. First, the two researchers thoroughly familiarized themselves with the data, and the refined aim and research questions of the study were formulated inductively in collaboration based on the content of the data (see [ 26 ], page 84). After this, a thorough coding process was mainly carried out by the first author (female), who holds a master’s degree, is an advanced-level paramedic who worked in EMS during the pandemic, and at the time of the analysis was pursuing her doctoral studies in a different subject area related to EMS. Generating the initial codes involved making notes of interesting features of anything that stood out or seemed relevant to the research question systematically across the entire dataset. During this process, the original paragraphs and sentences were copied from the essay material into a table in Microsoft Word, with each research question in separate documents and each paragraph or sentence in its own row. The content of these data extracts was then coded in the adjacent column, carefully preserving the original content but in a more concise form. Then, the content was analyzed, and codes were combined to identify themes. After that, the authors reviewed the themes together by moving back and forth between the original material, the data in the Word documents, and the potential themes. During this process, the authors worked closely and refined the themes, allowing them to be separated and combined into new themes. For example, emotions depicting frustration and a shift to indifference formed their own theme in this kind of process. Finally, the themes were defined into main, major and minor themes and named. In the results, the main themes form the core in response to the research questions and include the most descriptions from the data. The major themes are significant but not as central as the main themes. Major themes provide additional depth and context to the results. One minor theme was formed as the analysis process progressed, and it provided valuable insights and details that deepened the response to the research question. All the coded data was utilized in the formed themes. The full content of the themes is reported in the Results section.

The emotions experienced by the EMS personnel in their new working circumstances formed three themes: New concerns arose that were constantly present (main theme); Surviving without proper guidance (major theme); and Rapidly approaching breaking point (major theme) (Fig.  1 ). Work-related factors identified as resources for the well-being of EMS personnel formed three themes: A high level of organizational efficiency was achieved (main theme); Adaptable EMS operations (major theme); and Encouraging atmosphere (minor theme) (Fig.  2 ).

Emotions experienced by the EMS personnel in their new working circumstances

Main theme: New concerns arose that were constantly present

The main theme included several kinds of new concerns. In the beginning, the uncertainty about the virus raised concerns about work safety and the means to prevent the spread of the disease. The initial lack of training and routines led to uncertainty. In addition, the decrease in the number of EMS missions raised fears of units being reduced and unilateral decisions by the management to change the EMS personnel’s work responsibilities. The future was also a source of uncertainty in the early stages. For example, the transition to exceptional circumstances, concerns about management and the supervisors’ familiarity with national guidelines and lack of information related to sickness absence procedures, leave, personal career progression, and even the progress of vaccine development, all contributed to this feeling of uncertainty. The initial uncertainty was described as the most challenging phase, but the uncertainty was also described as long-lasting.

Being on the front line with an unknown, potentially dangerous, and easily transmissible virus caused daily concerns about the personnel’s own health, especially when some patients hid their symptoms. The thought of working without proper PPE was frightening. On the other hand, waiting for a patient’s test result was stressful, as it often resulted in many colleagues being quarantined. A constant concern for the health of loved ones and the fear of contracting the virus and unknowingly bringing it home or transmitting it to colleagues led the EMS personnel to change their behavior by limiting contact.

Being part of a high-risk group , I often wondered , in the case of coronavirus , who would protect me and other paramedics from human vanity and selfishness [of those refusing to follow the public health guidelines]? (Participant 25)

The EMS personnel felt a weight of responsibility to act correctly, especially from the perspective of keeping their skills up to date. The proper selection of PPE and aseptic procedures were significant sources of concern, as making mistakes was feared to lead to quarantine and increase their colleagues’ workloads. At the same time, concerns about the adequacy of PPE weighed on the personnel, and they felt pressure on this matter to avoid wastage of PPEs. The variability in the quality of PPE also caused concerns.

Concerns about acting correctly were also tied to ethical considerations and feelings of inadequacy when the personnel were unable to explain to patients why COVID-19 caused restrictions on healthcare services. The presence of students also provoked such ethical concerns. Recognizing patients’ symptoms correctly also felt distressing due to the immense responsibility. This concern was also closely tied to fear and even made some question their career choices. The EMS personnel were also worried about adequate treatment for the patients and sometimes felt that the patients were left alone at home to cope. A reduction in patient numbers in the early stages of the pandemic raised concerns about whether acutely ill individuals were seeking help. At the same time, the time taken to put on PPE stressed the personnel because it increased delays in providing care. In the early phase of the pandemic, the EMS personnel were stressed that patients were not protected from them.

I’m vexed in the workplace. I felt it was immediately necessary to protect patients from us paramedics as well. It wasn’t specifically called for , mostly it felt like everyone had a strong need to protect themselves. (Participant 30)

All these concerns caused a particularly heavy psychological burden on some personnel. They described feeling more fatigued and irritable than usual. They had to familiarize themselves with new guidelines even during their free time, which was exhausting. The situation felt unjust, and there was a looming fear of the entire healthcare system collapsing. COVID-19 was omnipresent. Even at the base station of the EMS services, movement was restricted and social distancing was mandated. Such segregation, even within the professional community, added to the strain and reduced opportunities for peer support. The EMS personnel felt isolated, and thoughts about changing professions increased.

It was inevitable that the segregation of the work community would affect the community spirit , and a less able work community has a significant impact on the individual level. (Participant 8)

Major theme: Surviving without proper guidance

At the onset of the pandemic, the job description of the EMS personnel underwent changes, and employers could suddenly relocate them to other work. There was not always adequate support for familiarizing oneself with the new roles, leading to a feeling of loss of control. The management was described as commanding and restricting the personnel’s actions. As opportunities to influence one’s work diminished, the sense of job satisfaction and motivation decreased.

Some felt that leadership was inadequate and neglectful, especially when the leaders switched to remote work. The management did not take the situation seriously enough, leaving the EMS personnel feeling abandoned. The lack of consistent leadership and failure to listen to the personnel caused dissatisfaction and reduced occupational endurance. In addition, the reduced contact with colleagues and close ones reduced the amount of peer support. The existing models for psychological support were found to be inadequate.

Particularly in the early stages, guidelines were seen as ambiguous and deficient, causing frustration, irritation, and fear. The guidelines also changed constantly, even daily, and it was felt that the information did not flow properly from the management to the personnel. Changes in protection recommendations also led to skepticism about the correctness of the national guidance, and the lack of consistent guidelines perplexed the personnel. Internalizing the guidelines was not supported adequately, but the necessity to grasp new information was described as immense and cognitively demanding.

At times , it felt like the work was a kind of survival in a jungle of changing instructions , one mission at a time. (Participant 11)

Major theme: Rapidly approaching breaking point

Risking one’s own health at work caused contentious feelings while concurrently feeling angry that management could work remotely. The arrogant behavior of people toward COVID-19 left them frustrated, while the EMS personnel had to limit their contacts and lost their annual leave. There were fears about forced labor.

Incomplete and constantly changing guidelines caused irritation and indifference, as the same tasks had to be performed with different levels of PPE within a short time. Some guidelines were difficult to comply with in practice, which was vexing.

Using a protective mask was described as distressing, especially on long and demanding missions. Communication and operation became more difficult. Some described frustration with cleaning PPE meant for single use.

Ensuring the proper implementation of a work pair’s aseptic and equipment maintenance was burdensome, and explaining and repeating guidelines was exhausting. A feeling of indifference was emphasized toward the end of a long shift.

After the initial stage, many began to slip with the PPE guidelines and found the instructions excessive. COVID-19 information transmitted by the emergency center lost its meaning, and instructions were left unheeded, as there was no energy to believe that the patient would have COVID-19, especially if only a few disease cases had been reported in their area.

It was disheartening to hear personnel being labeled as selfish for demanding higher pay during exceptional circumstances. This lack of recognition eroded professionalism and increased thoughts of changing professions.

However , being a doormat and a human toilet , as well as a lack of appreciation , undermines my professionalism and the prolonged situation has led me to seriously consider a different job , where values other than dedication and constant flexibility carry weight. I have heard similar thoughts from other colleagues. None of us do this for money. (Participant 9)

Work-related factors identified as resources for the well-being of EMS personnel

Main theme: A high level of organizational efficiency was achieved

The main theme held several different efficient functions. In the early stages of the pandemic, some felt that the information flow was active. Organizations informed the EMS personnel about the disease, its spread, and its impact on the workplace and emergency care activities.

Some felt that managers were easily accessible during the pandemic, at least remotely. Some managers worked long days to be able to support their personnel.

The response to hate and uncertainty was that one of the supervisors was always present in the morning and evening meetings. Supervisors worked long hours so as to be accessible via remote access. (Participant 26)

The organizations took effective steps to control infections. Quick access to COVID-19 tests, clear guidelines for taking sick leave, and permission to take sick leave with a low threshold were seen as positive things. The consideration of personnel belonging to risk groups by moving them to other work tasks was also perceived as positive. In addition, efforts were made to prevent the emergence of infection chains by isolating EMS personnel in their own social facilities.

Established guidelines, especially on the correct use of protective measures, made it easier to work. Some mentioned that the guidelines were available in ambulances and on phones, allowing the protection guidelines to be checked before going on a mission.

The employers took into account the need for psychological support in a diverse manner. Some organizations provided psychological support such as peer debriefing activities, talking therapy with mental health professionals, actively inquiring about their personnel’s feelings, and training them as support workers. The pandemic situation also caused organizations to create their own standard operating models to decrease mental load.

Fortunately , the problem has now been addressed actively , as a peer-to-peer defusing model was built up at our workplace during the crisis , and group defusing has started , the purpose of which is to lighten the work-related mental load. (Participant 3)

Major theme: Adaptable EMS operations

There were several different resources that clarified mission activities. The amount of protective and cleaning equipment was ramped up, and the treatment equipment was quickly updated to meet the demands brought about by the pandemic and to enable safety distances for the EMS personnel. In addition, various guidelines were amended to reduce exposure. For example, personnel on the dedicated COVID-19 ambulances were separated to work without physical contact with others, and field supervisors joined the EMS missions less often than before. Moreover, people at the scene were contacted by phone in advance to ensure that there would be no exposure risk, which also allowed other occupational safety risks to be identified. New practices resulted from the pandemic, such as cleaning communication equipment during shift changes and regularly using PPE with infected patients. All of these were seen as positive resources for efficient work.

At the end of each shift , all keys , telephones , etc., were cleaned and handed over to the next shift. This practice was not previously established in our area , but this will become a permanent practice in the future and is perceived by everyone in our work community as a positive thing. (Participant 10)

Some stated that access to PPE was sufficient, especially in areas where the number of COVID-19 infections was low. PPE was upgraded to make it easier to wear. Further, organizations acquired a variety of cleaning equipment to speed up the disinfection of ambulances.

Organizations hired more employees to enable leave and the operation of dedicated COVID-19 ambulances. The overall number of ambulances was also increased. Non-urgent missions were handled through enhanced phone services, reducing the unnecessary exposure of EMS personnel to COVID-19.

Five extra holiday substitutes were hired for EMS so that the employer could guarantee the success of agreed leave , even if the Emergency Preparedness Act had given them opportunities to cancel or postpone it. (Participant 12)

Minor theme: Encouraging atmosphere

Peer support from colleagues, a positive, comfortable, pleasant work environment, and open discussion, as well as smooth cooperation with other healthcare employees were felt to be resources for work well-being by reducing the heavy workload experienced. Due to the pandemic, the appreciation of healthcare was felt to increase slightly, which was identified as a resource.

One factor affecting resilience in the healthcare sector is certainly that in exceptional circumstances , visibility and appreciation have somewhat increased. (Participant 23)

This study examined, according to the experiences and observations of paramedics, (1) what kinds of emotions the Emergency Medical Service (EMS) personnel experienced in their new working circumstances, and (2) what work-related factors became resources for the well-being of EMS personnel during the initial months of the COVID-19 pandemic. Each research question was answered with three themes.

Previous studies have shown that the pandemic increased the workload of paramedics, prompting changes in their operating models and the function of EMS to align with new pandemic-related requirements [ 9 , 27 ]. Initially, the paramedics in the current study described facing unclear and deficient guidelines and feeling obligated to follow instructions without adequate support to internalize them. Constantly changing instructions were linked to negative emotions in various ways. Moreover, the overwhelming flood of information was heavily connected to this, although the information flow was also perceived as a resource, especially when it was timely and well-structured. The study by Sangal et al. [ 15 ] has raised similar observations and points out the importance of paying special attention to the personnel working in the frontline, as in EMS, who might be more heavily impacted by too much information and anxiety about it. They also discovered that three factors are crucial for addressing the challenges of information overload and anxiety: consolidating information before distributing it, maintaining consistent communication, and ensuring communication is two-way. McAlearney et al. [ 11 ] found that first responders, including EMS personnel, reported frustration regarding COVID-19 information because of inconsistencies between sources, misinformation on social media, and the impact of politics. A Finnish study also recognized that health systems were not sufficiently prepared for the flood of information in the current media environment [ 12 ]. Based on these previous results and our findings, it can be concluded that proper implementation of crisis communication should be an integral part of organizations’ preparedness in the future, ensuring that communication effectively supports employee actions in real-life situations. Secondly, this topic highlights the need for precise guidelines and their implementation. With better preparedness, similar chaos could be avoided in the future [ 17 ].

Many other factors also caused changes in work. The EMS mission profile changed [ 3 , 4 , 5 , 6 ], where paramedics in this study saw concerns. To prevent infection risk, the number of pre-arrival calls increased [ 7 ], the duration of EMS missions increased [ 8 , 9 ], and the continuous use of PPE and enhanced hygiene standards imposed additional burdens [ 9 , 10 ]. In Finland, there was no preparedness for the levels of PPE usage required in the early stages of the pandemic [ 12 ]. In this study, paramedics described that working with potentially inadequate PPE caused fear and frustration, which was increased by a lack of training, causing them to feel a great deal of responsibility for acting aseptically and caring for patients correctly. Conversely, providing adequate PPE, information and training has been found to increase the willingness to work [ 28 ] and the sense of safety in working in a pandemic situation [ 29 ], meaning that the role of precise training, operating instructions and leadership in the use of PPE is emphasized [ 30 ].

The paramedics in this study described many additional new concerns in their work, affecting their lives comprehensively. It has been similarly described that the pandemic adversely affected the overall well-being of healthcare personnel [ 31 ]. The restrictions implemented also impacted their leisure time [ 32 ], and the virus caused concerns for their own and their families’ health [ 11 , 28 ]. In line with this, the pandemic increased stress, burnout [ 10 , 33 ], and anxiety among EMS personnel and other healthcare personnel working on the frontline [ 11 , 14 , 34 , 35 ]. These kinds of results underscore the need for adequate guidance and support, a lack of which paramedics reported experiencing in the current study.

Personnel play a crucial role in the efficient operation of an organization and comprise the main identified resource in this study. Previous studies and summaries have highlighted that EMS personnel did not receive sufficient support during the COVID-19 pandemic [ 11 , 14 , 17 , 18 ]. Research has also brought to light elements of adequate support related to the pandemic, such as a review by Dickson et al. [ 16 ] that presents six tentative theories for healthful leadership, all of which are intertwined with genuine encounter, preparedness, and information use. In this current study, the results showed numerous factors related to these contexts that were identified as resources, specifically underlined by elements of caring, effective operational change, knowledge-based actions, and present leadership, similarly described in a study by Eaton-Williams & Williams [ 18 ]. Moreover, the paramedics in our study highlighted the importance of encouragement and identified peer support from colleagues as a resource, which is in line with studies in the UK and Finland [ 12 , 23 , 37 ].

In the early stages of the pandemic, it was noted that the EMS personnel lacked adequate training to manage their mental health, and there was a significant shortage of psychosocial support measures [ 14 ], although easy access to support would have been significant [ 18 ]. In the current study, some paramedics felt that mental health support was inadequate and delayed, while others observed an increase in mental health support during the pandemic, seeing it as an incentive for organizations to develop standard operating models for mental support, for example. This awakening was identified as a resource. This is consistent, as providing psychological support to personnel has been highlighted as a core aspect of crisis management in a Finnish study assessing health system resilience related to COVID-19 [ 12 ]. In a comprehensive recommendation commentary, Isakov et al. [ 17 ] suggest developing a national strategy to improve resilience by addressing the mental health consequences of COVID-19 and other occupational stressors for EMS personnel. This concept, applicable beyond the US, supports the view that EMS organizations are becoming increasingly aware of the need to prepare for and invest in this area.

A fundamental factor likely underlying all the described emotions was that changes in the job descriptions of the EMS personnel due to the pandemic were significant and, in part, mandated from above. In this study, paramedics described feelings of concern and frustration related to these many changes and uncertainties. According to Zamoum and Gorpe (2018), efficient crisis management emphasizes the importance of respecting emotions, recognizing rights, and making appropriate decisions. Restoring trust is a significant challenge in a crisis situation, one that cannot be resolved without complete transparency and open communication [ 38 ]. This perspective is crucial to consider in planning for future preparedness. Overall, the perspective of employee rights and obligations in exceptional circumstances has been relatively under-researched, but in Australia, grounding research on this perspective has been conducted with paramedics using various approaches [ 39 , 40 , 41 ]. The researchers conclude that there is a lack of clarity about the concept of professional obligation, specifically regarding its boundaries, and the issue urgently needs to be addressed by developing clear guidelines that outline the obligation to respond, both in normal day-to-day operations and during exceptional circumstances [ 39 ].

Complex adaptive systems (CAS) theory recognizes that in a resilient organization, different levels adapt to changing environments [ 19 , 20 ]. Barasa et al. (2018) note that planned resilience and adaptive resilience are both important [ 19 ]. Kihlström et al. (2022) note that the health system’s resilience was strengthened by a certain expectation of crisis, and they also recognized further study needs on how effectively management is responding to weak signals [ 12 ]. This could be directly related to how personnel can prepare for future changes. The results of this study revealed many negative emotions related to sudden changes, but at the same time, effective organizational adaptation was identified as a resource for the well-being of EMS personnel. Dissecting different elements of system adaptation in a crisis has been recognized as a highly necessary area for further research [ 20 ]. Kihlström et al. (2022) emphasize the importance of ensuring a healthy workforce across the entire health system. These frameworks suggest numerous potential areas for future research, which would also enhance effective preparedness [ 12 ].

Limitations of the study

In this study, we utilized essay material written in the fall of 2020, in which experienced paramedics reflected on the early stages of the COVID-19 pandemic from a work-oriented perspective. The essays were approached inductively, meaning that they were not directly written to answer our research questions, but the aim and the research questions were shaped based on the content [ 26 ]. The essays included extensive descriptions that aligned well with the aim of this study. However, it is important to remember when interpreting the results that asking specifically about this topic, for instance, in an interview, might have yielded different descriptions. It can be assessed that the study achieved a tentative descriptive level, as the detailed examination of complex phenomena such as emotions and resources would require various methods and observations.

Although the essays were mostly profound, well-thought-out, and clearly written, their credibility [ 42 ] may be affected by the fact that several months had passed between the time the essays were written and the events described. Memories may have altered, potentially influencing the content of the writings. Diary-like material from the very onset of the pandemic might have yielded more precise data, and such a data collection method could be considered in future research on exceptional circumstances.

The credibility [ 42 ] could also have been enhanced if the paramedics who wrote the essays had commented on the results and provided additional perspectives on the material and analysis through a multi-phase data collection process. This was not deemed feasible in this study, mainly because there was a 2.5-year gap between data collection and the start of the analysis. However, this also strengthened the overall trustworthiness of the study, as it allowed the first author, who had worked in prehospital emergency care during the initial phase of the pandemic, to maintain a distance from the subject, and enabled a comparison of our own findings with previously published research that investigated the same period in different contexts. The comparison was made when writing the discussion, with the analysis itself being inductive and following the thematic analysis process described by Braun & Clarke [ 26 ].

When evaluating credibility [ 42 ], it should also be noted that the participants who wrote the essays, i.e., the data for the study, were experienced paramedics but also students and one of the researchers was their principal lecturer. This could potentially limit credibility if the students, for some reason, did not want to produce truthful content for their lecturer to read. However, this risk can be considered small because the essays’ topics did not concern the students’ academic progress, the essays’ content was quite consistent, and the results aligned with other studies. As a strength, it can be considered that the students shared their experiences without holding back, as the thoughts were not for workplace use, and they could trust the data privacy statement.

To enhance transferability [ 42 ], the context of the study was described in detail, highlighting the conditions prevailing in Finnish prehospital emergency care during the early stages of the pandemic. Moreover, including a diverse range of perspectives from paramedics working in different regions of Finland (except Northern Finland) contributes to the transferability of the study, indicating that the results may be applicable and relevant to a wider context beyond a single specific region.

Dependability [ 42 ] was reinforced by the close involvement of two researchers from different backgrounds in the analysis of the material, but a limitation is that no separate analyses were conducted. However, the original data was repeatedly revisited during the analysis, which strengthened the dependability. Moreover, the first author kept detailed notes throughout the analysis process, and the last author supervised the progress while also contributing to the analysis and reporting. The research process is also reported in detail.

This study highlighted numerous, mainly negative emotions experienced by EMS personnel during the initial months of the COVID-19 pandemic due to new working circumstances. At the same time, several work-related factors were identified as resources for their well-being. The findings suggest that crisis management practices should be more attentive to personnel needs, ensuring that personnel have the necessary support, both managerial and psychological, readily available in crisis situations. Effective organizational adaptation in a crisis situation also supports personnel well-being, emphasizing the importance of effective preparedness. Future research should particularly focus on considering personnel well-being as part of organizational adaptation during exceptional circumstances and utilize these findings to enhance preparedness.

Data availability

The datasets generated and analyzed during the current study are not publicly available due to the inclusion of sensitive information and the extent of the informed consent provided by the participants.

Abbreviations

Complex Adaptive Systems (theory)

Coronavirus Disease 2019

Emergency Medical Services

Personal Protective Equipment

United Kingdom

Drennan IR, Blanchard IE, Buick JE. Opportunity for change: is it time to redefine the role of paramedics in healthcare? CJEM. 2021;23(2):139–40. https://doi.org/10.1007/s43678-021-00105-y

Boechler L, Cameron C, Smith JC, Ford-Jones P, Suthers P. Impactful approaches to Leadership on the Front lines of the COVID-19 pandemic: lived experiences of Canadian paramedics. Healthc Q. 2021;24(3):42–7. https://doi.org/10.12927/hcq.2021.26620 .

Article   PubMed   Google Scholar  

Lerner EB, Newgard CD, Mann NC. Effect of the Coronavirus Disease 2019 (COVID-19) pandemic on the U.S. Emergency Medical Services System: a preliminary Report. Acad Emerg Med. 2020;27(8):693–9. https://doi.org/10.1111/acem.14051 .

Article   PubMed   PubMed Central   Google Scholar  

O’Connor AW, Hannah HA, Burnor EA, Fukutaki KG, Peterson T, Ballard DW, et al. Emergency Medical Service Utilization and response following COVID-19 emergency and stay-at-home policies: an interrupted time-series analysis. Cureus. 2021;21(11). https://doi.org/10.7759/cureus.19794 .

Azbel M, Heinänen M, Lääperi M, Kuisma M. Effects of the COVID-19 pandemic on trauma-related emergency medical service calls: a retrospective cohort study. BMC Emerg Med. 2021;9(1):102. https://doi.org/10.1186/s12873-021-00495-3 .

Article   CAS   Google Scholar  

Lane DJ, Blanchard IE, Buick JE, Shaw M, McRae AD. Changes in presentation, presenting severity and disposition among patients accessing emergency services during the first months of the COVID-19 pandemic in Calgary, Alberta: a descriptive study. CMAJ Open. 2021;9(2):592–601. https://doi.org/10.9778/cmajo.20200313 .

Article   Google Scholar  

Shukla V, Lau CSM, Towns M, Mayer J, Kalkbrenner K, Beuerlein S, et al. COVID-19 exposure among First Responders in Arizona. J Occup Environ Med. 2020;62(12):981–5. https://doi.org/10.1097/JOM.0000000000002027 .

Article   CAS   PubMed   Google Scholar  

Andrew E, Nehme Z, Stephenson M, Walker T, Smith K. The impact of the COVID-19 pandemic on demand for emergency ambulances in Victoria, Australia. Prehosp Emerg Care. 2021;16:1–7. https://doi.org/10.1080/10903127.2021.1944409 .

Eskol JR, Zegers FD, Wittrock D, Lassen AT, Mikkelsen S. Increased ambulance on-scene times but unaffected response times during the first wave of the COVID-19 pandemic in Southern Denmark. BMC Emerg Med. 2022;9(1):61. https://doi.org/10.1186/s12873-022-00623-7 .

Schumann H, Böckelmann I, Thielmann B. Relaxation and strain among emergency medical service personnel and emergency control center dispatchers during the first two waves of the SARS-CoV-2 pandemic. Med Pr. 2023;15(5):353–62. https://doi.org/10.13075/mp.5893.01401 .

McAlearney AS, Gaughan AA, MacEwan SR, Gregory ME, Rush LJ, Volney J, et al. Pandemic experience of first responders: fear, frustration, and stress. Int J Environ Res Public Health. 2022;13(8):4693. https://doi.org/10.3390/ijerph19084693 .

Kihlström L, Huhtakangas M, Karreinen S, Viita-Aho M, Keskimäki I, Tynkkynen LK. Local cooperation has been the cornerstone: facilitators and barriers to resilience in a decentralized health system during COVID-19 in Finland. J Health Organ Manag. 2022. https://doi.org/10.1108/JHOM-02-2022-0069 .

Hendrickson RC, Slevin RA, Hoerster KD, Chang BP, Sano E, McCall CA, et al. The impact of the COVID-19 pandemic on Mental Health, Occupational Functioning, and Professional Retention among Health Care workers and First Responders. J Gen Intern Med. 2022;37(2):397–408. https://doi.org/10.1007/s11606-021-07252-z .

Awais SB, Martins RS, Khan MS. Paramedics in pandemics: protecting the mental wellness of those behind enemy lines. Br J Psychiatry. 2021;218(2):75–6. https://doi.org/10.1192/bjp.2020.193 .

Sangal RB, Bray A, Reid E, Ulrich A, Liebhardt B, Venkatesh AK, et al. Leadership communication, stress, and burnout among frontline emergency department staff amid the COVID-19 pandemic: a mixed methods approach. Healthc (Amst). 2021;9(4):100577. https://doi.org/10.1016/j.hjdsi.2021.100577 .

Dickson CAW, Davies C, McCormack B, Westcott L, Merrell J, Mcilfatrick S, et al. UK nurses’ and midwives’ experiences of healthful leadership practices during the COVID-19 pandemic: a rapid realist review. J Nurs Manag. 2022;30(8):3942–57. https://doi.org/10.1111/jonm.13790 .

Isakov A, Carr M, Munjal KG, Kumar L, Gausche-Hill MEMS. Agenda 2050 meets the COVID-19 pandemic. Health Secur. 2022;20(S1):S97–106. https://doi.org/10.1089/hs.2021.0179 .

Eaton-Williams PJ, Williams J. See us as humans. Speak to us with respect. Listen to us. A qualitative study on UK ambulance staff requirements of leadership while working during the COVID-19 pandemic. BMJ Lead. 2023;7(2):102–7. https://doi.org/10.1136/leader-2022-000622 .

Barasa E, Mbau R, Gilson L. What is resilience and how can it be nurtured? A systematic review of empirical literature on Organizational Resilience. Int J Health Policy Manag. 2018;7(6):491–503. https://doi.org/10.15171/ijhpm.2018.06 .

Coetzee C, Van Niekerk D, Raju E. Disaster resilience and complex adaptive systems theory: finding common grounds for risk reduction. Disaster Prev Manage. 2016;25(2):196–211. https://doi.org/10.1108/DPM-07-2015-0153 .

American Psychological Association. Emotion. APA Dictionary of Psychology. https://dictionary.apa.org/emotion . Accessed on 26.6.2024.

Demerouti E, Bakker AB, Nachreiner F, Schaufeli WB. The job demands-resources model of burnout. J Appl Psychol. 2001;86(3):499–512.

Ericsson CR, Lindström V, Rudman A, Nordquist H. Paramedics’ perceptions of job demands and resources in Finnish emergency medical services: a qualitative study. BMC Health Serv Res. 2022;22(1):1469. https://doi.org/10.1186/s12913-022-08856-9 .

Rinkinen T, Kinnula M, Nordquist H. Technological development roles and needs in pre-hospital emergency care from the advanced level paramedics’ perspective. Int Emerg Nurs. 2024;73:101406. https://doi.org/10.1016/j.ienj.2024.101406 .

Moodle Pty Ltd. 2020. https://moodle.org/ . Accessed on date 17.6.2024.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:2:77–101. https://doi.org/10.1191/1478088706qp063oa .

Prezant DJ, Lancet EA, Zeig-Owens R, Lai PH, Appel D, Webber MP, et al. System impacts of the COVID-19 pandemic on New York City’s emergency medical services. J Am Coll Emerg Physicians Open. 2020;9(6):1205–13. https://doi.org/10.1002/emp2.12301 .

Alwidyan MT, Oteir AO, Trainor J. Working during pandemic disasters: views and predictors of EMS providers. Disaster Med Public Health Prep. 2022;16(1):116–22. https://doi.org/10.1017/dmp.2020.131 .

Rebmann T, Charney RL, Loux TM, Turner JA, Abbyad YS, Silvestros M. Emergency Medical Services Personnel’s pandemic influenza training received and willingness to work during a future pandemic. Prehosp Emerg Care. 2020;24(5):601–9. https://doi.org/10.1080/10903127.2019.1701158 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

McCann-Pineo M, Li T, Barbara P, Levinsky B, Berkowitz J. Factors influencing Use of Personal Protective Equipment among Emergency Medical services Responders during the COVID-19 pandemic: a Retrospective Chart Review. West J Emerg Med. 2022;23(3):396–407. https://doi.org/10.5811/westjem.2022.2.55217 .

Vanhaecht K, Seys D, Bruyneel L, Cox B, Kaesemans G, Cloet M, et al. COVID-19 is having a destructive impact on health-care workers’ mental well-being. Int J Qual Health Care. 2021;20(1):mzaa158. https://doi.org/10.1093/intqhc/mzaa158 .

Zolnikov TR, Furio F. First responders and social distancing during the COVID-19 pandemic. J Hum Behav Soc Environ. 2021;31(1–4):244–53. https://doi.org/10.1080/10911359.2020.1811826 .

Spychała A, Piwowarska M, Piekut A. The COVID-19 pandemic as a stress factor in the work of a paramedic. Med Pr. 2023;8(1):9–17. https://doi.org/10.13075/mp.5893.01278 .

Roberts R, Wong A, Jenkins S, Neher A, Sutton C, O’Meara P, et al. Mental health and well-being impacts of COVID-19 on rural paramedics, police, community nurses and child protection workers. Aust J Rural Health. 2021;29(5):753–67. https://doi.org/10.1111/ajr.12804 .

Chang YT, Hu YJ. Burnout and Health issues among Prehospital Personnel in Taiwan Fire Departments during a Sudden Spike in Community COVID-19 cases: a cross-sectional study. Int J Environ Res Public Health. 2022;16(4):2257. https://doi.org/10.3390/ijerph19042257 .

Mausz J, Donnelly EA, Moll S, Harms S, McConnell M. Mental disorder symptoms and the relationship with resilience among paramedics in a single Canadian site. Int J Environ Res Public Health. 2022;17(8):4879. https://doi.org/10.3390/ijerph19084879 .

Phung VH, Sanderson K, Pritchard G, Bell F, Hird K, Wankhade P, et al. The experiences and perceptions of wellbeing provision among English ambulance services staff: a multi-method qualitative study. BMC Health Serv Res. 2022;15(1):1352. https://doi.org/10.1186/s12913-022-08729-1 .

Zamoum K, Gorpe TS. Crisis Management: a historical and conceptual Approach for a better understanding of today’s crises. Crisis Manage - Theory Pract InTech. 2018. https://doi.org/10.5772/intechopen.76198 .

Anderson C, Pooley JA, Mills B, Anderson E, Smith EC. Do paramedics have a Professional Obligation to work during a pandemic? A qualitative exploration of Community Member expectations. Disaster Med Public Health Prep. 2020;14(3):406–12. https://doi.org/10.1017/dmp.2020.212 .

Smith E, Burkle FM, Gebbie K, Ford D, Bensimon C. Acceptable limitations on Paramedic Duty to treat during disaster: a qualitative exploration. Prehosp Disaster Med. 2018;33(5):466–70. https://doi.org/10.1017/S1049023X18000857 .

Smith E, Burkle F, Gebbie K, Ford D, Bensimon C. A qualitative study of paramedic duty to treat during disaster response. Disaster Med Public Health Prep. 2019;13(2):191–6. https://doi.org/10.1017/dmp.2018.15 .

Cypress BS. Rigor or reliability and validity in qualitative research: perspectives, strategies, reconceptualization, and recommendations. Dimens Crit Care Nurs. 2017;36(4):253–63. https://doi.org/10.1097/DCC.0000000000000253 .

TENK. Guidelines for the responsible conduct of research and for handling allegations of misconduct in Finland [Internet]. Helsinki: Finnish National Board on Research Integrity TENK; 2023. https://tenk.fi/sites/default/files/2023-11/RI_Guidelines_2023.pdf . Accessed 13 Jan 2024.

TENK. Ethical review in human sciences [Internet]. Helsinki: Finnish National Board on Research Integrity TENK; 2020. https://tenk.fi/sites/default/files/2021-1/Ethical_review_in_human_sciences_2020.pdf . Accessed 13 Jan 2024.

Download references

Acknowledgements

We want to sincerely thank all the paramedics who participated in this study.

Open access funded by Helsinki University Library.

Open Access funding provided by University of Helsinki (including Helsinki University Central Hospital).

Author information

Authors and affiliations.

Faculty of Medicine, University of Helsinki, Helsinki, Finland

Henna Myrskykari

Emergency Medical Services, University of Turku and Turku University Hospital, Turku, Finland

Department of Healthcare and Emergency Care, South-Eastern Finland University of Applied Sciences, Kotka, Finland

Hilla Nordquist

You can also search for this author in PubMed   Google Scholar

Contributions

Study design (HM, HN). Data collection (HN). Methodology (HN). Analysis (HM, HN). Writing (HM, HN). Review and editing (HM, HN). Supervision (HN). Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Henna Myrskykari .

Ethics declarations

Ethics approval and consent to participate.

The study followed the good scientific practice defined by the Finnish National Board on Research Integrity TENK [ 43 ]. The study was conducted in accordance with the Helsinki Declaration and applicable national guidelines. Adhering to the Finnish National Board on Research Integrity (TENK) guidelines on ethical principles of research with human participants and ethical review in the human sciences in Finland, an ethical review statement from a human sciences ethics committee was not required for this type of study. The participants consisted of adult students engaged in regular employment. Their involvement in the research was grounded on informed consent. The study did not involve concerns regarding the participants’ physical integrity, nor were they subjected to exceptionally strong stimuli. The potential for causing mental harm was not beyond what is typically encountered in everyday life, and their participation did not pose any safety risks [ 44 ].

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Myrskykari, H., Nordquist, H. Paramedics’ experiences and observations: work-related emotions and well-being resources during the initial months of the COVID-19 pandemic—a qualitative study. BMC Emerg Med 24 , 152 (2024). https://doi.org/10.1186/s12873-024-01072-0

Download citation

Received : 25 April 2024

Accepted : 13 August 2024

Published : 26 August 2024

DOI : https://doi.org/10.1186/s12873-024-01072-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Emergency medical services
• Health personnel
• Qualitative research

BMC Emergency Medicine

ISSN: 1471-227X

• General enquiries: [email protected]

No products in the cart.

NVivo 15: Grow Your Research Team with Qualitative Data Analysis Software (QDA Software)

The Most Trusted QDA Software is Even Better

We're thrilled to announce the release of NVivo 15 , the latest version of Lumivero's most trusted qualitative data analysis (QDA) software*. This release is designed to revolutionize the way researchers analyze qualitative data, incorporating advanced Lumivero AI Assistant features and new tools that enhance efficiency across the board.

Driven by customer feedback, NVivo 15 addresses the growing need for more streamlined workflows and efficient qualitative research methods, ensuring researchers can achieve insightful and accurate results with greater ease and speed. With NVivo 15, researchers can organize and explore data from any source, visualize connections that are difficult to find manually, and use our Lumivero AI Assistant to start to dive into the details.

Continue reading to learn more about NVivo 15 or request a demo to see for yourself!

SEE NVIVO IN ACTION

Contact us to learn how you can provide NVivo to all researchers at your college or university and be assured that they are using the most widely cited QDA software in the world.

Revolutionizing Qualitative Data Analysis with AI

The standout feature of NVivo 15 is its groundbreaking Lumivero AI Assistant. This powerful tool is designed to enhance data analysis efficiency and accuracy, giving researchers more time to focus on data analysis for their research project.

New Features with NVivo 15’s Lumivero AI Assistant:

• Unpack Jargon or Technical Language: Leverage AI text summarization to explain unfamiliar terms or local idioms, saving you time from searching for answers. Quickly select the text and have NVivo 15 summarize the selection which is then added to the document as an annotation.
• Dive into Reflective Thinking: Enjoy diving into the depths of rich data but need a break from reading endless pages? NVivo’s AI document summarization brings memoing to the forefront of analysis. Seamlessly select any type of document or document(s) within NVivo 15 and summarize each of them in seconds allowing you to get a sense of each document immediately. The summaries are added in a memo linked to its document where you can reflect further, iteratively returning to each document for further insights and/or start to develop a coding framework.
• Fine tune your coding: Ever lose the rhythm when coding for a while? Dig deeper into codes you already created by using NVivo’s AI coding suggestions to suggest finer level codes. The Lumivero AI Assistant shows the evidence for the proposed codes, giving the researcher total control over what to accept. With NVivo, nothing is cast in stone, giving you the freedom to explore your data dynamically and intuitively.

All your data stays your data when using NVivo 15’s Lumivero AI Assistant. Once the AI completes its task, the information is sent to your NVivo project and deleted from the local servers.

Enhanced Data Organization: Unclutter Your Desktop, Unclutter Your Mind

Working with qualitative data can be messy, but NVivo makes organizing your data a breeze. With NVivo, you can keep all your data organized and accessible, allowing you to pull out insights as needed for effective qualitative analysis. This organizational capability helps to maintain a clear mind and a focused research process, making it easier to manage complex data sets and ensure nothing is overlooked.

Intelligent Data Analysis: Differences Analyzed and Patterns Revealed

NVivo excels in providing deeper insights into your data. It gives a voice to every data point, enabling intricate comparisons not only among respondents but also their organizations or any unit under analysis. Cases, classifications, and attributes highlight differences and unexpected similarities, providing a richer understanding of your data.

For those not satisfied with one-dimensional analysis, NVivo’s crosstabs feature uncovers hidden patterns by contrasting differences among respondents. This tool for qualitative data analysis offers profound insights and clarity, and for those needing more statistical analysis, NVivo allows for exporting data to Lumivero’s XLSTAT . Additionally, NVivo offers the most comprehensive set of text analysis tools and coding queries, enabling you to cut your data in myriad ways to reveal hidden patterns and fresh insights.

Prioritizing Data Privacy and Security in Qualitative Data Analysis Software

As with all Lumivero solutions, NVivo 15 prioritizes data privacy and security. The Lumivero AI Assistant ensures that all your data remains secure throughout the analysis process. Once the AI completes its task, the information is securely sent to your NVivo project and deleted from local servers, preventing any data from residing on outside servers. This ensures that your data stays confidential and protected.

QDA Tools for Enhanced Cross-Platform Compatibility

Beyond its innovative AI features, NVivo 15 excels in cross-platform compatibility, allowing seamless syncing of projects across Mac and Windows. This functionality goes beyond eliminating the file conversion process, enabling users to effortlessly open and manage projects on both operating systems.

With NVivo 15, users can not only open work on projects across both Mac and Windows with ease but also all Mac teams can use NVivo Collaboration Cloud . This ensures greater flexibility and productivity in their qualitative data analysis.

Try NVivo 15 for Your Qualitative Research Today

NVivo 15 is more than just an update; it's a revolution in qualitative data analysis. With the advanced Lumivero AI Assistant and user-centric enhancements, NVivo 15 empowers researchers to achieve more insightful and accurate results in less time. Embrace the future of qualitative data analysis with NVivo 15.

Need help choosing QDA software for your research? Request your free trial of NVivo 15 or request a free demo to see NVivo in action!

*(Scopus Database, NVivo citations compared with competitors 2010-2023)

Recent Articles

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Harris Energizes Democrats in Transformed Presidential Race

1. the presidential matchup: harris, trump, kennedy, table of contents.

• Other findings: Both Harris and Trump are viewed more favorably than a few months ago
• Voting preferences among demographic groups
• How have voters shifted their preferences since July?
• Harris’ supporters back her more strongly than Biden’s did last month
• Large gap in motivation to vote emerges between the candidates’ younger supporters
• Harris and Trump have gained ground with their own coalitions
• Share of ‘double negatives’ drops significantly with change in presidential candidates
• Views of Biden have changed little since his withdrawal from the 2024 presidential race
• Acknowledgments
• The American Trends Panel survey methodology

Nationally, Vice President Kamala Harris and former President Donald Trump are essentially tied among registered voters in the current snapshot of the presidential race: 46% prefer Harris, 45% prefer Trump and 7% prefer Robert F. Kennedy Jr.

Following Biden’s exit from the race, Trump’s support among voters has remained largely steady (44% backed him in July against Biden, while 45% back him against Harris today). However, Harris’ support is 6 percentage points higher than Biden’s was in July . In addition to holding on to the support of those who backed Biden in July, Harris’ bump has largely come from those who had previously said they supported or leaned toward Kennedy.

Harris performs best among the same demographic groups as Biden. But this coalition of voters is now much more likely to say they strongly support her: In July, 43% of Biden’s supporters characterized their support as strong – today, 62% of Harris’ do.

Overall, many of the same voting patterns that were evident in the Biden-Trump matchup from July continue to be seen today. Harris fares better than Trump among younger voters, Black voters, Asian voters and voters with college degrees. By comparison, the former president does better among older voters, White voters and voters without a college degree.

But Harris performs better than Biden across many of these groups – making the race tighter than it was just a few weeks ago.

• In July, women’s presidential preferences were split: 40% backed Biden, 40% preferred Trump and 17% favored Kennedy. With Harris at the top of the ticket, 49% of women voters now support her, while 42% favor Trump and 7% back Kennedy.
• Among men, Trump draws a similar level of support as he did in the race against Biden (49% today, compared with 48% in July). But the share of men who now say they support Harris has grown (to 44% today, up from 38% last month). As a result, Trump’s 10-point lead among men has narrowed to a 5-point lead today.

Race and ethnicity

Harris has gained substantial ground over Biden’s position in July among Black, Hispanic and Asian voters. Most of this movement is attributable to declining shares of support for Kennedy. Trump performs similarly among these groups as he did in July.

• 77% of Black voters support or lean toward Harris. This compares with 64% of Black voters who said they backed Biden a few weeks ago. Trump’s support is unchanged (13% then vs. 13% today). And while 21% of Black voters supported Kennedy in July, this has dropped to 7% in the latest survey.
• Hispanic voters now favor Harris over Trump by a 17-point margin (52% to 35%). In July, Biden and Trump were tied among Hispanic voters with 36% each.
• By about two-to-one, Asian voters support Harris (62%) over Trump (28%). Trump’s support among this group is essentially unchanged since July, but the share of Asian voters backing Harris is 15 points higher than the share who backed Biden in July.
• On balance, White voters continue to back Trump (52% Trump, 41% Harris), though that margin is somewhat narrower than it was in the July matchup against Biden (50% Trump, 36% Biden).

While the age patterns present in the Harris-Trump matchup remain broadly the same as those in the Biden-Trump matchup in July, Harris performs better across age groups than Biden did last month. That improvement is somewhat more pronounced among voters under 50 than among older voters.

• Today, 57% of voters under 30 say they support Harris, while 29% support Trump and 12% prefer Kennedy. In July, 48% of these voters said they backed Biden. Trump’s support among this group is essentially unchanged. And 12% now back Kennedy, down from 22% in July.
• Voters ages 30 to 49 are now about evenly split (45% Harris, 43% Trump). This is a shift from a narrow Trump lead among this group in July.
• Voters ages 50 and older continue to tilt toward Trump (50% Trump vs. 44% Harris).

With Harris now at the top of the Democratic ticket, the race has become tighter.

Much of this is the result of shifting preferences among registered voters who, in July, said they favored Kennedy over Trump or Biden.

Among the same group of voters surveyed in July and early August, 97% of those who backed Biden a few weeks ago say they support or lean toward Harris today. Similarly, Trump holds on to 95% of those who supported him a few weeks ago.

But there has been far more movement among voters who previously expressed support for Kennedy. While Kennedy holds on to 39% of those who backed him in July, the majority of these supporters now prefer one of the two major party candidates: By about two-to-one, those voters are more likely to have moved to Harris (39%) than Trump (20%). This pattern is evident across most voting subgroups.

In July, Trump’s voters were far more likely than Biden’s voters to characterize their support for their candidate as “strong” (63% vs. 43%). But that gap is no longer present in the Harris-Trump matchup.

Today, 62% of Harris voters say they strongly support her, while about a third (32%) say they moderately support her. Trump’s voters are just about as likely to say they strongly back him today as they were in July (64% today, 63% then).

Kennedy’s voters make up a smaller share of voters today than a month ago – and just 18% of his voters say they strongly support him, similar to the 15% who said the same in July.

Across demographic groups, strong support for Harris is higher than it was for Biden

Among women voters who supported Biden in July, 45% said they did so strongly. That has grown to 65% today among women voters who support Harris.

Increased intensity of support is similar among men voters who back the Democratic candidate: In July, 42% of men voters who supported Biden said they did so strongly. This has since grown to 59% of Harris’ voters who are men.

Across racial and ethnic groups, Harris’ supporters are more likely than Biden’s were to say they back their candidates strongly.

Among White voters, 43% who supported Biden in July did so strongly. Today, Harris’ strong support among White voters sits at 64%.

A near identical share of Harris’ Black supporters (65%) characterize their support for her as strong today. This is up from the 52% of Biden’s Black supporters who strongly backed him in July. Among Harris’ Hispanic supporters, 56% support her strongly, while 45% of Asian Harris voters feel the same. Strong support for Harris among these voters is also higher than it was for Biden in July.

Across all age groups, Harris’ strength of support is higher than Biden’s was. But the shift from Biden is less pronounced among older Democratic supporters than among younger groups.

Still, older Harris voters are more likely than younger Harris voters to describe their support as strong. For instance, 51% of Harris’ voters under 50 say they strongly support her, while 71% of Harris supporters ages 50 and older characterize their support as strong.

Today, about seven-in-ten of both Trump supporters (72%) and Harris supporters (70%) say they are extremely motivated to vote.

Motivation to vote is higher in both the Democratic and Republican coalitions than it was in July .

These shifts have occurred across groups but are more pronounced among younger voters.

Today, half of voters under 30 say they are extremely motivated to vote, up 16 points since July. Motivation is up 11 points among voters ages 30 to 49 and 50 to 64, and up 6 points among those ages 65 and older.

Among the youngest voters, the increased motivation to vote is nearly all driven by shifts among Democratic supporters.

• In July, 38% of 18- to 29-year-old Trump voters said they were extremely motivated to vote. Today, a similar share of his voters (42%) report that level of motivation.
• But 18- to 29-year-old Harris supporters are far more likely to say they are extremely motivated to vote than Biden’s supporters in this age group were about a month ago. Today, 61% of Harris’ voters under 30 say this. In July, 42% of voters under 30 who supported Biden said they were extremely motivated to vote.

Sign up for our weekly newsletter

Fresh data delivery Saturday mornings

Sign up for The Briefing

Weekly updates on the world of news & information

• Donald Trump
• Election 2024
• Kamala Harris
• More Leaders
• Political & Civic Engagement

The Political Values of Harris and Trump Supporters

As robert f. kennedy jr. exits, a look at who supported him in the 2024 presidential race, many americans are confident the 2024 election will be conducted fairly, but wide partisan differences remain, joe biden, public opinion and his withdrawal from the 2024 race, amid doubts about biden’s mental sharpness, trump leads presidential race, most popular, report materials.

• August 2024 Presidential Preference Detailed Tables
• Questionnaire

901 E St. NW, Suite 300 Washington, DC 20004 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

© 2024 Pew Research Center

• Open access
• Published: 22 August 2024

Incidence and factors influencing delayed onset of lactation: a systematic review and meta-analysis

• Yijuan Peng 1 , 2   na1 ,
• Ke Zhuang 1 , 2   na1 &
• Yan Huang 1 , 2  

International Breastfeeding Journal volume  19 , Article number:  59 ( 2024 ) Cite this article

238 Accesses

1 Altmetric

Metrics details

Breastfeeding has many benefits for mothers and infants. Lactogenesis II is one of the key steps in the implementation of breastfeeding. If lactogenesis II occurs more than 72 h after delivery, it is termed delayed onset of lactation (DOL). DOL is associated with decreased milk production, shortened breastfeeding time, and pathological neonatal weight loss. A comprehensive summary of the incidence and factors influencing DOL is needed to provide a basis for improving breastfeeding practices and health outcomes.

Studies on the incidence and factors influencing DOL were retrieved from 13 Chinese and English databases (PubMed, Embase, Web of Science, Cochrane Library, CINAHL, etc.) from database inception to August 2023. Two researchers independently conducted the study screening, data extraction and quality evaluation. Stata 16.0 SE software was used for data analysis, and sensitivity analysis and publication bias tests were also performed. The qualitative description method was used to analyse studies that could not be combined quantitatively.

A total of 35 studies involving 19,176 parturients, including 4,922 who had DOL, were included. The mean Newcastle‒Ottawa scale score of the included studies was ≥ 6, indicating that the quality was relatively high. Finally, the incidence of DOL was 30%, and 13 factors influencing DOL with robust results and no publication bias were obtained: prepregnancy body mass index (overweight or obesity), gestational diabetes, gestational hypertension, thyroid disease during pregnancy, serum albumin levels (< 35 g/L), parity, (unscheduled) caesarean section, caesarean section history, daily sleep duration, gestational age, birth weight (< 2.5 kg), breastfeeding guidance and daily breastfeeding frequency. However, there were still six influencing factors with undetermined associations: age, gestational weight gain, birth weight (≥ 4 kg), anxiety, time of first breastfeeding session (maternal separation) and breast massage or treatment.

Conclusions

The incidence of DOL is high. Clinicians should pay attention to parturients at high risk of DOL and formulate targeted prevention strategies according to the influencing factors to reduce the occurrence of DOL and promote better maternal and infant outcomes.

Trial registration

PROSPERO (ID: CRD42023458786), September 10, 2023.

Lactation involves four stages: secretory differentiation, secretory activation, reaching volume, and maintenance of established lactation [ 1 , 2 ]. Among these stages, secretory activation (lactogenesis II) is triggered by a decrease in progesterone levels after delivery of the placenta and involves changes in prolactin and cortisol (glucocorticoid) secretion and the closure of paracellular pathways [ 3 ], indicating that a large amount of milk is being secreted by the mother [ 4 ]. Delayed onset of lactation (DOL) is defined as the occurrence of lactogenesis II 72 h after birth [ 4 ]. The most commonly used evaluation method for the onset of lactation (OL) is maternal perception of milk coming in [ 1 ]. Importantly, studies have shown that the time of OL is negatively correlated with the amount of milk produced on the 14th day postpartum [ 5 ]. DOL independently increases the risk of the cessation of any or exclusive breastfeeding at 4 weeks postpartum by 62% [ 6 ], thereby shortening the duration of breastfeeding [ 7 , 8 ] and reducing the rate of exclusive breastfeeding [ 7 , 9 ]. Moreover, DOL can increase the risk of pathological neonatal weight loss (more than 10% of the birth weight) by 7.1-fold [ 10 , 11 ]. Consequently, actively taking effective intervention measures to prevent DOL has an important impact on improving maternal and child health outcomes and breastfeeding practices.

Liu et al. [ 12 ] and Miao et al. [ 13 ] published systematic reviews on the prevalence and factors influencing DOL in Chinese women in 2021 and 2023, respectively. The results revealed that the prevalence of DOL was 24% [ 12 ] and 31% [ 13 ], respectively, and an increasing trend of DOL, which should attract the attention of clinical workers. In addition, the existing systematic reviews may not be sufficiently comprehensive in literature retrieval and statistical analysis strategies, and their reporting of results may also be inadequate [ 12 , 13 , 14 ], potentially affecting the comprehensiveness and consistency of the findings. Therefore, this study focused on the global perspective and prospective research to determine the incidence of DOL and analyse the factors influencing DOL quantitatively through meta-analysis and to summarize the influencing factors that cannot be quantitatively analysed via qualitative description, to provide evidence supporting the development of effective evaluation and intervention measures for preventing DOL.

This review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement [ 15 ] and was registered in the International Prospective Register of Systematic Reviews (PROSPERO) (ID: CRD42023458786).

Inclusion and exclusion criteria

Studies were considered eligible if they met all of the following criteria: (1) included women who chose to breastfeed after delivery; (2) studied the incidence or factors influencing DOL; (3) included the occurrence of DOL as the outcome and used the time of obvious breast tenderness or the sensation of milk coming in more than 72 h after delivery as the diagnostic criteria; and (4) used a prospective observational design.

Studies that met one of the following exclusion criteria were excluded: (1) studies with incomplete or erroneous data on variables; (2) studies for which data could not be directly or indirectly extracted; (3) studies for which the original text could not be obtained or the type of article was a review, conference paper, correspondence, comments, or study protocol; (4) studies published in different articles including the same participants: (a) for multiple studies of the same research object, the study with the most abundant research content or the most detailed description of the data was included; (b) for multiple studies with overlapping samples, the study with the longest study period was included; otherwise, the most recent study was included; 5) studies not published in Chinese or English; 6) studies with a sample size < 60; 7) nonhuman studies; or 8) studies for which the literature quality was low (quality assessment score ≤ 5).

Systematic search and strategy

Three researchers (YJP, KZ, and YH) jointly developed the search strategy and comprehensively searched the following databases for all relevant Chinese and English studies from database inception to August 2023: (1) Chinese databases: China National Knowledge Infrastructure (CNKI), Wanfang Database, China Science and Technology Journal Database (Weipu Database), and Chinese Biomedical Database (CBM); (2) English databases: PubMed, Ovid-Embase, Web of Science (via Web of Science Core Collection), Cochrane Library (the Cochrane central register of controlled trials, CENTRAL), CINAHL Plus (via EBSCOhost), APA Psycinfo (via EBSCOhost), Scopus, OpenGrey, and ProQuest Dissertations and Theses database (see full search strategy in Supplementary Material 1). To prevent the omission of relevant research, we reviewed the references of the included studies and relevant reviews. After the search was completed, duplicates were automatically removed by Endnote X9, followed by manual screening.

Study selection process and data extraction

Two researchers (YJP and KZ) independently screened the studies and extracted the data, and a third senior researcher (YH) independently reviewed and discussed the differences. Preliminary screening was performed by reading the title and abstract, followed by rescreening by reading the full text. If the title and abstract were not sufficient to make a decision, a decision was done by reading the full text. After the screening process was completed using Endnote X9, data were extracted from the final included studies, including the title, first author, year, study design, population characteristics, number of cases of DOL, sample size, study period, country, incidence, follow-up method and endpoint. In addition, all the influencing factors mentioned in the original study were extracted, and the influencing factors mentioned in two or more studies with the same definition were then identified. Finally, the exposure and outcome data of these influencing factors were extracted. For exposure variables, the number of cases for categorical variables and the mean ± standard deviation or median [interquartile range] for continuous variables were extracted.

Quality assessment

Prospective observational studies were included, so the Newcastle‒Ottawa Scale (NOS) [ 16 ] was used to evaluate the quality of the included studies. The NOS includes three columns and eight items. The three columns specifically include the selection of the research population, comparability between groups, and the measurement of results or exposure factors, and the total score ranges from 0 to 9 points. Because the data extracted were the number of cases, the influence of confounding factors could not be controlled for, which might lead to the deterioration of ‘comparability between groups’. The diagnostic criteria for DOL were based on maternal self-reported breast distension, which would lead to an insufficient evaluation; therefore, two points were deducted for all included studies. On this basis, studies with a score ≥ 6 points were considered high-quality studies. Two researchers (YJP and KZ) independently evaluated the quality of the included studies according to the evaluation criteria of the NOS [ 17 ]. When the opinions of the two researchers were inconsistent, the study was assigned to a third senior researcher (YH) for independent evaluation and discussion.

Statistical analysis

The extracted data and quality evaluation results were collated into Microsoft Excel 2021, and the data that could be used for quantitative analysis were entered into Stata 16.0 SE software for statistical analysis. In this study, a combined analysis or qualitative description of the influencing factors was performed only for variables included in at least two or more original studies. Because these were prospective observational studies, the risk ratio (RR) was used to combine the effect values for categorical variables, and the weighted mean difference (WMD) was used to combine the effect values for continuous variables. Cochran’s Q test and the I 2 statistic were used to quantitatively analyse the heterogeneity between studies: (1) if I 2  < 50% and p  > 0.05, the heterogeneity between studies was considered low, then a fixed effect model was used; (2) if I 2  ≥ 50% or p  ≤ 0.05, the heterogeneity between studies was considered high, then a random effect model was used for more conservative statistical analysis. Subgroup analysis was performed to explore the source of heterogeneity.

When a certain influencing factor was included in three or more original studies, sensitivity analysis was carried out by eliminating the studies one by one and merging the remaining studies to test whether the results of the meta-analysis were robust, and we explored the reasons for nonrobust results. If not, the meta-analysis was abandoned, and a qualitative description was carried out instead. When the number of original studies including a certain influencing factor was ≥ 10, a funnel plot was drawn, and Egger’s test was performed to explore whether publication bias existed. If so, the clipping method was used to correct the asymmetry of the funnel plot and the combined effect caused by publication bias. p  < 0.05 was considered statistically significant.

Search results and selection

We retrieved 13,112 studies from Chinese and English databases conducted before August 2023, and 9,489 studies were obtained by removing duplicate studies through Endnote and manual methods. After title and abstract screening, 189 studies were included. After rescreening by reading the full text, 32 studies that reported both the incidence and factors influencing DOL and 3 studies that reported only the incidence were ultimately included. No new studies were found after reviewing the references of the included studies and related reviews. The study screening process is shown in Fig.  1 .

PRISMA flowchart for the identification of studies

Characteristics and quality evaluation of the included studies

A total of 35 included studies were conducted from 1999 to 2023 in 8 countries, including China ( n  = 23), the United States of America (USA, n  = 6), Canada ( n  = 1), Peru ( n  = 1), India ( n  = 1), Australia ( n  = 1), Brazil ( n  = 1), and Ghana ( n  = 1). A total of 19,176 women were included in these studies, 4,922 of whom had DOL. The methods of follow-up involved medical records, questionnaires, or interviews. The mean NOS score of all the studies was ≥ 6 points, indicating that these studies had good methodological quality. The general characteristics and NOS scores of the included studies, sorted alphabetically by author name, are summarized in Table  1 .

Meta-analysis and systematic review results

The combined incidence of DOL was obtained via meta-analysis. The meta-analysis results of factors influencing DOL are summarized in Table  2 according to the order of reporting. Combined with the qualitative description of the influencing factors, all the influencing factors involved could be divided into three categories: maternal-related factors, infant-related factors, and breastfeeding-related factors.

DOL incidence

A random-effects model was used to assess the incidence of DOL in 35 studies, and the result was 30% (95% CI 26, 34) (Fig.  2 ). Subgroup analysis by country category, combined with at least two or more studies, revealed an incidence of DOL of 30% in China (95% CI 26, 35) and 34% in the USA (95% CI 24, 43). The incidence of DOL in the USA was slightly higher than that in China (Fig.  3 ).

Forest plot of DOL incidence

Forest plot of the subgroup analysis of DOL incidence

Maternal-related influencing factors

There was a statistically significant difference in age [ 5 , 26 , 28 , 33 ] between the DOL and non-DOL groups (WMD =-0.30; 95% CI -0.573, -0.40), but the sensitivity analysis result was not robust. The combined results of 9 studies [ 24 , 31 , 36 , 41 , 42 , 43 , 45 , 46 , 48 ] and 5 studies [ 4 , 20 , 30 , 40 , 49 ], respectively, could not determine the association between a maternal age ≥ 35 years (RR = 1.40; 95% CI 0.96, 2.04) and ≥ 30 years (RR = 1.33; 95% CI 0.98, 1.80) and DOL.

The pooled results of 8 studies [ 20 , 21 , 23 , 24 , 37 , 40 , 45 , 48 ] and 3 studies [ 27 , 29 , 46 ], respectively, revealed that a prepregnancy BMI ≥ 25.0 kg/m 2 (RR = 1.47; 95% CI 1.17, 1.84) and a prepregnancy BMI ≥ 24.0 kg/m 2 (RR = 1.41; 95% CI: 1.14, 1.74) were risk factors for DOL. However, the correlation between prepregnancy BMI [ 26 , 30 , 33 ] (WMD = 1.26; 95% CI -1.22, 3.75) and DOL was uncertain. Excessive gestational weight gain (GWG) [ 24 , 37 , 40 , 41 , 45 , 46 , 48 ] (RR = 1.38; 95% CI 1.07, 1.77) was a risk factor for DOL, but the sensitivity analysis result was not robust.

The combined results of 14 [ 20 , 21 , 24 , 26 , 28 , 29 , 30 , 33 , 36 , 37 , 38 , 41 , 45 , 48 ], 13 [ 5 , 20 , 21 , 27 , 28 , 29 , 30 , 36 , 37 , 38 , 41 , 42 , 45 ], 6 [ 21 , 29 , 35 , 38 , 41 , 47 ] and 2 [ 5 , 29 ] studies, respectively, revealed that gestational diabetes mellitus (GDM) (RR = 1.32; 95% CI 1.18, 1.49), hypertensive disorders of pregnancy (HDP) (RR = 1.66; 95% CI 1.30, 2.12), thyroid disease during pregnancy (RR = 1.18; 95% CI 1.05, 1.32), and a serum albumin level < 35 g/L (RR = 1.57; 95% CI 1.12, 2.20) were risk factors for DOL. The descriptive analysis could not determine whether anaemia [ 21 , 29 , 38 ] was associated with DOL, and there might be no association between ovarian cysts during pregnancy [ 21 , 29 ] and DOL.

The pooled results of 23 studies [ 5 , 19 , 20 , 21 , 22 , 23 , 25 , 26 , 27 , 28 , 29 , 31 , 33 , 35 , 36 , 38 , 39 , 40 , 41 , 43 , 44 , 46 , 48 ] revealed that primiparity (RR = 1.40; 95% CI 1.25, 1.56) was a risk factor for DOL. The combined results of 20 [ 4 , 5 , 24 , 26 , 28 , 30 , 31 , 32 , 33 , 35 , 36 , 37 , 38 , 40 , 41 , 44 , 45 , 46 , 48 , 49 ] and 2 [ 21 , 29 ] studies, respectively, revealed that caesarean delivery (RR = 1.33; 95% CI 1.17, 1.52) and unscheduled caesarean delivery (RR = 1.24; 95% CI 1.02, 1.51) were risk factors for DOL. A history of caesarean delivery [ 21 , 29 ] (RR = 0.75; 95% CI 0.60, 0.93) was a protective factor against DOL. The correlations between the duration of labour [ 23 , 45 ] (WMD = 1.97; 95% CI -2.21, 6.16), vaginal delivery, and the duration of the second stage of labour > 1 h [ 4 , 19 ] (RR = 1.41; 95% CI 0.73, 2.72) and DOL were undefined.

Daily sleep duration [ 5 , 28 ] (WMD =-0.24; 95% CI -0.45, -0.02) was a protective factor against DOL. The correlation between an Edinburgh Postnatal Depression Scale (EPDS) score ≥ 9 points [ 35 , 48 ] (RR = 1.24; 95% CI 0.80, 1.93) and DOL was unknown. Moreover, descriptive analysis revealed that the relationship between depression [ 4 , 31 , 45 ] and DOL remained unknown, but there might be a correlation between anxiety [ 22 , 31 , 45 ] and DOL.

The results of the meta-analysis revealed that the relationships between the following variables and DOL could not be determined: education level (≥ high school [ 22 , 32 ], ≥junior college [ 20 , 41 , 44 ], ≥junior undergraduate [ 4 , 21 , 23 , 24 , 37 , 46 , 48 ], and > 9 years [ 29 , 49 ]), occupational status [ 21 , 23 , 29 , 44 , 48 ], mean monthly household income per person (≤ 5000 RMB [ 29 , 31 , 44 ], and ≥ 10000 RMB [ 21 , 26 , 41 ]), nationality (Hispanic [ 4 , 24 , 35 ], White [ 33 , 35 ], and Han Chinese [ 20 , 22 , 26 , 44 , 45 ]), prenatal smoking status [ 4 , 24 , 26 , 35 ], prenatal alcohol consumption status [ 21 , 26 , 27 , 35 , 37 ], assisted reproductive technology (ART) use [ 21 , 41 ], planned pregnancy [ 22 , 43 ], insulin treatment [ 21 , 31 ], and fluid infusion [ 5 , 28 ]. The descriptive results revealed that height [ 4 , 26 ], intraoperative or delivery blood loss [ 4 , 21 , 29 ], and drug-induced labour [ 4 , 35 , 37 ] might not be related to DOL, whereas the relationships between stressful life events [ 22 , 45 , 49 ] during pregnancy, exercise during pregnancy [ 20 , 48 ], and anaesthesia or painkiller use [ 4 , 21 , 24 , 27 , 29 , 35 , 37 ] and DOL remained unclear.

Infant-related influencing factors

The pooled results of 7 [ 29 , 30 , 31 , 40 , 41 , 42 , 49 ] and 2 [ 5 , 28 ] studies, respectively, revealed that a gestational age < 37 weeks (RR = 1.29; 95% CI 1.06, 1.57) and a young gestational age at birth (WMD =-0.47; 95% CI -0.89, -0.06) were risk factors for DOL. Nevertheless, whether gestational age ≥ 39 weeks [ 4 , 32 ] (RR = 1.11; 95% CI 0.86, 1.43) and gestational age (full-term) [ 26 , 45 , 46 ] (WMD =-0.04; 95% CI -0.26, 0.19) were associated with DOL could not be determined.

A birth weight < 2.5 kg [ 29 , 31 ] (RR = 1.34; 95% CI 1.07, 1.67) was a risk factor for DOL. A birth weight ≥ 4 kg [ 22 , 26 , 48 ] (RR = 1.29; 95% CI 1.07, 1.56) was also a risk factor, but the sensitivity analysis result was not robust. However, the correlations between neonatal birth weight [ 30 , 33 , 45 ] (WMD =-0.36; 95% CI -0.86, 0.14) and preterm birth weight [ 5 , 28 ] (WMD =-17.09; 95% CI -102.28, 68.09) and DOL were unclear.

The meta-analysis results could not determine whether neonatal sex [ 4 , 21 , 22 , 26 , 29 , 32 , 46 , 48 , 49 ] and the 1-min Apgar score (< 7 points [ 21 , 28 , 38 ] and < 8 points [ 4 , 32 , 35 ]) were associated with DOL. The descriptive results revealed that the relationships between maternal separation [ 21 , 36 , 43 ] and skin-to-skin contact [ 4 , 35 , 36 , 37 ] and DOL were unknown.

Breastfeeding-related influencing factors

The combined results of 7 studies [ 21 , 22 , 29 , 36 , 37 , 46 , 48 ] revealed that receiving breastfeeding guidance (RR = 0.72; 95% CI 0.64, 0.81) was a protective factor against DOL. The correlation between ≥ 3 breastfeeding information sources [ 22 , 43 ] (RR = 0.50; 95% CI 0.15, 1.65) and DOL was unknown. Descriptive analysis revealed that breast massage or treatment [ 22 , 36 ] might be associated with DOL.

The descriptive analysis results revealed that there might be a relationship between a first breastfeeding session after maternal separation [ 5 , 28 , 42 ] and DOL, but it was not clear whether the first breastfeeding session of general mothers [ 22 , 29 , 32 , 33 , 35 , 37 , 45 ] was related to DOL. The combined results of 5 studies [ 5 , 28 , 42 , 45 , 46 ] revealed that breastfeeding frequency (WMD =-0.63; 95% CI -1.10, -0.16) was a protective factor against DOL. Similarly, a breastfeeding frequency ≤ 2 times on the first day [ 22 , 44 ] (RR = 1.92; 95% CI 1.36, 2.72) and the second day after surgery [ 22 , 44 ] (RR = 1.71; 95% CI 1.34, 2.20) was a risk factor for DOL. The correlation between a breastfeeding frequency < 8 times [ 4 , 32 ] from 0 to 24 h after birth (RR = 1.00; 95% CI 0.78, 1.28) and DOL was unknown.

The meta-analysis results revealed that the relationships between a history of breastfeeding [ 5 , 22 , 28 ], previous insufficient lactation [ 21 , 29 ], prenatal breast enlargement [ 4 , 19 ], flat or sunken nipples [ 4 , 21 , 32 , 35 ], and a bra cup size ≥ D [ 4 , 46 ] and DOL could not be confirmed. The descriptive results revealed that the relationships between formula milk use (within 24 h [ 4 , 32 , 46 ] or 48 h [ 4 , 32 , 35 , 48 ]), LATCH score [ 26 , 35 , 46 ] and nipple pain during lactation [ 4 , 22 , 35 ] and DOL remained unknown.

Sensitivity and publishing bias analysis

The sensitivity analysis results for age (continuous variable), GWG and neonatal birth weight ≥ 4 kg were not robust, whereas the results for the remaining variables were robust. An analysis of the full texts of the studies including these three variables and the sensitivity analysis results revealed that when studies with relatively small samples (< 400 participants) were excluded one by one, the sensitivity analysis results were robust; however, when studies with relatively large samples (> 1,000 participants) were excluded, the sensitivity analysis results became not robust (see Supplementary Material 2).

The results of Egger’s test indicated that there was no publication bias for GDM ( p  = 0.129), HDP ( p  = 0.136), primipara ( p  = 0.125), and caesarean delivery ( p  = 0.675) (see Table  2 ). The funnel plots for these four variables also exhibited a basically symmetrical distribution, further suggesting the absence of significant publication bias (see Supplementary Material 3, 4, 5, and 6).

The total incidence of DOL was 30% among the 35 included studies. Subgroup analysis revealed that the incidence of DOL was 30% in China and 34% in the USA. Both China and the USA have made many efforts to support breastfeeding and have introduced policies related to breastfeeding, which focus on the positive role of baby-friendly hospitals and policy support, as well as the ‘Ten Steps to Successful Breastfeeding’ framework, breastfeeding clinics and human milk donation programs [ 51 , 52 ]. Additionally, the American Academy of Pediatrics policy mentions relevant content regarding OL [ 52 ]; however, attention to the important impact of OL on breastfeeding is lacking, and China’s policy does not consider OL [ 51 ], which may be the reason for the high DOL incidence. Moreover, Patel et al.’s systematic review [ 53 ] revealed the effectiveness of dedicated certified lactation consultants or counsellors in promoting breastfeeding, which suggests that they may also have a positive effect on OL support, but this remains to be verified.

The analysis of potential factors influencing DOL revealed statistically significant correlations between DOL and 15 factors: maternal age, prepregnancy BMI (overweight or obesity), GWG, GDM, HDP, thyroid disease during pregnancy, serum albumin levels (< 35 g/L), parity, (unscheduled) caesarean section, caesarean section history, daily sleep duration, gestational age, birth weight (< 2.5 kg or ≥ 4 kg), breastfeeding guidance and daily breastfeeding frequency. However, the sensitivity analysis results for age, GWG and birth weight ≥ 4 kg were not robust. Through descriptive analysis, three factors were found to be likely related to DOL: anxiety, time of first breastfeeding session (maternal separation), and breast massage or treatment.

Combined with the meta-analysis and sensitivity analysis results, although there was a correlation between age and DOL, the result was not robust. A relationship between a maternal age ≥ 35 years or ≥ 30 years and DOL was not found. This suggests that the relationship between maternal age and DOL is still controversial, and more research is needed.

Although the WHO [ 54 ] and China [ 55 ] have slightly different BMI classification criteria, when the prepregnancy BMI reaches the overweight or obese range, the risk for DOL increases. Studies have shown that women who are overweight or obese before pregnancy have a lower response to prolactin stimulated by sucking [ 56 ]. Animal experiments have shown that obesity may impair lactation performance by inducing prolactin resistance [ 57 ]. Obesity is an important risk factor for insulin resistance and impaired insulin secretion; insulin is now thought to play a direct role in lactation, including secretory differentiation, secretory activation and mature milk production [ 58 ]. The results of this study also revealed that a high prepregnancy BMI was a risk factor for DOL after the standard of overweight or obesity was reached. High GWG may increase the risk of DOL. Although the sensitivity analysis results were not robust, considering the adverse effects of overweight or obesity on DOL, these findings still suggest that GWG has a potentially dangerous effect on DOL, which still requires exploration and verification in further research.

Our analysis revealed that GDM, HDP and thyroid disease during pregnancy were risk factors for DOL. De Bortoli et al.’s systematic review [ 59 ] also supports that GDM is a risk factor for DOL. The possible mechanism is that insulin resistance and/or insulin secretion disorders in β cells lead to GDM, of which insulin resistance is the main cause [ 60 ], and insulin resistance affects lactation [ 61 ]. The ratio of insulin to glucose and adiponectin may also be related to the start time of lactation [ 62 ]. Combined with the relationship between obesity and insulin resistance, GDM, obesity and insulin resistance may be associated with DOL in some way [ 63 ]. HDP can affect the initiation and duration of breastfeeding [ 64 ], and the treatment of HDP may also affect lactation; for example, diuretics may reduce milk production [ 65 ]. HDP may also lead to placental dysfunction and decreased prolactin secretion, thereby affecting lactation [ 66 ]. Endothelial dysfunction caused by preeclampsia may lead to hypoalbuminaemia in women [ 67 ], and lower serum albumin levels indicate poor nutritional status, which may be the cause of DOL [ 29 ]. Consistently, this study also revealed that low serum albumin levels were a risk factor for DOL. Animal experiments have shown that hypothyroidism may hinder the ability of the breast to achieve normal milk synthesis and excretion, leading to lactation disorders in pregnant women with thyroid dysfunction [ 68 ]. Similarly, hyperthyroidism can also induce impaired release of oxytocin, resulting in milk deposition, apoptosis of glandular cells that secrete milk, and lactation effects [ 69 ].

This study revealed that primiparity and (unscheduled) caesarean section were risk factors for DOL, whereas a history of caesarean section was a protective factor against DOL. Compared with multiparas, primiparas may experience longer delivery times, resulting in higher cortisol levels [ 70 , 71 ], and multiparas may have more prolactin receptors than primiparas [ 72 ]. Multiparas may also have better breastfeeding skills than primiparas [ 73 ]; hence, primiparas are more likely to experience DOL than multiparas. Similarly, women with a history of caesarean section could have certain breastfeeding experiences; their fear of childbirth in late pregnancy is relatively low [ 74 ], and the pressure of childbirth seems to be related to DOL [ 71 ]. These factors may be why a history of caesarean section is a protective factor against DOL. Compared with vaginal delivery, (unscheduled) caesarean section may lead to lower levels of oxytocin and prolactin secretion [ 75 , 76 ], thus increasing the risk of DOL.

Sleep and emotional state may affect the occurrence of DOL. Prolactin secretion has circadian rhythm changes, and sleep deprivation may lead to decreased levels of prolactin secretion [ 77 ]. Anxiety and depression are associated with lower oxytocin during feeding [ 78 ], and mothers with depression may have insufficient confidence in their ability to breastfeed [ 79 ]. In addition, studies have shown that poor sleep quality is associated with depression and anxiety [ 80 , 81 ]. However, combined with the results of the meta-analysis and descriptive analysis of this study, the relationship between depression and DOL needs further exploration.

This study revealed that young gestational age and low birth weight were risk factors for DOL. The shortening of pregnancy may lead to insufficient prenatal breast preparation, and the immature sucking skills of premature infants can lead to insufficient milk discharge [ 82 ]. A low birth weight may mean that an infant’s motor development is deficient, which may also affect the infant’s sucking skills, subsequently increasing the risk of DOL [ 73 , 83 ]. Colostrum is produced before OL (paracellular pathway closure) [ 84 ]. Unlike mature milk formed after OL, colostrum is rich in immune factors and cytokines, and the concentration of these substances is inversely proportional to the duration of gestation [ 84 ]. Newborns have immature immune systems, especially premature infants whose immune substance transport through the placenta is interrupted prematurely, and colostrum can address this lack of development by providing many bioactive substances [ 85 ]. In addition, preterm birth may trigger delayed closure of the paracellular pathway to prolong the supply of protective substances in colostrum [ 84 ], although this can lead to DOL. These findings also suggest that medical staff should pay attention to the special therapeutic effect of colostrum on premature infants. For example, oropharyngeal colostrum administration, as proposed by Rodriguez et al. [ 84 ] in 2009, has been shown to have a positive effect on the outcomes of preterm infants [ 86 ]. Therefore, in clinical practice, it is necessary to help mothers start breastfeeding early and express colostrum, especially for mothers with premature or low-birth-weight infants, to facilitate the use of colostrum for neonatal immune protection.

The WHO recommends that breastfeeding counselling be provided to all pregnant women and women with babies to help enhance their skills, abilities and confidence in breastfeeding [ 87 ]. McFadden et al.’s systematic review [ 88 ] also revealed that breastfeeding counselling has a positive effect on breastfeeding. Consistent with the results of our study, breastfeeding counselling had a positive protective effect on OL. This study revealed that the frequency of breastfeeding was a protective factor against DOL and should not be less than 2 times/day. Sucking stimulation can trigger the pituitary to release oxytocin, which may be beneficial for uterine involution [ 89 ], and frequent breastfeeding and effective milk emptying have positive effects on milk secretion [ 90 , 91 ]. These findings suggest that in the case of maternal separation, due to the lack of infant sucking, it is necessary to start hand expressing or using a breast pump as soon as possible to mechanically stimulate the areola to promote the release of oxytocin [ 91 ], thereby reducing the risk of DOL; moreover, informing mothers of the potential benefits of frequent sucking on uterine involution is recommended to improve their compliance. However, whether the time of the first breastfeeding session of general parturients is related to DOL still needs further exploration. This study revealed that breast massage or treatment might be a protective factor against DOL, and the protective effect may be achieved by simulating sucking and dredging the mammary duct [ 36 ].

Strengths and limitations

In the original studies included, different researchers might use different criteria for the same influencing factor. Therefore, this study combined quantitative and qualitative analyses to comprehensively summarize the available studies on the incidence and factors influencing DOL. However, this study inevitably has several limitations: (1) Since most of the original studies reported only statistically significant multivariate analysis results and the multivariate analysis methods used were inconsistent, we chose to extract the exposure and outcome data corresponding to the influencing factors after weighing the effects of bias and confounding on the results; regrettably, few studies reported only statistically significant univariate analysis results, but no publication bias was found by funnel plots or Egger’s tests. (2) There were three factors for which the sensitivity analysis results were not robust. A review of the original studies included revealed that some studies had relatively small sample sizes, which might have resulted in insufficient statistical power. When large-sample studies are eliminated, the results might be affected by the combination of small-sample studies; moreover, sensitivity analysis is not applicable to factors that sourced from only two original studies, so robustness cannot be evaluated. Accordingly, larger samples and higher-quality studies are needed to improve the accuracy and robustness of the results. (3) At present, there are no objective and unified diagnostic criteria for DOL, and the most commonly used method is still the subjective perception of maternal breast distension; however, this method may have a large bias. Therefore, it is still necessary to research milk biomarkers to develop an objective and standard evaluation method for use in clinical practice. (4) The selection of the qualitative description method for some influencing factors was due to the high degree of heterogeneity among the studies, so the results of the qualitative description only have implications, and exact conclusions cannot be drawn. (5) As Chinese researchers, considering the accessibility of the Chinese language, we searched Chinese databases, which may have resulted in the inclusion of many Chinese studies. The studies included were mainly from China and the USA, owing to differences in culture and policy, the results concerning the incidence and factors influencing DOL may vary greatly across countries and even within individual countries. Nevertheless, the results may play a role in the implementation of DOL incidence and influencing factor research by researchers from other countries.

This study revealed that the incidence of DOL was 30%, and the factors influencing DOL may include prepregnancy BMI (overweight or obesity), GDM, HDP, thyroid disease during pregnancy, serum albumin levels (< 35 g/L), parity, (unscheduled) caesarean section, caesarean section history, daily sleep duration, gestational age, birth weight (< 2.5 kg), breastfeeding guidance and daily breastfeeding frequency; however, the relationships between age, GWG, birth weight (≥ 4 kg), anxiety, time of first breastfeeding session (maternal separation) and breast massage or treatment and DOL remain unknown. Considering the adverse effects of DOL, policymakers should pay more attention to OL, a critical period of breastfeeding, and formulate corresponding supportive policies. Researchers are advised to explore and verify objective diagnostic criteria for DOL and the influencing factors for which the associations with DOL remain unknown. In addition, establishing breastfeeding support teams in hospitals is recommended, and clinicians should conduct targeted assessments, risk stratification management, health education and interventions for mothers according to the influencing factors to reduce the occurrence of DOL in the case of rational medical resource use.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Assisted reproductive technology

Body mass index

• Delayed onset of lactation

Edinburgh Postnatal Depression Scale

Gestational diabetes mellitus

Gestational weight gain

Hypertensive disorders of pregnancy

Maternal intensive care unit

Maternal separation

Newcastle‒Ottawa Scale

Onset of lactation

United States of America

World Health Organization

Weighted mean difference

Medina Poeliniz C, Engstrom JL, Hoban R, Patel AL, Meier P. Measures of secretory activation for research and practice: an integrative review. Breastfeed Med. 2020;15(4):191–212.

Article   PubMed   Google Scholar  

Meier PP, Patel AL, Hoban R, Engstrom JL. Which breast pump for which mother: an evidence-based approach to individualizing breast pump technology. J Perinatol. 2016;36(7):493–9.

Article   CAS   PubMed   PubMed Central   Google Scholar  

McManaman JL, Neville MC. Mammary physiology and milk secretion. Adv Drug Deliv Rev. 2003;55(5):629–41.

Article   CAS   PubMed   Google Scholar  

Nommsen-Rivers LA, Chantry CJ, Peerson JM, Cohen RJ, Dewey KG. Delayed onset of lactogenesis among first-time mothers is related to maternal obesity and factors associated with ineffective breastfeeding. Am J Clin Nutr. 2010;92(3):574–84.

Luan D. [The research on milk volume and influencing factors in mothers with preterm infants hospitalized in NlCU] [Thesis]: Binzhou Medical University; 2018.

Brownell E, Howard CR, Lawrence RA, Dozier AM. Delayed onset lactogenesis II predicts the cessation of any or exclusive breastfeeding. J Pediatr. 2012;161(4):608–14.

Article   PubMed   PubMed Central   Google Scholar  

Huang L, Xu S, Chen X, Li Q, Lin L, Zhang Y, et al. Delayed lactogenesis is associated with suboptimal breastfeeding practices: a prospective cohort study. J Nutr. 2020;150(4):894–900.

Chapman DJ, Pérez-Escamilla R. Does delayed perception of the onset of lactation shorten breastfeeding duration? J Hum Lactation. 1999;15(2):107–11.

Article   CAS   Google Scholar  

Hruschka DJ, Sellen DW, Stein AD, Martorell R. Delayed onset of lactation and risk of ending full breast-feeding early in rural Guatemala. J Nutr. 2003;133(8):2592–9.

Michel MP, Gremmo-Féger G, Oger E, Sizun J. [Pilot study of early breastfeeding difficulties of term newborns: incidence and risk factors]. Archives de Pédiatrie. 2007;14(5):454–60.

Dewey KG, Nommsen-Rivers LA, Heinig MJ, Cohen RJ. Risk factors for suboptimal infant breastfeeding behavior, delayed onset of lactation, and excess neonatal weight loss. Pediatrics. 2003;112(3 Pt 1):607–19.

Liu J, Liu M, Jiang P, Chen H, Hu S, Fu J. [Incidence and influencing factors of delayed onset of lactogenesis II in Chinese parturient women: a systematic review]. Chin J Mod Nurs. 2021;27(10):1300–5.

Google Scholar  

Miao Y, Zhao S, Liu W, Jiang H, Li Y, Wang A, et al. Prevalence and risk factors of delayed onset lactogenesis II in China: a systematic review and meta-analysis. J Maternal-Fetal Neonatal Med. 2023;36(1):2214833.

Article   Google Scholar  

Hu S, Liu J, Jiang P, Sun Z, Zhu Q, Fu J. [A systematic review of the incidence and influencing factors of delayed onset of lactogenesis II]. Chin Gen Pract. 2021;24(24):3110–5.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Stang A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol. 2010;25(9):603–5.

Zeng X, Liu H, Chen X, Leng W. [Meta-analysis series (IV): quality assessment tools for observational studies]. Chin J Evidence-Based Cardiovasc Med. 2012;4(04):297–9.

Bai T, Yang Y, Fu X, Zhou Y, Wei X, Zhang F. [Analysis of the impact of postpartum fatigue on the onset of lactation]. Chin J Nurs. 2018;53(4):438–42.

Chapman DJ, Perez-Escamilla R. Identification of risk factors for delayed onset of lactation. J Am Diet Assoc. 1999;99(4):450–512.

Ding P, Zhao M, Zhang F, Wang L, Yao J, Qiu J, et al. [Nutrients intake in the third trimester and associated factors of delayed onset of lactogenesis II in maternal women]. Chin Gen Pract. 2020;23(5):534–9.

Ding J, Lian W, Ma X. [Construction and validation of risk prediction model for delayed onset of lactogenesis stage II following cesarean section]. Chin J Perinat Med. 2022;25(9):661–9.

Ding Q. [Analysis of factors influencing delayed onset of lactogenesis and construction of a predictive model in advanced cesarean section mothers] [Thesis]: Guangxi University of Chinese Medicine; 2022.

Dong F, Li L, Zhu K, Zhang S, Guan Y, Han J, et al. [Analysis of current status and influencing factors of lactation initiation delay in women with vaginal delivery]. Chin J Practical Nurs. 2022;38(19):1496–502.

Haile ZT, Chavan BB, Teweldeberhan A, Chertok IR. Association between gestational weight gain and delayed onset of lactation: the moderating effects of race/ethnicity. Breastfeed Med. 2017;12:79–85.

Hilson JA, Rasmussen KM, Kjolhede CL. High prepregnant body mass index is associated with poor lactation outcomes among white, rural women independent of psychosocial and demographic correlates. J Hum Lactation. 2004;20(1):18–29.

Huang L, Chen X, Zhang Y, Sun G, Zhong C, Wang W, et al. Gestational weight gain is associated with delayed onset of lactogenesis in the TMCHC study: a prospective cohort study. Clin Nutr. 2019;38(5):2436–41.

Kong M, Bajorek B. Medications in pregnancy: impact on time to lactogenesis after parturition. J Pharm Pract Res. 2008;38(3):205–8.

Li J, Yu X, Wang Y, Liu W, Kong H. [The influencing factors of delayed onset of lactogenesis II in preterm parturient women separated from their infants]. Chin J Nurs Educ. 2022;19(4):368–73.

Lian W, Ding J, Xiong T, Liuding J, Nie L. Determinants of delayed onset of lactogenesis II among women who delivered via cesarean section at a tertiary hospital in China: a prospective cohort study. Int Breastfeed J. 2022;17:81.

Lu Y, Yin Y, Jiang H. [Influence of pregnancy stress and coping style on delayed onset of lactogenesis in maternal separation]. Chin Nurs Res. 2022;36(23):4148–53.

Luo F, Bao N, Li S. [Establishment and validation of a predictive model for the risk of delayed lactation initiation in pregnant women with gestational diabetes mellitus]. J Clin Pathological Res. 2020;40(6):1394–400.

Matias SL, Nommsen-Rivers LA, Creed-Kanashiro H, Dewey KG. Risk factors for early lactation problems among Peruvian primiparous mothers. Matern Child Nutr. 2010;6(2):120–33.

PubMed   Google Scholar  

Mullen AJ, O’Connor DL, Hanley AJ, Piedimonte G, Wallace M, Ley SH. Associations of metabolic and obstetric risk parameters with timing of lactogenesis II. Nutrients. 2022;14(4):876.

Otoo GE, Marquis GS, Sellen DW, Chapman DJ, Pérez-Escamilla R. HIV-negative status is associated with very early onset of lactation among Ghanaian women. J Hum Lactation. 2010;26(2):107–17.

Preusting I, Brumley J, Odibo L, Spatz DL, Louis JM. Obesity as a predictor of delayed lactogenesis II. J Hum Lactation. 2017;33(4):684–91.

Quan Y, Liu S, Zhou H. [First lactation time and delayed lactation of parturient in Zhumadian area]. South China J Prev Med. 2022;48(9):1037–40.

Rocha BO, Machado MP, Bastos LL, Barbosa Silva L, Santos AP, Santos LC, et al. Risk factors for delayed onset of lactogenesis II among primiparous mothers from a Brazilian baby-friendly hospital. J Hum Lactation. 2020;36(1):146–56.

Salahudeen MS, Koshy AM, Sen S. A study of the factors affecting time to onset of lactogenesis-II after parturition. J Pharm Res. 2013;6(1):68–72.

CAS   Google Scholar  

Si M. [Correlation between gestational diabetes mellitus (GDM) and delayed onset of lactogenesis (DOL) and risk factors of DOL among GDM puerperae] [Thesis]: Nanjing Medical University; 2017.

Wang S, Guo N, Jiang H. [Correlations of pre-pregnancy body mass index and gestational weight gain with delayed onset of lactogenesis]. Mod Clin Nurs. 2020;19(9):1–6.

Wei Z, Zhao Z, Meng T, Ye Q, Wan Y, Liu Y. [Occurrence and related factors of delayed lactation initiation of parturients in Tongzhou District, Beijing]. South China J Prev Med. 2022;48(7):817–21.

Xie X, Zhao M. [Current status of delayed onset of lactogenesis II of high-risk pregnant women in maternal intensive care unit and its influence factors: a 229-case study]. J Nursing(China). 2021;28(7):49–53.

Xu H. [A study on the correlation between delayed onset of lactogenesis and breastfeeding self-efficacy and analysis of the factors influencing them] [Thesis]: Soochow University; 2021.

Xue Y, Xu Q, Liu L, Liu Z, Huang X, Ma J. [Dietary intake and factors influencing delayed onset of lactation among postpartum women in Guangzhou]. South China J Prev Med. 2015;41(3):218–23.

Zhang L, Li Z, Wang F, Du Q. [Analysis of influencing factors of postpartum delayed onset of lactation in primiparas in family delivery room]. J Med Theory Pract. 2023;36(14):2483–6.

Zhang Y, Zhou H, Wang J, Zhang J, Cai Q. [Effects of pre-pregnancy body mass index, gestational weight gain and early feeding behavior on lactogenesis stage II: a prospective study]. Chin J Perinat Med. 2022;25(7):504–12.

Zhang Z, Liu J, Hu S, Jiang P, Liu M, Zhou Y. [Correlation between pregnancy women with hypothyroidism and delayed lactation initiation]. J Mod Med Health. 2022;38(15):2536–9.

Zhao J. [Effect of physical activity of pregnant women in the third trimester on the delayed onset of lactogenesis and exclusive breastfeeding] [Thesis]: Anhui Medical University; 2022.

Zhu P, Hao J, Jiang X, Huang K, Tao F. New insight into onset of lactation: mediating the negative effect of multiple perinatal biopsychosocial stress on breastfeeding duration. Breastfeed Med. 2013;8(2):151–8.

Institute of Medicine (US) and National Research Council (US) Committee to Reexamine IOM Pregnancy Weight Guidelines. Rasmussen KM, Yaktine AL, editors. Weight gain during pregnancy: reexamining the guidelines. Washington (DC): National Academies Press (US); 2009. Summary. https://www.ncbi.nlm.nih.gov/books/NBK32799/ . Accessed 25 Feb 2024.

[Circular on issuing the breastfeeding promotion action plan(. 2021-2025)]. Gazette of the National Health Commission of the People’s Republic of China. 2021(11):12 – 5.

Meek JY, Noble L. Policy statement: breastfeeding and the use of human milk. Pediatrics. 2022;150(1).

Patel S, Patel S. The effectiveness of lactation consultants and lactation counselors on breastfeeding outcomes. J Hum Lactation. 2016;32(3):530–41.

Obesity: preventing and managing the global epidemic. Report of a WHO consultation. World Health Organization Technical Report Series. 2000;894:i-xii, 1-253.

Zhou B. [Predictive values of body mass index and waist circumference to risk factors of related diseases in Chinese adult population]. Chin J Epidemiol. 2002;23(1):5–10.

Buonfiglio DC, Ramos-Lobo AM, Freitas VM, Zampieri TT, Nagaishi VS, Magalhães M, et al. Obesity impairs lactation performance in mice by inducing prolactin resistance. Sci Rep. 2016;6:22421.

Rasmussen KM, Kjolhede CL. Prepregnant overweight and obesity diminish the prolactin response to suckling in the first week postpartum. Pediatrics. 2004;113(5):e465–71.

Nommsen-Rivers LA. Does insulin explain the relation between maternal obesity and poor lactation outcomes? An overview of the literature. Adv Nutr. 2016;7(2):407–14.

De Bortoli J, Amir LH. Is onset of lactation delayed in women with diabetes in pregnancy? A systematic review. Diabet Med. 2016;33(1):17–24.

Powe CE, Allard C, Battista MC, Doyon M, Bouchard L, Ecker JL, et al. Heterogeneous contribution of insulin sensitivity and secretion defects to gestational diabetes mellitus. Diabetes Care. 2016;39(6):1052–5.

Lemay DG, Ballard OA, Hughes MA, Morrow AL, Horseman ND, Nommsen-Rivers LA. RNA sequencing of the human milk fat layer transcriptome reveals distinct gene expression profiles at three stages of lactation. PLoS ONE. 2013;8(7):e67531.

Nommsen-Rivers LA, Dolan LM, Huang B. Timing of stage II lactogenesis is predicted by antenatal metabolic health in a cohort of primiparas. Breastfeed Med. 2012;7(1):43–9.

Ramanjaneya M, Butler AE, Alkasem M, Bashir M, Jerobin J, Godwin A, et al. Association of complement-related proteins in subjects with and without second trimester gestational diabetes. Front Endocrinol. 2021;12:641361.

Leeners B, Rath W, Kuse S, Neumaier-Wagner P. Breast-feeding in women with hypertensive disorders in pregnancy. J Perinat Med. 2005;33(6):553–60.

Vest AR, Cho LS. Hypertension in pregnancy. Curr Atheroscler Rep. 2014;16(3):395.

Garrido-Gomez T, Quiñonero A, Dominguez F, Rubert L, Perales A, Hajjar KA, et al. Preeclampsia: a defect in decidualization is associated with deficiency of annexin A2. Am J Obstet Gynecol. 2020;222(4):376. e1-e17.

Saitou T, Watanabe K, Kinoshita H, Iwasaki A, Owaki Y, Matsushita H, et al. Hypoalbuminemia is related to endothelial dysfunction resulting from oxidative stress in parturients with preeclampsia. Nagoya J Med Sci. 2021;83(4):741–8.

CAS   PubMed   PubMed Central   Google Scholar  

Campo Verde Arboccó F, Sasso CV, Nasif DL, Hapon MB, Jahn GA. Effect of hypothyroidism on the expression of nuclear receptors and their co-regulators in mammary gland during lactation in the rat. Mol Cell Endocrinol. 2015;412:26–35.

Varas SM, Muñoz EM, Hapon MB, Aguilera Merlo CI, Giménez MS, Jahn GA. Hyperthyroidism and production of precocious involution in the mammary glands of lactating rats. Reproduction. 2002;124(5):691–702.

Chen DC, Nommsen-Rivers L, Dewey KG, Lönnerdal B. Stress during labor and delivery and early lactation performance. Am J Clin Nutr. 1998;68(2):335–44.

Dimitraki M, Tsikouras P, Manav B, Gioka T, Koutlaki N, Zervoudis S, et al. Evaluation of the effect of natural and emotional stress of labor on lactation and breast-feeding. Arch Gynecol Obstet. 2016;293(2):317–28.

Zuppa AA, Tornesello A, Papacci P, Tortorolo G, Segni G, Lafuenti G, et al. Relationship between maternal parity, basal prolactin levels and neonatal breast milk intake. Biol Neonate. 1988;53(3):144–7.

Doucet S, Soussignan R, Sagot P, Schaal B. An overlooked aspect of the human breast: areolar glands in relation with breastfeeding pattern, neonatal weight gain, and the dynamics of lactation. Early Hum Dev. 2012;88(2):119–28.

Zhang T, Liu M, Min F, Wei W, Liu Y, Tong J, et al. Fear of childbirth and its determinants in pregnant women in the third trimester: a cross-sectional study. BMC Psychiatry. 2023;23:574.

Nissen E, Uvnäs-Moberg K, Svensson K, Stock S, Widström AM, Winberg J. Different patterns of oxytocin, prolactin but not cortisol release during breastfeeding in women delivered by caesarean section or by the vaginal route. Early Hum Dev. 1996;45(1–2):103–18.

Wang Y, Tao F, Zhu P, Jiang X, Yao Y, Xu Y. [The level of serum prolactin, self-estimated milk yield and growth factor level in colostrum of pregnant women adopting different delivery modes]. Maternal Child Health Care China. 2010;25(10):1411–4.

Fu M, Zhang L, Ahmed A, Plaut K, Haas DM, Szucs K, et al. Does circadian disruption play a role in the metabolic-hormonal link to delayed lactogenesis II? Front Nutr. 2015;2:4.

Stuebe AM, Grewen K, Meltzer-Brody S. Association between maternal mood and oxytocin response to breastfeeding. J Women’s Health. 2013;22(4):352–61.

Zubaran C, Foresti K. The correlation between breastfeeding self-efficacy and maternal postpartum depression in southern Brazil. Sex Reproductive Healthc. 2013;4(1):9–15.

Khadka R, Hong SA, Chang YS. Prevalence and determinants of poor sleep quality and depression among postpartum women: a community-based study in Ramechhap district, Nepal. Int Health. 2020;12(2):125–31.

Okun ML, Mancuso RA, Hobel CJ, Schetter CD, Coussons-Read M. Poor sleep quality increases symptoms of depression and anxiety in postpartum women. J Behav Med. 2018;41(5):703–10.

Krebs NF, Belfort MB, Meier PP, Mennella JA, O’Connor DL, Taylor SN, et al. Infant factors that impact the ecology of human milk secretion and composition-a report from Breastmilk Ecology: Genesis of Infant Nutrition (BEGIN) Working Group 3. Am J Clin Nutr. 2023;117:S43–60.

Canals J, Fernández-Ballart J, Esparó G. Evolution of neonatal behavior assessment scale scores in the first month of life. Infant Behav Dev. 2003;26(2):227–37.

Rodriguez NA, Meier PP, Groer MW, Zeller JM. Oropharyngeal administration of colostrum to extremely low birth weight infants: theoretical perspectives. J Perinatol. 2009;29(1):1–7.

Garofoli F, Civardi E, Pisoni C, Angelini M, Ghirardello S. Anti-inflammatory and anti-allergic properties of colostrum from mothers of full-term and preterm babies: the importance of maternal lactation in the first days. Nutrients. 2023;15(19):4249.

Fu ZY, Huang C, Lei L, Chen LC, Wei LJ, Zhou J, et al. The effect of oropharyngeal colostrum administration on the clinical outcomes of premature infants: a meta-analysis. Int J Nurs Stud. 2023;144:104527.

Guideline: counselling of women to improve breastfeeding practices. Geneva: World Health Organization. 2018. Evidence and Recommendations. https://www.ncbi.nlm.nih.gov/books/NBK539310/ . Accessed 25 Feb 2024.

McFadden A, Siebelt L, Marshall JL, Gavine A, Girard LC, Symon A, et al. Counselling interventions to enable women to initiate and continue breastfeeding: a systematic review and meta-analysis. Int Breastfeed J. 2019;14:42.

Neville MC. Anatomy and physiology of lactation. Pediatr Clin North Am. 2001;48(1):13–34.

Weaver SR, Hernandez LL. Autocrine-paracrine regulation of the mammary gland. J Dairy Sci. 2016;99(1):842–53.

Truchet S, Honvo-Houéto E. Physiology of milk secretion. Best Pract Res Clin Endocrinol Metab. 2017;31(4):367–84.

Download references

Acknowledgements

Not applicable.

Author information

Yijuan Peng and Ke Zhuang contributed equally to this study and should be considered co-first authors.

Authors and Affiliations

Department of Nursing, West China Second University Hospital, Sichuan University/West China School of Nursing, Sichuan University, No. 20 Third Section, Renmin South Road, Chengdu, Sichuan Province, 610041, China

Yijuan Peng, Ke Zhuang & Yan Huang

Key Laboratory of Birth Defects and Related Diseases of Women and Children (Sichuan University), Ministry of Education, Chengdu, Sichuan, China

You can also search for this author in PubMed   Google Scholar

Contributions

YJP, KZ, and YH conceptualized and designed this review. YJP, KZ, and YH conducted the literature search, literature screening, data extraction, quality evaluation, and statistical analysis. YJP and KZ wrote the manuscript. YH reviewed and modified the manuscript. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to Yan Huang .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Peng, Y., Zhuang, K. & Huang, Y. Incidence and factors influencing delayed onset of lactation: a systematic review and meta-analysis. Int Breastfeed J 19 , 59 (2024). https://doi.org/10.1186/s13006-024-00666-5

Download citation

Received : 28 February 2024

Accepted : 16 August 2024

Published : 22 August 2024

DOI : https://doi.org/10.1186/s13006-024-00666-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Influencing factor
• Meta-analysis
• Systematic review

International Breastfeeding Journal

ISSN: 1746-4358

• General enquiries: [email protected]

• Brief Report
• Open access
• Published: 24 August 2024

The fentanyl made me feel like I needed more methadone”: changes in the role and use of medication for opioid use disorder (MOUD) due to fentanyl

• Maria Bolshakova 1 ,
• Kelsey A. Simpson 1 , 2 ,
• Siddhi S. Ganesh 1 ,
• Jesse L. Goldshear 1 , 2 ,
• Cheyenne J. Page 1 &
• Ricky N. Bluthenthal 1  

Harm Reduction Journal volume  21 , Article number:  156 ( 2024 ) Cite this article

276 Accesses

24 Altmetric

Metrics details

Fentanyl and fentanyl analogues have disrupted the illicit drug supply through contamination of other substances (i.e., methamphetamine and cocaine) and replacement of heroin in illicit markets. Increasingly, they are contributing to opioid-overdose related deaths. The rapid and growing presence of fentanyl has led to gaps in research on the impact of this illicit market change on people who use drugs (PWUD). We sought to examine how the changing opioid market and growing fentanyl availability influences the role and use of medication for opioid use disorder (MOUD).

Semi-structured qualitative interviews were conducted with a community recruited sample of PWUD (N = 22) in Los Angeles, California between September 2021 and April 2022. Interviews examined opioid use history, current opioid use behaviors and consumption patterns, and MOUD experiences and perceptions. Thematic analysis was used to systematically code and analyze textual interview data.

The following themes related to fentanyl use and MOUD emerged: (1) Use of deviated MOUD to address fentanyl contamination, (2) Changing perception of the effectiveness of MOUD on fentanyl, and (3) Regulatory limitations of MOUD for fentanyl use disorder.

Conclusions

PWUD described several repertoires for adjusting to changes in the illicit market of opioids. Clinicians treating PWUD should ask about recent fentanyl use prior to starting MOUD to account for increased tolerance to opioids. Harm reduction strategies such as naloxone kits, safe supply, and supervised consumption facilities can all prevent overdose deaths due to fentanyl.

Introduction

Illicitly manufactured fentanyl has become the dominant opioid in illicit markets [ 9 ]. Fentanyl containing opioids are increasingly responsible for opioid-overdose deaths [ 2 ]. Both intentional and unintentional fentanyl use can lead to increased overdose risk due to its higher potency [ 11 ]. Medication for opioid use disorder (MOUD) is a primary method of treatment, shown to decrease post-overdose mortality and reduce risk of relapse during treatment [ 10 , 14 ]. However, some literature indicates that the introduction of fentanyl into the US drug supply may necessitate changes in MOUD dosing and prescribing guidelines [ 12 ]. Fentanyl’s increased potency and shorter half-life may require a more aggressive methadone induction strategy [ 5 ].

To make MOUD easier to access in the United States, the Biden administration (via SAMHSA and Health and Human Services) has sought to lower methadone prescribing regulations and keep in place more flexible options offered during the height of the COVID-19 pandemic [ 16 ]. This change in regulations has taken effect in 2024 and allows for more take-home doses of methadone by patients. It also eliminates the minimum year-long length of opioid addiction that patients need to demonstrate before treatment can be provided. Significant barriers remain, however. Patients must still access methadone via authorized opioid treatment programs (OTP) and cannot pick up this medication at a local pharmacy.

In this qualitative brief report, we describe how the disruption of the drug market, and the resulting illicit opioid supply contamination and uptake of fentanyl affects perceptions and usage of MOUD among a population of people who use drugs (PWUD) in Los Angeles, California.

Participants and procedure

This study was conducted in conjunction with a larger prospective cohort study exploring cannabis use behaviors and opioid related health outcomes in people who inject drugs (PWID) residing in community settings in Los Angeles, California and Denver, Colorado. Inclusion criteria for the parent study were having injected drugs in the last 30 days, being 18 years of age or older, and having used opioids in the last 30 days. For the current study, participants were eligible to participate if they completed their initial baseline interview, resided in Los Angeles, and returned to the field site during data collection hours. Participants provided written informed consent and were renumerated $40 for their participation. Procedures were reviewed and approved by the Human Subjects Protection Committee at the University of Southern California.

Semi-structured interviews

In-depth qualitative interviews were conducted with a total of 22 PWUD between September 2021 and April 2022. All interviews occurred in-person and were facilitated by the two lead authors (KS and MB) at two field sites in Los Angeles: Boyle Heights and Hollywood. Face-to-face interviews employed a flexible, semi-structured format with open-ended questions that encouraged participants to spontaneously discuss new topics that arose during the conversation. Interview topics focused on opioid use behaviors and consumption habits, and experiences and viewpoints on MOUD (Supplemental Table  1 ). Duration of interviews varied, lasting between 30 and 75 min, with participant recruitment continuing until theoretical saturation was achieved.

Data analysis

All interview sessions were audio recorded and transcribed verbatim first using a digital transcription software program (Otter.ai), and then checked for accuracy by trained research staff. Transcripts were then imported onto NVivo (Version 12.5), where they were read in their entirety and analyzed thematically using an iterative, multi-step procedure. We first identified all quotations pertaining to MOUD experiences and current/previous opioid use behaviors. The team then reviewed and discussed these quotations, and then sifted and organized them by early themes or clusters of meaning to reduce the data toward a narrower focus. We continued to iteratively review and organize these quotations several different occasions during separate team meetings until consensus of themes was reached.

The analytic sample included a total of 22 participants, with a median age of 46.5 years (interquartile range [IQR] = 39–61; Table  1 ). Participants were mostly male (N = 14; 63.6%), and racially/ethnically diverse (35% White, 5% Black/African American, 9% Native American, 50% mixed race or another race/ethnicity, and 64% having Hispanic or Latino descent), with about 60% having a high school education or more. 36% of our sample (N = 8) reported being homeless or unstably housed in the past 3 months, and a large majority (72.3%) reporting a past 30-day income of less than $1400. Most participants had experiences using methadone (91%), and 32% reported past buprenorphine or suboxone use.

While qualitative interview questions targeted overall opioid consumption patterns, fentanyl quickly emerged as a primary response in participant narratives surrounding current opioid product use, frequency of use, route of administration, opioid withdrawal symptoms, and MOUD experiences and perceptions. Three primary themes emerged from semi-structured interviews: (1) Relying on methadone to alleviate overdose risk from fentanyl contaminated heroin; (2) Changing perceptions of the effectiveness of MOUD on fentanyl; and (3) Regulatory limitations of MOUD for fentanyl specific opioid use disorder.

Relying on methadone due to fentanyl contamination

Some participants indicated anxiety around fentanyl, fentanyl contamination, and its health effects, including potential overdose. Methadone, by contrast, was a known quantity to these participants and likely functioned as a proxy for safe supply of uncontaminated drugs.

For example, the following participant described an experience where he fell asleep for several hours after injecting opioids. Upon waking up he tested their heroin supply with fentanyl test strips and noted it was positive for fentanyl contamination. He then used methadone as it was safe, reliable in preventing withdrawal, and available and threw out the contaminated heroin:

“So when you go to a needle exchange, they give you these fentanyl testers. And I checked it with that fentanyl strip and yeah, it did have fentanyl in it. It wasn't fentanyl straight. It was fentanyl with heroin. It was heroin with fentanyl to cut it...So yeah, I just happened to ask my wife. Baby, you know, could I have some methadone? She had a bottle couple of bottles. So I drank half and I was alright till the sun came up and the day got born. I went out tossed out the dope and got well” (Male, 54 years)

Similarly, this participant described how she comes to the methadone clinic and relies on methadone due to fentanyl contamination of the illicit market heroin supply:

“So I do that. And then I come to them at the methadone [clinic] and I take methadone because this stuff is kinda getting bunk. So and then I’ve been real scared about the fentanyl?” (Female, 61 years)

Impacts of fentanyl on effectiveness of MOUD: methadone and buprenorphine

Despite the evolving role of methadone due to supply contamination noted in the first theme, many participants with experience of formal MOUD treatment discussed situations in which these medications were less effective—needing to use more MOUD to feel the expected physiological effects. They attributed this lack of effectiveness to the increased opioid tolerance brought on by fentanyl use.

“It [buprenorphine] didn’t take the edge off. It barely took the edge off. I mean I took five of them [buprenorphine pills]. And it still barely took the edge off, so I took five more. And it still barely did anything for me.” (Male, 45 years)

The added potency of fentanyl and subsequent feeling of needing higher doses of MOUD could be frustrating for participants in our study. In one case, an interviewee spoke to how this frustration resulted in withdrawal symptoms when he refused to increase his dose:

“The fentanyl made me feel like I needed more methadone, but I refused to give myself more methadone, so as for me to get back to the way it was before that, it took about a month, and I had symptoms like trouble sleeping, agitations.” (Male, 43 years)

Regulatory barriers to methadone access

Two major regulatory barriers to methadone uptake were reported: (1) dosing regulations that were non-specific for fentanyl and (2) inflexibility around accessing methadone clinics. For example, one participant described how MOUD dosing for fentanyl was inadequate for his withdrawal symptoms along with his preferred MOUD treatment plan:

“We should start the [buprenorphine] with Subutex. That would stop me. But see, the withdrawals from methadone are worse than the withdrawals from heroin. However, they’re not worse than withdrawals from fentanyl. And methadone, [HCPs/clinic preceptors] want to start with a low dosage. And that's just a waste of my fucking time. Give me 120 mg three times a day. Even though you’re only supposed to do it like twice a day. Because unbeknownst to [HCPs/clinic preceptors], my tolerance is through the fucking roof.” (Male, 45 years)

This participant went on to describe how barriers to MOUD included commuting to the methadone clinic, the inadequate dosing, and finally the clinic hours:

“I can’t do methadone... I have to go to the methadone clinic at their time and their hours and their thing ... ’we’re gonna start with this low dosage and then we’ll move you up as time goes by,’ you can kiss my ass. And I’m fucking smoking a [eight] ball [1/8th of an ounce] of fent a day.” (Male, 45 years)

Findings from our study highlight the important role of MOUD during an unpredictable and transitioning opioid drug supply [ 8 ]. When PWUD are unsure about using fentanyl-contaminated drugs, methadone may serve as a proxy to safe supply. Given this, it is especially necessary to make low threshold methadone access a priority during this public health emergency [ 8 ].

Our findings also conform with other research by highlighting changes in the effectiveness of MOUD due to fentanyl use, particularly buprenorphine. This may indicate a necessary revaluation of guidelines for initiating buprenorphine treatment for people who are using fentanyl [ 3 ]. A study analyzing Rhode Island statewide data between 2016 and 2020, when fentanyl was the dominant illicit opioid in the local market, reported that patients prescribed a 24 mg dose of buprenorphine showed higher treatment retention than those prescribed the current FDA recommendation of 16 mg. [ 6 ]. Prolonged-release subcutaneous buprenorphine injections such as Sublocade have been shown to achieve higher and steadier buprenorphine levels than sublingual products, offering a promising alternative treatment [ 7 ]. Nonetheless, our findings underscore the need for comprehensive prescription and dosing guidelines for MOUD specific to fentanyl that address the withdrawal and pain related symptomology reported by PWUD.

Our study also found that regulatory barriers inhibited access to methadone maintenance therapy (MMT). Extant literature has indicated that MMT reduces fentanyl overdose risk by relieving withdrawal symptoms and cravings, thus reducing fentanyl use and associated overdose risks [ 4 ]. Additionally, long-term MMT may create cross-tolerance to fentanyl’s respiratory depression and maintain opioid tolerance, reducing the risk of overdose from high potency exposure [ 4 ]. Participants in our study noted access barriers including dosing, clinic commute and timing. Recent studies have emphasized the need for low barrier methadone access such as local pharmacy dispensing [ 8 ]. Some evidence-based recommendations on methadone treatment for people who use fentanyl include more aggressive initiation dosing and slow-release oral morphine (SROM) [ 4 ]. Recently proposed legislation would continue to expand access to MOUD via dispensing strategies but would leave dosing limitations unaddressed (S.644—modernizing opioid treatment access act 118th). Our findings indicate that in the absence of heroin assisted treatment (HAT) and other structural safe supply interventions, these guidelines should be re-evaluated to center the perspectives of PWUD who are at high risk of overdose.

Study limitations include: (1) the Los Angeles location may not be generalizable to other MOUD treatment landscapes, (2) the ongoing COVID-19 pandemic may have influenced participant experiences, and (3) since interviews were conducted, psychostimulants have become more prominent in drug overdoses, which we do not address here.

This study contributes to the growing literature on fentanyl use and offers unique information on the evolving role, necessity, and implications of expanded MOUD access. Future research should investigate optimal MOUD dosing and low-barrier access strategies for people who use fentanyl. Federal policy should consider adopting a strategy of safe supply to reduce ongoing opioid overdose deaths.

Data availability

The data are available from the corresponding authors upon reasonable request.

Abadie R. “I don’t want to die”: a qualitative study of coping strategies to prevent fentanyl-related overdose deaths among people who inject drugs and its implications for harm reduction policies. Harm Reduct J. 2023;20(1):75. https://doi.org/10.1186/s12954-023-00805-x .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ahmad FB, Cisewski JA, Rossen LM, Sutton P. Provisional drug overdose death counts. Hyattsville: National Center for Health Statistics; 2022.

Google Scholar  

Antoine D, Huhn AS, Strain EC, et al. Method for successfully inducting individuals who use illicit fentanyl onto buprenorphine/naloxone. Am J Addict. 2021;30(1):83–7.

Article   PubMed   Google Scholar  

Bromley L, Kahan M, Regenstreif L, Srivastava A, Wyman J, Dabam F. Methadone treatment for people who use fentanyl: recommendations. Toronto: META: PHI. 2021;24.

Buresh M, Nahvi S, Steiger S, Weinstein ZM. Adapting methadone inductions to the fentanyl era. J Subst Abuse Treat. 2022;141:108832.

Article   CAS   PubMed   Google Scholar  

Chambers LC, Hallowell BD, Zullo AR, Paiva TJ, Berk J, Gaither R, Wightman RS. Buprenorphine dose and time to discontinuation among patients with opioid use disorder in the era of fentanyl. JAMA Netw Open. 2023;6(9):e2334540–e2334540.

Article   PubMed   PubMed Central   Google Scholar  

Chappuy M, Trojak B, Nubukpo P, Bachellier J, Bendimerad P, Brousse G, Rolland B. Prolonged-release buprenorphine formulations: perspectives for clinical practice. Therapies. 2020;75(5):397–406.

Article   Google Scholar  

Frank D, Elliott L, Cleland CM, Walters SM, Joudrey PJ, Russell DM, Meyerson BE, Bennett AS. “As safe as possible”: a qualitative study of opioid withdrawal and risk behavior among people who use illegal opioids. Harm Reduct J. 2023;20(1):158. https://doi.org/10.1186/s12954-023-00893-9 .

Friedman J, Shover CL. Charting the fourth wave: Geographic, temporal, race/ethnicity and demographic trends in polysubstance fentanyl overdose deaths in the United States, 2010–2021. Addiction (Abingdon Engl). 2023;118(12):2477–85. https://doi.org/10.1111/add.16318 .

Greiner MG, Shulman M, Choo TH, Scodes J, Pavlicova M, Campbell AN, Nunes EV. Naturalistic follow-up after a trial of medications for opioid use disorder: medication status, opioid use, and relapse. J Subst Abuse Treat. 2021;131:108447.

Lambdin BH, Bluthenthal RN, Zibbell JE, Wenger L, Simpson K, Kral AH. Associations between perceived illicit fentanyl use and infectious disease risks among people who inject drugs. Int J Drug Policy. 2019;74:299–304.

Morris NP. Opioid use disorder treatment in the age of fentanyl. JAMA Intern Med. 2022;182(3):249–50.

LaForge K, Stack E, Shin S, Pope J, Larsen JE, Leichtling G, Leahy JM, Seaman A, Hoover D, Byers M, Barrie C, Chisholm L, Korthuis PT. Knowledge, attitudes, and behaviors related to the fentanyl-adulterated drug supply among people who use drugs in Oregon. J Subst Abuse Treat. 2022;141:108849. https://doi.org/10.1016/j.jsat.2022.108849 .

Larochelle MR, Bernson D, Land T, Stopka TJ, Wang N, Xuan Z, Bagley SM, Liebschutz JM, Walley AY. Medication for opioid use disorder after nonfatal opioid overdose and association with mortality: a cohort study. Ann Intern Med. 2018;169(3):137–45. https://doi.org/10.7326/M17-3107 .

Sen. Markey EJ. Senate—health, education, labor, and pensions. S.644—modernizing opioid treatment access act 118th. 2023.

United States Government. Substance abuse and mental health services administration and department of health and human services. Medications for the treatment of opioid use disorder. (89 FR 7528). 2024.

Download references

Acknowledgements

We thank study participants for the time they devoted to this project and research assistants at the Bluthenthal Lab who contributed meaningfully to our data collection efforts.

Author information

Authors and affiliations.

Keck School of Medicine, University of Southern California, Los Angeles, CA, USA

Maria Bolshakova, Kelsey A. Simpson, Siddhi S. Ganesh, Jesse L. Goldshear, Cheyenne J. Page & Ricky N. Bluthenthal

University of California San Diego, La Jolla, CA, USA

Kelsey A. Simpson & Jesse L. Goldshear

You can also search for this author in PubMed   Google Scholar

Contributions

MB and KAS conceptualised the study, collected data, analysed the data, prepared the tables, edited, wrote, and reviewed the first draft of the manuscript. JLG and SSG edited, wrote, and reviewed the manuscript text. RNB is the PI. All authors reviewed the manuscript.

Corresponding author

Correspondence to Kelsey A. Simpson .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1 (docx 27 kb), rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Bolshakova, M., Simpson, K.A., Ganesh, S.S. et al. The fentanyl made me feel like I needed more methadone”: changes in the role and use of medication for opioid use disorder (MOUD) due to fentanyl. Harm Reduct J 21 , 156 (2024). https://doi.org/10.1186/s12954-024-01075-x

Download citation

Received : 28 June 2024

Accepted : 08 August 2024

Published : 24 August 2024

DOI : https://doi.org/10.1186/s12954-024-01075-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Harm Reduction Journal

ISSN: 1477-7517

• Submission enquiries: [email protected]