types of study design in research methodology pdf

• En español – ExME
• Em português – EME

An introduction to different types of study design

Posted on 6th April 2021 by Hadi Abbas

Study designs are the set of methods and procedures used to collect and analyze data in a study.

Broadly speaking, there are 2 types of study designs: descriptive studies and analytical studies.

Descriptive studies

• Describes specific characteristics in a population of interest
• The most common forms are case reports and case series
• In a case report, we discuss our experience with the patient’s symptoms, signs, diagnosis, and treatment
• In a case series, several patients with similar experiences are grouped.

Analytical Studies

Analytical studies are of 2 types: observational and experimental.

Observational studies are studies that we conduct without any intervention or experiment. In those studies, we purely observe the outcomes.  On the other hand, in experimental studies, we conduct experiments and interventions.

Observational studies

Observational studies include many subtypes. Below, I will discuss the most common designs.

Cross-sectional study:

• This design is transverse where we take a specific sample at a specific time without any follow-up
• It allows us to calculate the frequency of disease ( p revalence ) or the frequency of a risk factor
• This design is easy to conduct
• For example – if we want to know the prevalence of migraine in a population, we can conduct a cross-sectional study whereby we take a sample from the population and calculate the number of patients with migraine headaches.

Cohort study:

• We conduct this study by comparing two samples from the population: one sample with a risk factor while the other lacks this risk factor
• It shows us the risk of developing the disease in individuals with the risk factor compared to those without the risk factor ( RR = relative risk )
• Prospective : we follow the individuals in the future to know who will develop the disease
• Retrospective : we look to the past to know who developed the disease (e.g. using medical records)
• This design is the strongest among the observational studies
• For example – to find out the relative risk of developing chronic obstructive pulmonary disease (COPD) among smokers, we take a sample including smokers and non-smokers. Then, we calculate the number of individuals with COPD among both.

Case-Control Study:

• We conduct this study by comparing 2 groups: one group with the disease (cases) and another group without the disease (controls)
• This design is always retrospective
•  We aim to find out the odds of having a risk factor or an exposure if an individual has a specific disease (Odds ratio)
•  Relatively easy to conduct
• For example – we want to study the odds of being a smoker among hypertensive patients compared to normotensive ones. To do so, we choose a group of patients diagnosed with hypertension and another group that serves as the control (normal blood pressure). Then we study their smoking history to find out if there is a correlation.

Experimental Studies

• Also known as interventional studies
• Can involve animals and humans
• Pre-clinical trials involve animals
• Clinical trials are experimental studies involving humans
• In clinical trials, we study the effect of an intervention compared to another intervention or placebo. As an example, I have listed the four phases of a drug trial:

I:  We aim to assess the safety of the drug ( is it safe ? )

II: We aim to assess the efficacy of the drug ( does it work ? )

III: We want to know if this drug is better than the old treatment ( is it better ? )

IV: We follow-up to detect long-term side effects ( can it stay in the market ? )

• In randomized controlled trials, one group of participants receives the control, while the other receives the tested drug/intervention. Those studies are the best way to evaluate the efficacy of a treatment.

Finally, the figure below will help you with your understanding of different types of study designs.

References (pdf)

You may also be interested in the following blogs for further reading:

An introduction to randomized controlled trials

Case-control and cohort studies: a brief overview

Cohort studies: prospective and retrospective designs

Prevalence vs Incidence: what is the difference?

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

No Comments on An introduction to different types of study design

you are amazing one!! if I get you I’m working with you! I’m student from Ethiopian higher education. health sciences student

Very informative and easy understandable

You are my kind of doctor. Do not lose sight of your objective.

Wow very erll explained and easy to understand

I’m Khamisu Habibu community health officer student from Abubakar Tafawa Balewa university teaching hospital Bauchi, Nigeria, I really appreciate your write up and you have make it clear for the learner. thank you

well understood,thank you so much

Well understood…thanks

Simply explained. Thank You.

Thanks a lot for this nice informative article which help me to understand different study designs that I felt difficult before

That’s lovely to hear, Mona, thank you for letting the author know how useful this was. If there are any other particular topics you think would be useful to you, and are not already on the website, please do let us know.

it is very informative and useful.

thank you statistician

Fabulous to hear, thank you John.

Thanks for this information

Thanks so much for this information….I have clearly known the types of study design Thanks

That’s so good to hear, Mirembe, thank you for letting the author know.

Very helpful article!! U have simplified everything for easy understanding

I’m a health science major currently taking statistics for health care workers…this is a challenging class…thanks for the simified feedback.

That’s good to hear this has helped you. Hopefully you will find some of the other blogs useful too. If you see any topics that are missing from the website, please do let us know!

Hello. I liked your presentation, the fact that you ranked them clearly is very helpful to understand for people like me who is a novelist researcher. However, I was expecting to read much more about the Experimental studies. So please direct me if you already have or will one day. Thank you

Dear Ay. My sincere apologies for not responding to your comment sooner. You may find it useful to filter the blogs by the topic of ‘Study design and research methods’ – here is a link to that filter: https://s4be.cochrane.org/blog/topic/study-design/ This will cover more detail about experimental studies. Or have a look on our library page for further resources there – you’ll find that on the ‘Resources’ drop down from the home page.

However, if there are specific things you feel you would like to learn about experimental studies, that are missing from the website, it would be great if you could let me know too. Thank you, and best of luck. Emma

Great job Mr Hadi. I advise you to prepare and study for the Australian Medical Board Exams as soon as you finish your undergrad study in Lebanon. Good luck and hope we can meet sometime in the future. Regards ;)

You have give a good explaination of what am looking for. However, references am not sure of where to get them from.

Subscribe to our newsletter

You will receive our monthly newsletter and free access to Trip Premium.

Related Articles

Cluster Randomized Trials: Concepts

This blog summarizes the concepts of cluster randomization, and the logistical and statistical considerations while designing a cluster randomized controlled trial.

Expertise-based Randomized Controlled Trials

This blog summarizes the concepts of Expertise-based randomized controlled trials with a focus on the advantages and challenges associated with this type of study.

A well-designed cohort study can provide powerful results. This blog introduces prospective and retrospective cohort studies, discussing the advantages, disadvantages and use of these type of study designs.

Research Strategies and Methods

• First Online: 22 July 2021

Cite this chapter

• Paul Johannesson 3 &
• Erik Perjons 3  

3176 Accesses

Researchers have since centuries used research methods to support the creation of reliable knowledge based on empirical evidence and logical arguments. This chapter offers an overview of established research strategies and methods with a focus on empirical research in the social sciences. We discuss research strategies, such as experiment, survey, case study, ethnography, grounded theory, action research, and phenomenology. Research methods for data collection are also described, including questionnaires, interviews, focus groups, observations, and documents. Qualitative and quantitative methods for data analysis are discussed. Finally, the use of research strategies and methods within design science is investigated.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Bhattacherjee A (2012) Social science research: principles, methods, and practices, 2 edn. CreateSpace Independent Publishing Platform, Tampa, FL

Google Scholar  

Blake W (2012) Delphi complete works of William Blake, 2nd edn. Delphi Classics

Bradburn NM, Sudman S, Wansink B (2004) Asking questions: the definitive guide to questionnaire design – for market research, political polls, and social and health questionnaires, revised edition. Jossey-Bass, San Francisco

Bryman A (2016) Social research methods, 5th edn. Oxford University Press, Oxford

Charmaz K (2014) Constructing grounded theory, 2nd edn. SAGE Publications, Thousand Oaks, CA

Coghlan D (2019) Doing action research in your own organization, 5th edn. SAGE Publications, Thousand Oaks, CA

Creswell JW, Creswell JD (2017) Research design: qualitative, quantitative, and mixed methods approaches, 5th edn. SAGE Publications, Thousand Oaks, CA

MATH   Google Scholar  

Denscombe M (2017) The good research guide, 6th edn. Open University Press, London

Dey I (2003) Qualitative data analysis: a user friendly guide for social scientists. Routledge, London

Book   Google Scholar  

Fairclough N (2013) Critical discourse analysis: the critical study of language, 2 edn. Routledge, London

Field A, Hole GJ (2003) How to design and report experiments, 1st edn. Sage Publications, Thousand Oaks, CA

Fowler FJ (2013) Survey research methods, 5th edn. SAGE Publications, Thousand Oaks, CA

Glaser B, Strauss A (1999) The discovery of grounded theory: strategies for qualitative research. Routledge, London

Kemmis S, McTaggart R, Nixon R (2016) The action research planner: doing critical participatory action research, 1st edn. Springer, Berlin

Krippendorff K (2018) Content analysis: an introduction to its methodology, 4th edn. SAGE Publications, Thousand Oaks, CA

LeCompte MD, Schensul JJ (2010) Designing and conducting ethnographic research: an introduction, 2nd edn. AltaMira Press, Lanham, MD

McNiff J (2013) Action research: principles and practice, 3rd edn. Routledge, London

Oates BJ (2006) Researching information systems and computing. SAGE Publications, Thousand Oaks, CA

Peterson RA (2000) Constructing effective questionnaires. Sage Publications, Thousand Oaks

Prior L (2008) Repositioning documents in social research. Sociology 42(5):821–836

Article   Google Scholar  

Seidman I (2019) Interviewing as qualitative research: a guide for researchers in education and the social sciences, 5th edn. Teachers College Press, New York

Silverman D (2018) Doing qualitative research, 5th edn. SAGE Publications, Thousand Oaks, CA

Stephens L (2004) Advanced statistics demystified, 1st edn. McGraw-Hill Professional, New York

Urdan TC (2016) Statistics in plain English, 4th ed. Routledge, London

Yin RK (2017) Case study research and applications: design and methods, 6th edn. SAGE Publications, Thousand Oaks, CA

Download references

Author information

Authors and affiliations.

Stockholm University, Kista, Sweden

Paul Johannesson & Erik Perjons

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Johannesson, P., Perjons, E. (2021). Research Strategies and Methods. In: An Introduction to Design Science. Springer, Cham. https://doi.org/10.1007/978-3-030-78132-3_3

Download citation

DOI : https://doi.org/10.1007/978-3-030-78132-3_3

Published : 22 July 2021

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-78131-6

Online ISBN : 978-3-030-78132-3

eBook Packages : Computer Science Computer Science (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Indian J Anaesth
• v.60(9); 2016 Sep

Types of studies and research design

Mukul chandra kapoor.

Department of Anesthesiology, Max Smart Super Specialty Hospital, New Delhi, India

Medical research has evolved, from individual expert described opinions and techniques, to scientifically designed methodology-based studies. Evidence-based medicine (EBM) was established to re-evaluate medical facts and remove various myths in clinical practice. Research methodology is now protocol based with predefined steps. Studies were classified based on the method of collection and evaluation of data. Clinical study methodology now needs to comply to strict ethical, moral, truth, and transparency standards, ensuring that no conflict of interest is involved. A medical research pyramid has been designed to grade the quality of evidence and help physicians determine the value of the research. Randomised controlled trials (RCTs) have become gold standards for quality research. EBM now scales systemic reviews and meta-analyses at a level higher than RCTs to overcome deficiencies in the randomised trials due to errors in methodology and analyses.

INTRODUCTION

Expert opinion, experience, and authoritarian judgement were the norm in clinical medical practice. At scientific meetings, one often heard senior professionals emphatically expressing ‘In my experience,…… what I have said is correct!’ In 1981, articles published by Sackett et al . introduced ‘critical appraisal’ as they felt a need to teach methods of understanding scientific literature and its application at the bedside.[ 1 ] To improve clinical outcomes, clinical expertise must be complemented by the best external evidence.[ 2 ] Conversely, without clinical expertise, good external evidence may be used inappropriately [ Figure 1 ]. Practice gets outdated, if not updated with current evidence, depriving the clientele of the best available therapy.

Triad of evidence-based medicine

EVIDENCE-BASED MEDICINE

In 1971, in his book ‘Effectiveness and Efficiency’, Archibald Cochrane highlighted the lack of reliable evidence behind many accepted health-care interventions.[ 3 ] This triggered re-evaluation of many established ‘supposed’ scientific facts and awakened physicians to the need for evidence in medicine. Evidence-based medicine (EBM) thus evolved, which was defined as ‘the conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients.’[ 2 ]

The goal of EBM was scientific endowment to achieve consistency, efficiency, effectiveness, quality, safety, reduction in dilemma and limitation of idiosyncrasies in clinical practice.[ 4 ] EBM required the physician to diligently assess the therapy, make clinical adjustments using the best available external evidence, ensure awareness of current research and discover clinical pathways to ensure best patient outcomes.[ 5 ]

With widespread internet use, phenomenally large number of publications, training and media resources are available but determining the quality of this literature is difficult for a busy physician. Abstracts are available freely on the internet, but full-text articles require a subscription. To complicate issues, contradictory studies are published making decision-making difficult.[ 6 ] Publication bias, especially against negative studies, makes matters worse.

In 1993, the Cochrane Collaboration was founded by Ian Chalmers and others to create and disseminate up-to-date review of randomised controlled trials (RCTs) to help health-care professionals make informed decisions.[ 7 ] In 1995, the American College of Physicians and the British Medical Journal Publishing Group collaborated to publish the journal ‘Evidence-based medicine’, leading to the evolution of EBM in all spheres of medicine.

MEDICAL RESEARCH

Medical research needs to be conducted to increase knowledge about the human species, its social/natural environment and to combat disease/infirmity in humans. Research should be conducted in a manner conducive to and consistent with dignity and well-being of the participant; in a professional and transparent manner; and ensuring minimal risk.[ 8 ] Research thus must be subjected to careful evaluation at all stages, i.e., research design/experimentation; results and their implications; the objective of the research sought; anticipated benefits/dangers; potential uses/abuses of the experiment and its results; and on ensuring the safety of human life. Table 1 lists the principles any research should follow.[ 8 ]

General principles of medical research

Types of study design

Medical research is classified into primary and secondary research. Clinical/experimental studies are performed in primary research, whereas secondary research consolidates available studies as reviews, systematic reviews and meta-analyses. Three main areas in primary research are basic medical research, clinical research and epidemiological research [ Figure 2 ]. Basic research includes fundamental research in fields shown in Figure 2 . In almost all studies, at least one independent variable is varied, whereas the effects on the dependent variables are investigated. Clinical studies include observational studies and interventional studies and are subclassified as in Figure 2 .

Classification of types of medical research

Interventional clinical study is performed with the purpose of studying or demonstrating clinical or pharmacological properties of drugs/devices, their side effects and to establish their efficacy or safety. They also include studies in which surgical, physical or psychotherapeutic procedures are examined.[ 9 ] Studies on drugs/devices are subject to legal and ethical requirements including the Drug Controller General India (DCGI) directives. They require the approval of DCGI recognized Ethics Committee and must be performed in accordance with the rules of ‘Good Clinical Practice’.[ 10 ] Further details are available under ‘Methodology for research II’ section in this issue of IJA. In 2004, the World Health Organization advised registration of all clinical trials in a public registry. In India, the Clinical Trials Registry of India was launched in 2007 ( www.ctri.nic.in ). The International Committee of Medical Journal Editors (ICMJE) mandates its member journals to publish only registered trials.[ 11 ]

Observational clinical study is a study in which knowledge from treatment of persons with drugs is analysed using epidemiological methods. In these studies, the diagnosis, treatment and monitoring are performed exclusively according to medical practice and not according to a specified study protocol.[ 9 ] They are subclassified as per Figure 2 .

Epidemiological studies have two basic approaches, the interventional and observational. Clinicians are more familiar with interventional research, whereas epidemiologists usually perform observational research.

Interventional studies are experimental in character and are subdivided into field and group studies, for example, iodine supplementation of cooking salt to prevent hypothyroidism. Many interventions are unsuitable for RCTs, as the exposure may be harmful to the subjects.

Observational studies can be subdivided into cohort, case–control, cross-sectional and ecological studies.

• Cohort studies are suited to detect connections between exposure and development of disease. They are normally prospective studies of two healthy groups of subjects observed over time, in which one group is exposed to a specific substance, whereas the other is not. The occurrence of the disease can be determined in the two groups. Cohort studies can also be retrospective
• Case–control studies are retrospective analyses performed to establish the prevalence of a disease in two groups exposed to a factor or disease. The incidence rate cannot be calculated, and there is also a risk of selection bias and faulty recall.

Secondary research

Narrative review.

An expert senior author writes about a particular field, condition or treatment, including an overview, and this information is fortified by his experience. The article is in a narrative format. Its limitation is that one cannot tell whether recommendations are based on author's clinical experience, available literature and why some studies were given more emphasis. It can be biased, with selective citation of reports that reinforce the authors' views of a topic.[ 12 ]

Systematic review

Systematic reviews methodically and comprehensively identify studies focused on a specified topic, appraise their methodology, summate the results, identify key findings and reasons for differences across studies, and cite limitations of current knowledge.[ 13 ] They adhere to reproducible methods and recommended guidelines.[ 14 ] The methods used to compile data are explicit and transparent, allowing the reader to gauge the quality of the review and the potential for bias.[ 15 ]

A systematic review can be presented in text or graphic form. In graphic form, data of different trials can be plotted with the point estimate and 95% confidence interval for each study, presented on an individual line. A properly conducted systematic review presents the best available research evidence for a focused clinical question. The review team may obtain information, not available in the original reports, from the primary authors. This ensures that findings are consistent and generalisable across populations, environment, therapies and groups.[ 12 ] A systematic review attempts to reduce bias identification and studies selection for review, using a comprehensive search strategy and specifying inclusion criteria. The strength of a systematic review lies in the transparency of each phase and highlighting the merits of each decision made, while compiling information.

Meta-analysis

A review team compiles aggregate-level data in each primary study, and in some cases, data are solicited from each of the primary studies.[ 16 , 17 ] Although difficult to perform, individual patient meta-analyses offer advantages over aggregate-level analyses.[ 18 ] These mathematically pooled results are referred to as meta-analysis. Combining data from well-conducted primary studies provide a precise estimate of the “true effect.”[ 19 ] Pooling the samples of individual studies increases overall sample size, enhances statistical analysis power, reduces confidence interval and thereby improves statistical value.

The structured process of Cochrane Collaboration systematic reviews has contributed to the improvement of their quality. For the meta-analysis to be definitive, the primary RCTs should have been conducted methodically. When the existing studies have important scientific and methodological limitations, such as smaller sized samples, the systematic review may identify where gaps exist in the available literature.[ 20 ] RCTs and systematic review of several randomised trials are less likely to mislead us, and thereby help judge whether an intervention is better.[ 2 ] Practice guidelines supported by large RCTs and meta-analyses are considered as ‘gold standard’ in EBM. This issue of IJA is accompanied by an editorial on Importance of EBM on research and practice (Guyat and Sriganesh 471_16).[ 21 ] The EBM pyramid grading the value of different types of research studies is shown in Figure 3 .

The evidence-based medicine pyramid

In the last decade, a number of studies and guidelines brought about path-breaking changes in anaesthesiology and critical care. Some guidelines such as the ‘Surviving Sepsis Guidelines-2004’[ 22 ] were later found to be flawed and biased. A number of large RCTs were rejected as their findings were erroneous. Another classic example is that of ENIGMA-I (Evaluation of Nitrous oxide In the Gas Mixture for Anaesthesia)[ 23 ] which implicated nitrous oxide for poor outcomes, but ENIGMA-II[ 24 , 25 ] conducted later, by the same investigators, declared it as safe. The rise and fall of the ‘tight glucose control’ regimen was similar.[ 26 ]

Although RCTs are considered ‘gold standard’ in research, their status is at crossroads today. RCTs have conflicting interests and thus must be evaluated with careful scrutiny. EBM can promote evidence reflected in RCTs and meta-analyses. However, it cannot promulgate evidence not reflected in RCTs. Flawed RCTs and meta-analyses may bring forth erroneous recommendations. EBM thus should not be restricted to RCTs and meta-analyses but must involve tracking down the best external evidence to answer our clinical questions.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

• Open access
• Published: 07 September 2020

A tutorial on methodological studies: the what, when, how and why

• Lawrence Mbuagbaw   ORCID: orcid.org/0000-0001-5855-5461 1 , 2 , 3 ,
• Daeria O. Lawson 1 ,
• Livia Puljak 4 ,
• David B. Allison 5 &
• Lehana Thabane 1 , 2 , 6 , 7 , 8  

BMC Medical Research Methodology volume  20 , Article number:  226 ( 2020 ) Cite this article

42k Accesses

60 Citations

60 Altmetric

Metrics details

Methodological studies – studies that evaluate the design, analysis or reporting of other research-related reports – play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a “frequently asked questions” format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

Peer Review reports

The field of meta-research (or research-on-research) has proliferated in recent years in response to issues with research quality and conduct [ 1 , 2 , 3 ]. As the name suggests, this field targets issues with research design, conduct, analysis and reporting. Various types of research reports are often examined as the unit of analysis in these studies (e.g. abstracts, full manuscripts, trial registry entries). Like many other novel fields of research, meta-research has seen a proliferation of use before the development of reporting guidance. For example, this was the case with randomized trials for which risk of bias tools and reporting guidelines were only developed much later – after many trials had been published and noted to have limitations [ 4 , 5 ]; and for systematic reviews as well [ 6 , 7 , 8 ]. However, in the absence of formal guidance, studies that report on research differ substantially in how they are named, conducted and reported [ 9 , 10 ]. This creates challenges in identifying, summarizing and comparing them. In this tutorial paper, we will use the term methodological study to refer to any study that reports on the design, conduct, analysis or reporting of primary or secondary research-related reports (such as trial registry entries and conference abstracts).

In the past 10 years, there has been an increase in the use of terms related to methodological studies (based on records retrieved with a keyword search [in the title and abstract] for “methodological review” and “meta-epidemiological study” in PubMed up to December 2019), suggesting that these studies may be appearing more frequently in the literature. See Fig.  1 .

Trends in the number studies that mention “methodological review” or “meta-

epidemiological study” in PubMed.

The methods used in many methodological studies have been borrowed from systematic and scoping reviews. This practice has influenced the direction of the field, with many methodological studies including searches of electronic databases, screening of records, duplicate data extraction and assessments of risk of bias in the included studies. However, the research questions posed in methodological studies do not always require the approaches listed above, and guidance is needed on when and how to apply these methods to a methodological study. Even though methodological studies can be conducted on qualitative or mixed methods research, this paper focuses on and draws examples exclusively from quantitative research.

The objectives of this paper are to provide some insights on how to conduct methodological studies so that there is greater consistency between the research questions posed, and the design, analysis and reporting of findings. We provide multiple examples to illustrate concepts and a proposed framework for categorizing methodological studies in quantitative research.

What is a methodological study?

Any study that describes or analyzes methods (design, conduct, analysis or reporting) in published (or unpublished) literature is a methodological study. Consequently, the scope of methodological studies is quite extensive and includes, but is not limited to, topics as diverse as: research question formulation [ 11 ]; adherence to reporting guidelines [ 12 , 13 , 14 ] and consistency in reporting [ 15 ]; approaches to study analysis [ 16 ]; investigating the credibility of analyses [ 17 ]; and studies that synthesize these methodological studies [ 18 ]. While the nomenclature of methodological studies is not uniform, the intents and purposes of these studies remain fairly consistent – to describe or analyze methods in primary or secondary studies. As such, methodological studies may also be classified as a subtype of observational studies.

Parallel to this are experimental studies that compare different methods. Even though they play an important role in informing optimal research methods, experimental methodological studies are beyond the scope of this paper. Examples of such studies include the randomized trials by Buscemi et al., comparing single data extraction to double data extraction [ 19 ], and Carrasco-Labra et al., comparing approaches to presenting findings in Grading of Recommendations, Assessment, Development and Evaluations (GRADE) summary of findings tables [ 20 ]. In these studies, the unit of analysis is the person or groups of individuals applying the methods. We also direct readers to the Studies Within a Trial (SWAT) and Studies Within a Review (SWAR) programme operated through the Hub for Trials Methodology Research, for further reading as a potential useful resource for these types of experimental studies [ 21 ]. Lastly, this paper is not meant to inform the conduct of research using computational simulation and mathematical modeling for which some guidance already exists [ 22 ], or studies on the development of methods using consensus-based approaches.

When should we conduct a methodological study?

Methodological studies occupy a unique niche in health research that allows them to inform methodological advances. Methodological studies should also be conducted as pre-cursors to reporting guideline development, as they provide an opportunity to understand current practices, and help to identify the need for guidance and gaps in methodological or reporting quality. For example, the development of the popular Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) guidelines were preceded by methodological studies identifying poor reporting practices [ 23 , 24 ]. In these instances, after the reporting guidelines are published, methodological studies can also be used to monitor uptake of the guidelines.

These studies can also be conducted to inform the state of the art for design, analysis and reporting practices across different types of health research fields, with the aim of improving research practices, and preventing or reducing research waste. For example, Samaan et al. conducted a scoping review of adherence to different reporting guidelines in health care literature [ 18 ]. Methodological studies can also be used to determine the factors associated with reporting practices. For example, Abbade et al. investigated journal characteristics associated with the use of the Participants, Intervention, Comparison, Outcome, Timeframe (PICOT) format in framing research questions in trials of venous ulcer disease [ 11 ].

How often are methodological studies conducted?

There is no clear answer to this question. Based on a search of PubMed, the use of related terms (“methodological review” and “meta-epidemiological study”) – and therefore, the number of methodological studies – is on the rise. However, many other terms are used to describe methodological studies. There are also many studies that explore design, conduct, analysis or reporting of research reports, but that do not use any specific terms to describe or label their study design in terms of “methodology”. This diversity in nomenclature makes a census of methodological studies elusive. Appropriate terminology and key words for methodological studies are needed to facilitate improved accessibility for end-users.

Why do we conduct methodological studies?

Methodological studies provide information on the design, conduct, analysis or reporting of primary and secondary research and can be used to appraise quality, quantity, completeness, accuracy and consistency of health research. These issues can be explored in specific fields, journals, databases, geographical regions and time periods. For example, Areia et al. explored the quality of reporting of endoscopic diagnostic studies in gastroenterology [ 25 ]; Knol et al. investigated the reporting of p -values in baseline tables in randomized trial published in high impact journals [ 26 ]; Chen et al. describe adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement in Chinese Journals [ 27 ]; and Hopewell et al. describe the effect of editors’ implementation of CONSORT guidelines on reporting of abstracts over time [ 28 ]. Methodological studies provide useful information to researchers, clinicians, editors, publishers and users of health literature. As a result, these studies have been at the cornerstone of important methodological developments in the past two decades and have informed the development of many health research guidelines including the highly cited CONSORT statement [ 5 ].

Where can we find methodological studies?

Methodological studies can be found in most common biomedical bibliographic databases (e.g. Embase, MEDLINE, PubMed, Web of Science). However, the biggest caveat is that methodological studies are hard to identify in the literature due to the wide variety of names used and the lack of comprehensive databases dedicated to them. A handful can be found in the Cochrane Library as “Cochrane Methodology Reviews”, but these studies only cover methodological issues related to systematic reviews. Previous attempts to catalogue all empirical studies of methods used in reviews were abandoned 10 years ago [ 29 ]. In other databases, a variety of search terms may be applied with different levels of sensitivity and specificity.

Some frequently asked questions about methodological studies

In this section, we have outlined responses to questions that might help inform the conduct of methodological studies.

Q: How should I select research reports for my methodological study?

A: Selection of research reports for a methodological study depends on the research question and eligibility criteria. Once a clear research question is set and the nature of literature one desires to review is known, one can then begin the selection process. Selection may begin with a broad search, especially if the eligibility criteria are not apparent. For example, a methodological study of Cochrane Reviews of HIV would not require a complex search as all eligible studies can easily be retrieved from the Cochrane Library after checking a few boxes [ 30 ]. On the other hand, a methodological study of subgroup analyses in trials of gastrointestinal oncology would require a search to find such trials, and further screening to identify trials that conducted a subgroup analysis [ 31 ].

The strategies used for identifying participants in observational studies can apply here. One may use a systematic search to identify all eligible studies. If the number of eligible studies is unmanageable, a random sample of articles can be expected to provide comparable results if it is sufficiently large [ 32 ]. For example, Wilson et al. used a random sample of trials from the Cochrane Stroke Group’s Trial Register to investigate completeness of reporting [ 33 ]. It is possible that a simple random sample would lead to underrepresentation of units (i.e. research reports) that are smaller in number. This is relevant if the investigators wish to compare multiple groups but have too few units in one group. In this case a stratified sample would help to create equal groups. For example, in a methodological study comparing Cochrane and non-Cochrane reviews, Kahale et al. drew random samples from both groups [ 34 ]. Alternatively, systematic or purposeful sampling strategies can be used and we encourage researchers to justify their selected approaches based on the study objective.

Q: How many databases should I search?

A: The number of databases one should search would depend on the approach to sampling, which can include targeting the entire “population” of interest or a sample of that population. If you are interested in including the entire target population for your research question, or drawing a random or systematic sample from it, then a comprehensive and exhaustive search for relevant articles is required. In this case, we recommend using systematic approaches for searching electronic databases (i.e. at least 2 databases with a replicable and time stamped search strategy). The results of your search will constitute a sampling frame from which eligible studies can be drawn.

Alternatively, if your approach to sampling is purposeful, then we recommend targeting the database(s) or data sources (e.g. journals, registries) that include the information you need. For example, if you are conducting a methodological study of high impact journals in plastic surgery and they are all indexed in PubMed, you likely do not need to search any other databases. You may also have a comprehensive list of all journals of interest and can approach your search using the journal names in your database search (or by accessing the journal archives directly from the journal’s website). Even though one could also search journals’ web pages directly, using a database such as PubMed has multiple advantages, such as the use of filters, so the search can be narrowed down to a certain period, or study types of interest. Furthermore, individual journals’ web sites may have different search functionalities, which do not necessarily yield a consistent output.

Q: Should I publish a protocol for my methodological study?

A: A protocol is a description of intended research methods. Currently, only protocols for clinical trials require registration [ 35 ]. Protocols for systematic reviews are encouraged but no formal recommendation exists. The scientific community welcomes the publication of protocols because they help protect against selective outcome reporting, the use of post hoc methodologies to embellish results, and to help avoid duplication of efforts [ 36 ]. While the latter two risks exist in methodological research, the negative consequences may be substantially less than for clinical outcomes. In a sample of 31 methodological studies, 7 (22.6%) referenced a published protocol [ 9 ]. In the Cochrane Library, there are 15 protocols for methodological reviews (21 July 2020). This suggests that publishing protocols for methodological studies is not uncommon.

Authors can consider publishing their study protocol in a scholarly journal as a manuscript. Advantages of such publication include obtaining peer-review feedback about the planned study, and easy retrieval by searching databases such as PubMed. The disadvantages in trying to publish protocols includes delays associated with manuscript handling and peer review, as well as costs, as few journals publish study protocols, and those journals mostly charge article-processing fees [ 37 ]. Authors who would like to make their protocol publicly available without publishing it in scholarly journals, could deposit their study protocols in publicly available repositories, such as the Open Science Framework ( https://osf.io/ ).

Q: How to appraise the quality of a methodological study?

A: To date, there is no published tool for appraising the risk of bias in a methodological study, but in principle, a methodological study could be considered as a type of observational study. Therefore, during conduct or appraisal, care should be taken to avoid the biases common in observational studies [ 38 ]. These biases include selection bias, comparability of groups, and ascertainment of exposure or outcome. In other words, to generate a representative sample, a comprehensive reproducible search may be necessary to build a sampling frame. Additionally, random sampling may be necessary to ensure that all the included research reports have the same probability of being selected, and the screening and selection processes should be transparent and reproducible. To ensure that the groups compared are similar in all characteristics, matching, random sampling or stratified sampling can be used. Statistical adjustments for between-group differences can also be applied at the analysis stage. Finally, duplicate data extraction can reduce errors in assessment of exposures or outcomes.

Q: Should I justify a sample size?

A: In all instances where one is not using the target population (i.e. the group to which inferences from the research report are directed) [ 39 ], a sample size justification is good practice. The sample size justification may take the form of a description of what is expected to be achieved with the number of articles selected, or a formal sample size estimation that outlines the number of articles required to answer the research question with a certain precision and power. Sample size justifications in methodological studies are reasonable in the following instances:

Comparing two groups

Determining a proportion, mean or another quantifier

Determining factors associated with an outcome using regression-based analyses

For example, El Dib et al. computed a sample size requirement for a methodological study of diagnostic strategies in randomized trials, based on a confidence interval approach [ 40 ].

Q: What should I call my study?

A: Other terms which have been used to describe/label methodological studies include “ methodological review ”, “methodological survey” , “meta-epidemiological study” , “systematic review” , “systematic survey”, “meta-research”, “research-on-research” and many others. We recommend that the study nomenclature be clear, unambiguous, informative and allow for appropriate indexing. Methodological study nomenclature that should be avoided includes “ systematic review” – as this will likely be confused with a systematic review of a clinical question. “ Systematic survey” may also lead to confusion about whether the survey was systematic (i.e. using a preplanned methodology) or a survey using “ systematic” sampling (i.e. a sampling approach using specific intervals to determine who is selected) [ 32 ]. Any of the above meanings of the words “ systematic” may be true for methodological studies and could be potentially misleading. “ Meta-epidemiological study” is ideal for indexing, but not very informative as it describes an entire field. The term “ review ” may point towards an appraisal or “review” of the design, conduct, analysis or reporting (or methodological components) of the targeted research reports, yet it has also been used to describe narrative reviews [ 41 , 42 ]. The term “ survey ” is also in line with the approaches used in many methodological studies [ 9 ], and would be indicative of the sampling procedures of this study design. However, in the absence of guidelines on nomenclature, the term “ methodological study ” is broad enough to capture most of the scenarios of such studies.

Q: Should I account for clustering in my methodological study?

A: Data from methodological studies are often clustered. For example, articles coming from a specific source may have different reporting standards (e.g. the Cochrane Library). Articles within the same journal may be similar due to editorial practices and policies, reporting requirements and endorsement of guidelines. There is emerging evidence that these are real concerns that should be accounted for in analyses [ 43 ]. Some cluster variables are described in the section: “ What variables are relevant to methodological studies?”

A variety of modelling approaches can be used to account for correlated data, including the use of marginal, fixed or mixed effects regression models with appropriate computation of standard errors [ 44 ]. For example, Kosa et al. used generalized estimation equations to account for correlation of articles within journals [ 15 ]. Not accounting for clustering could lead to incorrect p -values, unduly narrow confidence intervals, and biased estimates [ 45 ].

Q: Should I extract data in duplicate?

A: Yes. Duplicate data extraction takes more time but results in less errors [ 19 ]. Data extraction errors in turn affect the effect estimate [ 46 ], and therefore should be mitigated. Duplicate data extraction should be considered in the absence of other approaches to minimize extraction errors. However, much like systematic reviews, this area will likely see rapid new advances with machine learning and natural language processing technologies to support researchers with screening and data extraction [ 47 , 48 ]. However, experience plays an important role in the quality of extracted data and inexperienced extractors should be paired with experienced extractors [ 46 , 49 ].

Q: Should I assess the risk of bias of research reports included in my methodological study?

A : Risk of bias is most useful in determining the certainty that can be placed in the effect measure from a study. In methodological studies, risk of bias may not serve the purpose of determining the trustworthiness of results, as effect measures are often not the primary goal of methodological studies. Determining risk of bias in methodological studies is likely a practice borrowed from systematic review methodology, but whose intrinsic value is not obvious in methodological studies. When it is part of the research question, investigators often focus on one aspect of risk of bias. For example, Speich investigated how blinding was reported in surgical trials [ 50 ], and Abraha et al., investigated the application of intention-to-treat analyses in systematic reviews and trials [ 51 ].

Q: What variables are relevant to methodological studies?

A: There is empirical evidence that certain variables may inform the findings in a methodological study. We outline some of these and provide a brief overview below:

Country: Countries and regions differ in their research cultures, and the resources available to conduct research. Therefore, it is reasonable to believe that there may be differences in methodological features across countries. Methodological studies have reported loco-regional differences in reporting quality [ 52 , 53 ]. This may also be related to challenges non-English speakers face in publishing papers in English.

Authors’ expertise: The inclusion of authors with expertise in research methodology, biostatistics, and scientific writing is likely to influence the end-product. Oltean et al. found that among randomized trials in orthopaedic surgery, the use of analyses that accounted for clustering was more likely when specialists (e.g. statistician, epidemiologist or clinical trials methodologist) were included on the study team [ 54 ]. Fleming et al. found that including methodologists in the review team was associated with appropriate use of reporting guidelines [ 55 ].

Source of funding and conflicts of interest: Some studies have found that funded studies report better [ 56 , 57 ], while others do not [ 53 , 58 ]. The presence of funding would indicate the availability of resources deployed to ensure optimal design, conduct, analysis and reporting. However, the source of funding may introduce conflicts of interest and warrant assessment. For example, Kaiser et al. investigated the effect of industry funding on obesity or nutrition randomized trials and found that reporting quality was similar [ 59 ]. Thomas et al. looked at reporting quality of long-term weight loss trials and found that industry funded studies were better [ 60 ]. Kan et al. examined the association between industry funding and “positive trials” (trials reporting a significant intervention effect) and found that industry funding was highly predictive of a positive trial [ 61 ]. This finding is similar to that of a recent Cochrane Methodology Review by Hansen et al. [ 62 ]

Journal characteristics: Certain journals’ characteristics may influence the study design, analysis or reporting. Characteristics such as journal endorsement of guidelines [ 63 , 64 ], and Journal Impact Factor (JIF) have been shown to be associated with reporting [ 63 , 65 , 66 , 67 ].

Study size (sample size/number of sites): Some studies have shown that reporting is better in larger studies [ 53 , 56 , 58 ].

Year of publication: It is reasonable to assume that design, conduct, analysis and reporting of research will change over time. Many studies have demonstrated improvements in reporting over time or after the publication of reporting guidelines [ 68 , 69 ].

Type of intervention: In a methodological study of reporting quality of weight loss intervention studies, Thabane et al. found that trials of pharmacologic interventions were reported better than trials of non-pharmacologic interventions [ 70 ].

Interactions between variables: Complex interactions between the previously listed variables are possible. High income countries with more resources may be more likely to conduct larger studies and incorporate a variety of experts. Authors in certain countries may prefer certain journals, and journal endorsement of guidelines and editorial policies may change over time.

Q: Should I focus only on high impact journals?

A: Investigators may choose to investigate only high impact journals because they are more likely to influence practice and policy, or because they assume that methodological standards would be higher. However, the JIF may severely limit the scope of articles included and may skew the sample towards articles with positive findings. The generalizability and applicability of findings from a handful of journals must be examined carefully, especially since the JIF varies over time. Even among journals that are all “high impact”, variations exist in methodological standards.

Q: Can I conduct a methodological study of qualitative research?

A: Yes. Even though a lot of methodological research has been conducted in the quantitative research field, methodological studies of qualitative studies are feasible. Certain databases that catalogue qualitative research including the Cumulative Index to Nursing & Allied Health Literature (CINAHL) have defined subject headings that are specific to methodological research (e.g. “research methodology”). Alternatively, one could also conduct a qualitative methodological review; that is, use qualitative approaches to synthesize methodological issues in qualitative studies.

Q: What reporting guidelines should I use for my methodological study?

A: There is no guideline that covers the entire scope of methodological studies. One adaptation of the PRISMA guidelines has been published, which works well for studies that aim to use the entire target population of research reports [ 71 ]. However, it is not widely used (40 citations in 2 years as of 09 December 2019), and methodological studies that are designed as cross-sectional or before-after studies require a more fit-for purpose guideline. A more encompassing reporting guideline for a broad range of methodological studies is currently under development [ 72 ]. However, in the absence of formal guidance, the requirements for scientific reporting should be respected, and authors of methodological studies should focus on transparency and reproducibility.

Q: What are the potential threats to validity and how can I avoid them?

A: Methodological studies may be compromised by a lack of internal or external validity. The main threats to internal validity in methodological studies are selection and confounding bias. Investigators must ensure that the methods used to select articles does not make them differ systematically from the set of articles to which they would like to make inferences. For example, attempting to make extrapolations to all journals after analyzing high-impact journals would be misleading.

Many factors (confounders) may distort the association between the exposure and outcome if the included research reports differ with respect to these factors [ 73 ]. For example, when examining the association between source of funding and completeness of reporting, it may be necessary to account for journals that endorse the guidelines. Confounding bias can be addressed by restriction, matching and statistical adjustment [ 73 ]. Restriction appears to be the method of choice for many investigators who choose to include only high impact journals or articles in a specific field. For example, Knol et al. examined the reporting of p -values in baseline tables of high impact journals [ 26 ]. Matching is also sometimes used. In the methodological study of non-randomized interventional studies of elective ventral hernia repair, Parker et al. matched prospective studies with retrospective studies and compared reporting standards [ 74 ]. Some other methodological studies use statistical adjustments. For example, Zhang et al. used regression techniques to determine the factors associated with missing participant data in trials [ 16 ].

With regard to external validity, researchers interested in conducting methodological studies must consider how generalizable or applicable their findings are. This should tie in closely with the research question and should be explicit. For example. Findings from methodological studies on trials published in high impact cardiology journals cannot be assumed to be applicable to trials in other fields. However, investigators must ensure that their sample truly represents the target sample either by a) conducting a comprehensive and exhaustive search, or b) using an appropriate and justified, randomly selected sample of research reports.

Even applicability to high impact journals may vary based on the investigators’ definition, and over time. For example, for high impact journals in the field of general medicine, Bouwmeester et al. included the Annals of Internal Medicine (AIM), BMJ, the Journal of the American Medical Association (JAMA), Lancet, the New England Journal of Medicine (NEJM), and PLoS Medicine ( n  = 6) [ 75 ]. In contrast, the high impact journals selected in the methodological study by Schiller et al. were BMJ, JAMA, Lancet, and NEJM ( n  = 4) [ 76 ]. Another methodological study by Kosa et al. included AIM, BMJ, JAMA, Lancet and NEJM ( n  = 5). In the methodological study by Thabut et al., journals with a JIF greater than 5 were considered to be high impact. Riado Minguez et al. used first quartile journals in the Journal Citation Reports (JCR) for a specific year to determine “high impact” [ 77 ]. Ultimately, the definition of high impact will be based on the number of journals the investigators are willing to include, the year of impact and the JIF cut-off [ 78 ]. We acknowledge that the term “generalizability” may apply differently for methodological studies, especially when in many instances it is possible to include the entire target population in the sample studied.

Finally, methodological studies are not exempt from information bias which may stem from discrepancies in the included research reports [ 79 ], errors in data extraction, or inappropriate interpretation of the information extracted. Likewise, publication bias may also be a concern in methodological studies, but such concepts have not yet been explored.

A proposed framework

In order to inform discussions about methodological studies, the development of guidance for what should be reported, we have outlined some key features of methodological studies that can be used to classify them. For each of the categories outlined below, we provide an example. In our experience, the choice of approach to completing a methodological study can be informed by asking the following four questions:

What is the aim?

Methodological studies that investigate bias

A methodological study may be focused on exploring sources of bias in primary or secondary studies (meta-bias), or how bias is analyzed. We have taken care to distinguish bias (i.e. systematic deviations from the truth irrespective of the source) from reporting quality or completeness (i.e. not adhering to a specific reporting guideline or norm). An example of where this distinction would be important is in the case of a randomized trial with no blinding. This study (depending on the nature of the intervention) would be at risk of performance bias. However, if the authors report that their study was not blinded, they would have reported adequately. In fact, some methodological studies attempt to capture both “quality of conduct” and “quality of reporting”, such as Richie et al., who reported on the risk of bias in randomized trials of pharmacy practice interventions [ 80 ]. Babic et al. investigated how risk of bias was used to inform sensitivity analyses in Cochrane reviews [ 81 ]. Further, biases related to choice of outcomes can also be explored. For example, Tan et al investigated differences in treatment effect size based on the outcome reported [ 82 ].

Methodological studies that investigate quality (or completeness) of reporting

Methodological studies may report quality of reporting against a reporting checklist (i.e. adherence to guidelines) or against expected norms. For example, Croituro et al. report on the quality of reporting in systematic reviews published in dermatology journals based on their adherence to the PRISMA statement [ 83 ], and Khan et al. described the quality of reporting of harms in randomized controlled trials published in high impact cardiovascular journals based on the CONSORT extension for harms [ 84 ]. Other methodological studies investigate reporting of certain features of interest that may not be part of formally published checklists or guidelines. For example, Mbuagbaw et al. described how often the implications for research are elaborated using the Evidence, Participants, Intervention, Comparison, Outcome, Timeframe (EPICOT) format [ 30 ].

Methodological studies that investigate the consistency of reporting

Sometimes investigators may be interested in how consistent reports of the same research are, as it is expected that there should be consistency between: conference abstracts and published manuscripts; manuscript abstracts and manuscript main text; and trial registration and published manuscript. For example, Rosmarakis et al. investigated consistency between conference abstracts and full text manuscripts [ 85 ].

Methodological studies that investigate factors associated with reporting

In addition to identifying issues with reporting in primary and secondary studies, authors of methodological studies may be interested in determining the factors that are associated with certain reporting practices. Many methodological studies incorporate this, albeit as a secondary outcome. For example, Farrokhyar et al. investigated the factors associated with reporting quality in randomized trials of coronary artery bypass grafting surgery [ 53 ].

Methodological studies that investigate methods

Methodological studies may also be used to describe methods or compare methods, and the factors associated with methods. Muller et al. described the methods used for systematic reviews and meta-analyses of observational studies [ 86 ].

Methodological studies that summarize other methodological studies

Some methodological studies synthesize results from other methodological studies. For example, Li et al. conducted a scoping review of methodological reviews that investigated consistency between full text and abstracts in primary biomedical research [ 87 ].

Methodological studies that investigate nomenclature and terminology

Some methodological studies may investigate the use of names and terms in health research. For example, Martinic et al. investigated the definitions of systematic reviews used in overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks [ 88 ].

Other types of methodological studies

In addition to the previously mentioned experimental methodological studies, there may exist other types of methodological studies not captured here.

What is the design?

Methodological studies that are descriptive

Most methodological studies are purely descriptive and report their findings as counts (percent) and means (standard deviation) or medians (interquartile range). For example, Mbuagbaw et al. described the reporting of research recommendations in Cochrane HIV systematic reviews [ 30 ]. Gohari et al. described the quality of reporting of randomized trials in diabetes in Iran [ 12 ].

Methodological studies that are analytical

Some methodological studies are analytical wherein “analytical studies identify and quantify associations, test hypotheses, identify causes and determine whether an association exists between variables, such as between an exposure and a disease.” [ 89 ] In the case of methodological studies all these investigations are possible. For example, Kosa et al. investigated the association between agreement in primary outcome from trial registry to published manuscript and study covariates. They found that larger and more recent studies were more likely to have agreement [ 15 ]. Tricco et al. compared the conclusion statements from Cochrane and non-Cochrane systematic reviews with a meta-analysis of the primary outcome and found that non-Cochrane reviews were more likely to report positive findings. These results are a test of the null hypothesis that the proportions of Cochrane and non-Cochrane reviews that report positive results are equal [ 90 ].

What is the sampling strategy?

Methodological studies that include the target population

Methodological reviews with narrow research questions may be able to include the entire target population. For example, in the methodological study of Cochrane HIV systematic reviews, Mbuagbaw et al. included all of the available studies ( n  = 103) [ 30 ].

Methodological studies that include a sample of the target population

Many methodological studies use random samples of the target population [ 33 , 91 , 92 ]. Alternatively, purposeful sampling may be used, limiting the sample to a subset of research-related reports published within a certain time period, or in journals with a certain ranking or on a topic. Systematic sampling can also be used when random sampling may be challenging to implement.

What is the unit of analysis?

Methodological studies with a research report as the unit of analysis

Many methodological studies use a research report (e.g. full manuscript of study, abstract portion of the study) as the unit of analysis, and inferences can be made at the study-level. However, both published and unpublished research-related reports can be studied. These may include articles, conference abstracts, registry entries etc.

Methodological studies with a design, analysis or reporting item as the unit of analysis

Some methodological studies report on items which may occur more than once per article. For example, Paquette et al. report on subgroup analyses in Cochrane reviews of atrial fibrillation in which 17 systematic reviews planned 56 subgroup analyses [ 93 ].

This framework is outlined in Fig.  2 .

A proposed framework for methodological studies

Conclusions

Methodological studies have examined different aspects of reporting such as quality, completeness, consistency and adherence to reporting guidelines. As such, many of the methodological study examples cited in this tutorial are related to reporting. However, as an evolving field, the scope of research questions that can be addressed by methodological studies is expected to increase.

In this paper we have outlined the scope and purpose of methodological studies, along with examples of instances in which various approaches have been used. In the absence of formal guidance on the design, conduct, analysis and reporting of methodological studies, we have provided some advice to help make methodological studies consistent. This advice is grounded in good contemporary scientific practice. Generally, the research question should tie in with the sampling approach and planned analysis. We have also highlighted the variables that may inform findings from methodological studies. Lastly, we have provided suggestions for ways in which authors can categorize their methodological studies to inform their design and analysis.

Availability of data and materials

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Abbreviations

Consolidated Standards of Reporting Trials

Evidence, Participants, Intervention, Comparison, Outcome, Timeframe

Grading of Recommendations, Assessment, Development and Evaluations

Participants, Intervention, Comparison, Outcome, Timeframe

Preferred Reporting Items of Systematic reviews and Meta-Analyses

Studies Within a Review

Studies Within a Trial

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

PubMed   Google Scholar  

Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66.

PubMed   PubMed Central   Google Scholar  

Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75.

Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.

Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10):1013–20.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Bmj. 2017;358:j4008.

Lawson DO, Leenus A, Mbuagbaw L. Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review. Pilot Feasibility Studies. 2020;6(1):13.

Puljak L, Makaric ZL, Buljan I, Pieper D. What is a meta-epidemiological study? Analysis of published literature indicated heterogeneous study designs and definitions. J Comp Eff Res. 2020.

Abbade LPF, Wang M, Sriganesh K, Jin Y, Mbuagbaw L, Thabane L. The framing of research questions using the PICOT format in randomized controlled trials of venous ulcer disease is suboptimal: a systematic survey. Wound Repair Regen. 2017;25(5):892–900.

Gohari F, Baradaran HR, Tabatabaee M, Anijidani S, Mohammadpour Touserkani F, Atlasi R, Razmgir M. Quality of reporting randomized controlled trials (RCTs) in diabetes in Iran; a systematic review. J Diabetes Metab Disord. 2015;15(1):36.

Wang M, Jin Y, Hu ZJ, Thabane A, Dennis B, Gajic-Veljanoski O, Paul J, Thabane L. The reporting quality of abstracts of stepped wedge randomized trials is suboptimal: a systematic survey of the literature. Contemp Clin Trials Commun. 2017;8:1–10.

Shanthanna H, Kaushal A, Mbuagbaw L, Couban R, Busse J, Thabane L: A cross-sectional study of the reporting quality of pilot or feasibility trials in high-impact anesthesia journals Can J Anaesthesia 2018, 65(11):1180–1195.

Kosa SD, Mbuagbaw L, Borg Debono V, Bhandari M, Dennis BB, Ene G, Leenus A, Shi D, Thabane M, Valvasori S, et al. Agreement in reporting between trial publications and current clinical trial registry in high impact journals: a methodological review. Contemporary Clinical Trials. 2018;65:144–50.

Zhang Y, Florez ID, Colunga Lozano LE, Aloweni FAB, Kennedy SA, Li A, Craigie S, Zhang S, Agarwal A, Lopes LC, et al. A systematic survey on reporting and methods for handling missing participant data for continuous outcomes in randomized controlled trials. J Clin Epidemiol. 2017;88:57–66.

CAS   PubMed   Google Scholar  

Hernández AV, Boersma E, Murray GD, Habbema JD, Steyerberg EW. Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading? Am Heart J. 2006;151(2):257–64.

Samaan Z, Mbuagbaw L, Kosa D, Borg Debono V, Dillenburg R, Zhang S, Fruci V, Dennis B, Bawor M, Thabane L. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc. 2013;6:169–88.

Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59(7):697–703.

Carrasco-Labra A, Brignardello-Petersen R, Santesso N, Neumann I, Mustafa RA, Mbuagbaw L, Etxeandia Ikobaltzeta I, De Stio C, McCullagh LJ, Alonso-Coello P. Improving GRADE evidence tables part 1: a randomized trial shows improved understanding of content in summary-of-findings tables with a new format. J Clin Epidemiol. 2016;74:7–18.

The Northern Ireland Hub for Trials Methodology Research: SWAT/SWAR Information [ https://www.qub.ac.uk/sites/TheNorthernIrelandNetworkforTrialsMethodologyResearch/SWATSWARInformation/ ]. Accessed 31 Aug 2020.

Chick S, Sánchez P, Ferrin D, Morrice D. How to conduct a successful simulation study. In: Proceedings of the 2003 winter simulation conference: 2003; 2003. p. 66–70.

Google Scholar  

Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106(3):485–8.

Sacks HS, Reitman D, Pagano D, Kupelnick B. Meta-analysis: an update. Mount Sinai J Med New York. 1996;63(3–4):216–24.

CAS   Google Scholar  

Areia M, Soares M, Dinis-Ribeiro M. Quality reporting of endoscopic diagnostic studies in gastrointestinal journals: where do we stand on the use of the STARD and CONSORT statements? Endoscopy. 2010;42(2):138–47.

Knol M, Groenwold R, Grobbee D. P-values in baseline tables of randomised controlled trials are inappropriate but still common in high impact journals. Eur J Prev Cardiol. 2012;19(2):231–2.

Chen M, Cui J, Zhang AL, Sze DM, Xue CC, May BH. Adherence to CONSORT items in randomized controlled trials of integrative medicine for colorectal Cancer published in Chinese journals. J Altern Complement Med. 2018;24(2):115–24.

Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors' implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ. 2012;344:e4178.

The Cochrane Methodology Register Issue 2 2009 [ https://cmr.cochrane.org/help.htm ]. Accessed 31 Aug 2020.

Mbuagbaw L, Kredo T, Welch V, Mursleen S, Ross S, Zani B, Motaze NV, Quinlan L. Critical EPICOT items were absent in Cochrane human immunodeficiency virus systematic reviews: a bibliometric analysis. J Clin Epidemiol. 2016;74:66–72.

Barton S, Peckitt C, Sclafani F, Cunningham D, Chau I. The influence of industry sponsorship on the reporting of subgroup analyses within phase III randomised controlled trials in gastrointestinal oncology. Eur J Cancer. 2015;51(18):2732–9.

Setia MS. Methodology series module 5: sampling strategies. Indian J Dermatol. 2016;61(5):505–9.

Wilson B, Burnett P, Moher D, Altman DG, Al-Shahi Salman R. Completeness of reporting of randomised controlled trials including people with transient ischaemic attack or stroke: a systematic review. Eur Stroke J. 2018;3(4):337–46.

Kahale LA, Diab B, Brignardello-Petersen R, Agarwal A, Mustafa RA, Kwong J, Neumann I, Li L, Lopes LC, Briel M, et al. Systematic reviews do not adequately report or address missing outcome data in their analyses: a methodological survey. J Clin Epidemiol. 2018;99:14–23.

De Angelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJPM, et al. Is this clinical trial fully registered?: a statement from the International Committee of Medical Journal Editors*. Ann Intern Med. 2005;143(2):146–8.

Ohtake PJ, Childs JD. Why publish study protocols? Phys Ther. 2014;94(9):1208–9.

Rombey T, Allers K, Mathes T, Hoffmann F, Pieper D. A descriptive analysis of the characteristics and the peer review process of systematic review protocols published in an open peer review journal from 2012 to 2017. BMC Med Res Methodol. 2019;19(1):57.

Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002;359(9302):248–52.

Porta M (ed.): A dictionary of epidemiology, 5th edn. Oxford: Oxford University Press, Inc.; 2008.

El Dib R, Tikkinen KAO, Akl EA, Gomaa HA, Mustafa RA, Agarwal A, Carpenter CR, Zhang Y, Jorge EC, Almeida R, et al. Systematic survey of randomized trials evaluating the impact of alternative diagnostic strategies on patient-important outcomes. J Clin Epidemiol. 2017;84:61–9.

Helzer JE, Robins LN, Taibleson M, Woodruff RA Jr, Reich T, Wish ED. Reliability of psychiatric diagnosis. I. a methodological review. Arch Gen Psychiatry. 1977;34(2):129–33.

Chung ST, Chacko SK, Sunehag AL, Haymond MW. Measurements of gluconeogenesis and Glycogenolysis: a methodological review. Diabetes. 2015;64(12):3996–4010.

CAS   PubMed   PubMed Central   Google Scholar  

Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in 'meta-epidemiological' research. Stat Med. 2002;21(11):1513–24.

Moen EL, Fricano-Kugler CJ, Luikart BW, O’Malley AJ. Analyzing clustered data: why and how to account for multiple observations nested within a study participant? PLoS One. 2016;11(1):e0146721.

Zyzanski SJ, Flocke SA, Dickinson LM. On the nature and analysis of clustered data. Ann Fam Med. 2004;2(3):199–200.

Mathes T, Klassen P, Pieper D. Frequency of data extraction errors and methods to increase data extraction quality: a methodological review. BMC Med Res Methodol. 2017;17(1):152.

Bui DDA, Del Fiol G, Hurdle JF, Jonnalagadda S. Extractive text summarization system to aid data extraction from full text in systematic review development. J Biomed Inform. 2016;64:265–72.

Bui DD, Del Fiol G, Jonnalagadda S. PDF text classification to leverage information extraction from publication reports. J Biomed Inform. 2016;61:141–8.

Maticic K, Krnic Martinic M, Puljak L. Assessment of reporting quality of abstracts of systematic reviews with meta-analysis using PRISMA-A and discordance in assessments between raters without prior experience. BMC Med Res Methodol. 2019;19(1):32.

Speich B. Blinding in surgical randomized clinical trials in 2015. Ann Surg. 2017;266(1):21–2.

Abraha I, Cozzolino F, Orso M, Marchesi M, Germani A, Lombardo G, Eusebi P, De Florio R, Luchetta ML, Iorio A, et al. A systematic review found that deviations from intention-to-treat are common in randomized trials and systematic reviews. J Clin Epidemiol. 2017;84:37–46.

Zhong Y, Zhou W, Jiang H, Fan T, Diao X, Yang H, Min J, Wang G, Fu J, Mao B. Quality of reporting of two-group parallel randomized controlled clinical trials of multi-herb formulae: A survey of reports indexed in the Science Citation Index Expanded. Eur J Integrative Med. 2011;3(4):e309–16.

Farrokhyar F, Chu R, Whitlock R, Thabane L. A systematic review of the quality of publications reporting coronary artery bypass grafting trials. Can J Surg. 2007;50(4):266–77.

Oltean H, Gagnier JJ. Use of clustering analysis in randomized controlled trials in orthopaedic surgery. BMC Med Res Methodol. 2015;15:17.

Fleming PS, Koletsi D, Pandis N. Blinded by PRISMA: are systematic reviewers focusing on PRISMA and ignoring other guidelines? PLoS One. 2014;9(5):e96407.

Balasubramanian SP, Wiener M, Alshameeri Z, Tiruvoipati R, Elbourne D, Reed MW. Standards of reporting of randomized controlled trials in general surgery: can we do better? Ann Surg. 2006;244(5):663–7.

de Vries TW, van Roon EN. Low quality of reporting adverse drug reactions in paediatric randomised controlled trials. Arch Dis Child. 2010;95(12):1023–6.

Borg Debono V, Zhang S, Ye C, Paul J, Arya A, Hurlburt L, Murthy Y, Thabane L. The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement. BMC Anesthesiol. 2012;12:13.

Kaiser KA, Cofield SS, Fontaine KR, Glasser SP, Thabane L, Chu R, Ambrale S, Dwary AD, Kumar A, Nayyar G, et al. Is funding source related to study reporting quality in obesity or nutrition randomized control trials in top-tier medical journals? Int J Obes. 2012;36(7):977–81.

Thomas O, Thabane L, Douketis J, Chu R, Westfall AO, Allison DB. Industry funding and the reporting quality of large long-term weight loss trials. Int J Obes. 2008;32(10):1531–6.

Khan NR, Saad H, Oravec CS, Rossi N, Nguyen V, Venable GT, Lillard JC, Patel P, Taylor DR, Vaughn BN, et al. A review of industry funding in randomized controlled trials published in the neurosurgical literature-the elephant in the room. Neurosurgery. 2018;83(5):890–7.

Hansen C, Lundh A, Rasmussen K, Hrobjartsson A. Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality. Cochrane Database Syst Rev. 2019;8:Mr000047.

Kiehna EN, Starke RM, Pouratian N, Dumont AS. Standards for reporting randomized controlled trials in neurosurgery. J Neurosurg. 2011;114(2):280–5.

Liu LQ, Morris PJ, Pengel LH. Compliance to the CONSORT statement of randomized controlled trials in solid organ transplantation: a 3-year overview. Transpl Int. 2013;26(3):300–6.

Bala MM, Akl EA, Sun X, Bassler D, Mertz D, Mejza F, Vandvik PO, Malaga G, Johnston BC, Dahm P, et al. Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis. J Clin Epidemiol. 2013;66(3):286–95.

Lee SY, Teoh PJ, Camm CF, Agha RA. Compliance of randomized controlled trials in trauma surgery with the CONSORT statement. J Trauma Acute Care Surg. 2013;75(4):562–72.

Ziogas DC, Zintzaras E. Analysis of the quality of reporting of randomized controlled trials in acute and chronic myeloid leukemia, and myelodysplastic syndromes as governed by the CONSORT statement. Ann Epidemiol. 2009;19(7):494–500.

Alvarez F, Meyer N, Gourraud PA, Paul C. CONSORT adoption and quality of reporting of randomized controlled trials: a systematic analysis in two dermatology journals. Br J Dermatol. 2009;161(5):1159–65.

Mbuagbaw L, Thabane M, Vanniyasingam T, Borg Debono V, Kosa S, Zhang S, Ye C, Parpia S, Dennis BB, Thabane L. Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: a systematic review. Contemporary Clin trials. 2014;38(2):245–50.

Thabane L, Chu R, Cuddy K, Douketis J. What is the quality of reporting in weight loss intervention studies? A systematic review of randomized controlled trials. Int J Obes. 2007;31(10):1554–9.

Murad MH, Wang Z. Guidelines for reporting meta-epidemiological methodology research. Evidence Based Med. 2017;22(4):139.

METRIC - MEthodological sTudy ReportIng Checklist: guidelines for reporting methodological studies in health research [ http://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-other-study-designs/#METRIC ]. Accessed 31 Aug 2020.

Jager KJ, Zoccali C, MacLeod A, Dekker FW. Confounding: what it is and how to deal with it. Kidney Int. 2008;73(3):256–60.

Parker SG, Halligan S, Erotocritou M, Wood CPJ, Boulton RW, Plumb AAO, Windsor ACJ, Mallett S. A systematic methodological review of non-randomised interventional studies of elective ventral hernia repair: clear definitions and a standardised minimum dataset are needed. Hernia. 2019.

Bouwmeester W, Zuithoff NPA, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW, Altman DG, Moons KGM. Reporting and methods in clinical prediction research: a systematic review. PLoS Med. 2012;9(5):1–12.

Schiller P, Burchardi N, Niestroj M, Kieser M. Quality of reporting of clinical non-inferiority and equivalence randomised trials--update and extension. Trials. 2012;13:214.

Riado Minguez D, Kowalski M, Vallve Odena M, Longin Pontzen D, Jelicic Kadic A, Jeric M, Dosenovic S, Jakus D, Vrdoljak M, Poklepovic Pericic T, et al. Methodological and reporting quality of systematic reviews published in the highest ranking journals in the field of pain. Anesth Analg. 2017;125(4):1348–54.

Thabut G, Estellat C, Boutron I, Samama CM, Ravaud P. Methodological issues in trials assessing primary prophylaxis of venous thrombo-embolism. Eur Heart J. 2005;27(2):227–36.

Puljak L, Riva N, Parmelli E, González-Lorenzo M, Moja L, Pieper D. Data extraction methods: an analysis of internal reporting discrepancies in single manuscripts and practical advice. J Clin Epidemiol. 2020;117:158–64.

Ritchie A, Seubert L, Clifford R, Perry D, Bond C. Do randomised controlled trials relevant to pharmacy meet best practice standards for quality conduct and reporting? A systematic review. Int J Pharm Pract. 2019.

Babic A, Vuka I, Saric F, Proloscic I, Slapnicar E, Cavar J, Pericic TP, Pieper D, Puljak L. Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent. J Clin Epidemiol. 2019.

Tan A, Porcher R, Crequit P, Ravaud P, Dechartres A. Differences in treatment effect size between overall survival and progression-free survival in immunotherapy trials: a Meta-epidemiologic study of trials with results posted at ClinicalTrials.gov. J Clin Oncol. 2017;35(15):1686–94.

Croitoru D, Huang Y, Kurdina A, Chan AW, Drucker AM. Quality of reporting in systematic reviews published in dermatology journals. Br J Dermatol. 2020;182(6):1469–76.

Khan MS, Ochani RK, Shaikh A, Vaduganathan M, Khan SU, Fatima K, Yamani N, Mandrola J, Doukky R, Krasuski RA: Assessing the Quality of Reporting of Harms in Randomized Controlled Trials Published in High Impact Cardiovascular Journals. Eur Heart J Qual Care Clin Outcomes 2019.

Rosmarakis ES, Soteriades ES, Vergidis PI, Kasiakou SK, Falagas ME. From conference abstract to full paper: differences between data presented in conferences and journals. FASEB J. 2005;19(7):673–80.

Mueller M, D’Addario M, Egger M, Cevallos M, Dekkers O, Mugglin C, Scott P. Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. BMC Med Res Methodol. 2018;18(1):44.

Li G, Abbade LPF, Nwosu I, Jin Y, Leenus A, Maaz M, Wang M, Bhatt M, Zielinski L, Sanger N, et al. A scoping review of comparisons between abstracts and full reports in primary biomedical research. BMC Med Res Methodol. 2017;17(1):181.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203.

Analytical study [ https://medical-dictionary.thefreedictionary.com/analytical+study ]. Accessed 31 Aug 2020.

Tricco AC, Tetzlaff J, Pham B, Brehaut J, Moher D. Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study. J Clin Epidemiol. 2009;62(4):380–6 e381.

Schalken N, Rietbergen C. The reporting quality of systematic reviews and Meta-analyses in industrial and organizational psychology: a systematic review. Front Psychol. 2017;8:1395.

Ranker LR, Petersen JM, Fox MP. Awareness of and potential for dependent error in the observational epidemiologic literature: A review. Ann Epidemiol. 2019;36:15–9 e12.

Paquette M, Alotaibi AM, Nieuwlaat R, Santesso N, Mbuagbaw L. A meta-epidemiological study of subgroup analyses in cochrane systematic reviews of atrial fibrillation. Syst Rev. 2019;8(1):241.

Download references

Acknowledgements

This work did not receive any dedicated funding.

Author information

Authors and affiliations.

Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada

Lawrence Mbuagbaw, Daeria O. Lawson & Lehana Thabane

Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph’s Healthcare—Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario, L8N 4A6, Canada

Lawrence Mbuagbaw & Lehana Thabane

Centre for the Development of Best Practices in Health, Yaoundé, Cameroon

Lawrence Mbuagbaw

Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia

Livia Puljak

Department of Epidemiology and Biostatistics, School of Public Health – Bloomington, Indiana University, Bloomington, IN, 47405, USA

David B. Allison

Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON, Canada

Lehana Thabane

Centre for Evaluation of Medicine, St. Joseph’s Healthcare-Hamilton, Hamilton, ON, Canada

Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada

You can also search for this author in PubMed   Google Scholar

Contributions

LM conceived the idea and drafted the outline and paper. DOL and LT commented on the idea and draft outline. LM, LP and DOL performed literature searches and data extraction. All authors (LM, DOL, LT, LP, DBA) reviewed several draft versions of the manuscript and approved the final manuscript.

Corresponding author

Correspondence to Lawrence Mbuagbaw .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

DOL, DBA, LM, LP and LT are involved in the development of a reporting guideline for methodological studies.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Mbuagbaw, L., Lawson, D.O., Puljak, L. et al. A tutorial on methodological studies: the what, when, how and why. BMC Med Res Methodol 20 , 226 (2020). https://doi.org/10.1186/s12874-020-01107-7

Download citation

Received : 27 May 2020

Accepted : 27 August 2020

Published : 07 September 2020

DOI : https://doi.org/10.1186/s12874-020-01107-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Methodological study
• Meta-epidemiology
• Research methods
• Research-on-research

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on September 5, 2024 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

You might have to write up a research design as a standalone assignment, or it might be part of a larger   research proposal or other project. In either case, you should carefully consider which methods are most appropriate and feasible for answering your question.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Prevent plagiarism. Run a free check.

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2024, September 05). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved September 16, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Home » Blog » Comprehensive Guide to Research Methodology – Design | Methods | Best Practices

Comprehensive Guide to Research Methodology – Design | Methods | Best Practices

• Other Laws | Blog |
• 19 Min Read
• Last Updated on 16 September, 2024

Recent Posts

Other Laws, Blog

Responsible Business Conduct and ESG in India – Best Practices | Emerging Regulation

News, Blog, Account & Audit

[Opinion] Internal Financial Control—Need to Focus?

Latest from taxmann.

Table of Contents

• Introduction
• Steps in Research Process
• Classification of Research Design

1. Introduction

This article describes the research process and different research designs in detail. Management and social science research, like other forms of scientific inquiry, require a structured sequence of highly interrelated steps (Zigmund et al., 2010). The research process involves a series of steps or actions essential for the smooth conduct of any research. The figure below illustrates the sequence of the research process. It is to be noted that these steps are not a road map to all kinds of research. Basically, it is applicable for deductive or functionalist research, and it can or needs to be revised to suit the requirements of a specific project. The research process doesn’t need to be followed successively; rather, the steps overlap frequently and are interrelated. The research process offers a comprehensive guideline that can be referred to for any management and social science research. It may happen that later stages can be accomplished before the earlier stages.

The steps involved in the research process are neither mutually exclusive nor separate and distinct. The selection of a research topic at the outset, defining the research problem and objectives, influences the selection of a sample and data collection. The sample selection may affect the design of questionnaire items. For example, suppose an organization wants to know the cause of attrition among lower-category employees with low educational qualifications. In that case, the wording for the questionnaire will be easier than for people in top management positions with professional educational qualifications. The steps may differ based on the objectives of the research. However, research based on deductive logic should follow the steps outlined below:

2. Steps in Research Process

• Problem Identification
• Literature Review
• Formulating Research Questions
• Research Design
• Data Collection
• Data Analysis
• Conclusions and Report Writing.

The quest for research must always be triggered by the longing to explore and gain more knowledge and understanding. The management dilemma encourages the need for a decision. The need may arise owing to the cause that the researchers want to discover or reestablish certain relationships. The orientation might be to solve immediate management issues, discover something new, or have purely academic intentions. For instance, in an organization, the manager may want to know the reason for high attrition and lack of job satisfaction, or a retail store may survey the post-purchase satisfaction among the customers.

2.1 Research Problem Identification

Defining the research problem is the first step in the research process. The researchers get the proper direction to conduct their research by first understanding the research problems. Hence, a well-defined research problem is crucial. When the problem is discovered, researchers and management can take further steps to define the problem clearly and precisely. A problem defined with accuracy and conscience helps the researchers utilize the available resources effectively. It is imperative for researchers to explore what exactly is the problem and what are the objectives of the research. The rule generally followed to define the research problem is that the definition should permit the researchers to acquire all details required to address the managerial issues and show guidelines for finding a solution. The researcher should be careful not to define the problem too broadly or narrowly. Examples of broad managerial problems are defining a strategy for enhancing organizational performance and a strategy to elevate the organization’s brand equity. An example of a narrow definition of a problem is how to match competitors’ recruitment strategies. To overcome the possibility of both errors while defining the research problem, the researchers must define the problem with broad, popular terms and devise its components. The broad general statement helps the researchers get a sound perspective on the research problem and avoid the error of defining the problem narrowly. On the other side, the specific component helps to identify the key aspects of the research problem, extend a transparent guideline to proceed further and avoid the error of defining the problem too broadly. In management and social science research, broad management problems need to be converted to information-oriented research problems that focus more on the cause than the symptoms. Some examples of managerial problems converted to research problems are presented in Table below. The conversion of management dilemma to managerial questions and further to research questions can be carried out through exploratory research. Such research incorporates an examination of past research studies, a review of extant literature and organizational records and interviewing experts (Cooper et al., 2016).

2.2 Literature Review

Exploring the existing literature is critical in the research process. Researchers must explore and investigate extant literature to observe whether other researchers have already addressed the identified research problem. A literature review is a systematic search of published work, including periodicals, books, journal papers (conceptual and empirical), and reports, representing theory and empirical work about the research problem and topic at hand. A survey of existing literature is customary in applied research and is an elementary requirement of a basic research report. The internet, electronic databases, websites, and e-library help the researcher to carry out literature surveys systematically and easily.

The literature review aims to study the existing state of knowledge in the domain of interest, to picture key authors, theories, methods, topics, and findings in that domain, and to explore the gaps in knowledge in that domain. A literature review conducted systematically reveals whether initial research questions have already gained substantial attention in the extant literature, whether more interesting newer research questions are available, whether past studies have consistent findings or contradictions exist, flaws in the body of research that the researchers can address, and whether the initial research questions need to be revised as per the findings of the literature review. Furthermore, the review can answer the proposed research questions and help identify theories used in previous studies to address similar research questions. For example, for an organization interested in determining the true cause of turnover, the researcher will study extensively the existing literature on attrition and its causes. By studying relevant journal articles, books, and book chapters, the researcher will discover the causes of attrition in general, find out the existing gaps, and suggest the management carry forward the research to find causes specific to the organization.

As deductive research primarily involves theory testing, the researchers must identify one or more theories that can illuminate the proposed research questions. Through an extensive literature review, researchers may uncover various concepts and constructs related to the phenomenon of interest. A theory will extend support to constructs/variables that are logically relevant to the chosen phenomenon. In the deductive approach, researchers use theory/theories as the logical basis for hypothesis testing. However, researchers must carefully select the theories appropriate for the identified problem to be studied. The hypotheses need to be logically formulated and connected to the research objectives.

2.3 Formulating Research Questions

After problem identification and clarification, with or without an exploratory research approach, the researchers should derive the research objectives. Cautious attention to problem definition helps the researchers devise proper research objectives. Research objectives are the goal to be achieved through research. The research objective drives the research process further. A well-devised research objective enhances the possibility of gathering, relevant information and avoiding unwanted information. The research objectives can be properly developed with the consensus of the researchers and management on the actual managerial and business problems. The researcher should ensure that the research objectives are clearly stated, appropriate, and will yield germane information. The research objective may involve exploring the likelihood of venturing into a new market or may necessitate examining the effect of a new organizational policy on employee performance. The nature and types of objectives lead to choosing an appropriate research design.

Research Objectives:  Research objectives represent the goal of the research the researchers want to accomplish.

2.3.1 Suitable Research Questions

Research questions are important to conduct effective research. Without a clear research question, the researcher may face the risk of unfocused research and will not be sure of what the research is about. Research questions are refined descriptions of the components of the research problem. These are questions related to behavior, events or phenomena of interest that the researchers search for answers in their research. Examples include what factors motivate the employees in an organization to apply the gained knowledge back to their jobs or what needs to be done to enhance the creativity of school-going students. Research questions can best state the objectives of the research. Each component of the research problem needs to be broken down into sub-parts or research questions. Research questions inquire about the information essential concerning the problem components. Properly answered research questions will lead to effective decision-making. While formulating research questions, researchers should be guided by the problem statement, theoretical background, and analytical framework.

Sources of Research Questions

• Extant Literature
• Personal experience
• Societal issues
• Managerial problems
• New theories
• Technological advancement
• Empirical cases
• Contradictory finding

2.3.1.1 Significance of Research Questions

Research questions are critical because they guide scientific and systematic literature search, the decision about appropriate research design, the decision about data collection and target audience, data analysis, selection of right tools and techniques and overall to move in the right direction.

The researcher can utilize different sources for formulating research questions, such as extant literature, personal experience, societal issues, managerial problems, new theories, technological advancement, and contradictory findings. The research question must portray certain attributes. Research questions in quantitative research are more specific compared to qualitative research. Sometimes, some qualitative research follows an open approach without any research questions. The main steps involved in formulating research questions are illustrated in Figure below.

Criteria of Effective Research Questions

• Rateability
• Systematic and logical
• Significant
• Fascinating
• Logical association among variables

The sequence in selecting research questions suggests that the researchers are engrossed in a process of progressive focusing down when developing the research questions. It helps them to slide down from the general research area to research questions. While formulating the research questions, the researchers should understand that ending a research question with a question mark is essential. Without a question mark, a statement cannot be considered as a research question. It is quite possible that the researchers may not get answers to all research questions. The research questions need to be related to each other.

2.4 Planning the Research Design

After formulating research problems and literature surveys, the next stage in the research process is to develop the research design. Research design is the blueprint of research activities to answer research questions. It is a master plan that includes research methods and procedures for gathering and analyzing the relevant information with minimum cost, time, and effort. A research design extends a plan for carrying out the research. The researchers need to decide the source to collect information, the techniques of research design (survey or experiment), sampling techniques, and the cost and schedule of the research. The success of these objectives depends on the purpose of the research. Usually, research purposes are segregated into four types: exploration, description, diagnosis, and experimentation.

There are varied designs, such as experimental or non-experimental hypotheses testing (details of different research designs are outlined in section 2.3 in this chapter). There are four primary research methods for descriptive and causal research: survey, experiments, secondary data, and observations. The selection of an appropriate research method relies on the research objectives, available data sources, the cost and effort of collecting data, and the importance of managerial decisions. If the research objective is exploration, a flexible research design can extend better opportunities to investigate different aspects of the research problem. On the other hand, if the intention is simply to describe any situation or phenomena of interest to examine the relationship between two or more variables, the appropriate design should prioritize minimizing bias and maximizing reliability in data collection and analysis. For example, suppose a researcher wants to conduct exploratory research to know the different types of arthritis common in India. In that case, it may require a flexible design relying on secondary data from hospital records or discussions with doctors or other experts to reach conclusions. However, to invent COVID-vaccination and medicine for the COVID-19 virus, the researchers conducted varied experiments to reach a conclusion.

2.4.1 Hypotheses Development

Exploratory research helps the researchers define the research questions, key variables, and theoretical underpinnings and formulate hypotheses if required in the research. The hypotheses must be logically derived based on the research questions and linked to research objectives. A hypothesis is a tentative proposition regarding a research phenomenon. It may be a tentative statement that indicates an association between two or more variables, guided by any supportive theory, theoretical framework, or analytical model. It is a viable answer to the research questions framed by the researchers. Hypotheses are statements of relationships or propositions that are declarative and can be tested with empirical data. Some examples are:

H 1 : Training influences organizational performance.

H 2 : Training enhances employee performance.

For two more research questions i.e., “to what extent does brand love determine purchase intention?” and “does age and family background moderate the relationship?”, the hypotheses are:

H 1 : Brand love is related to purchase intention.

H 2 : Age and Family status moderate the association between brand love and purchase intention. Figure below provides a pictorial representation of the hypotheses drawn.

However, it is not always feasible for researchers to formulate hypotheses in all situations. Sometimes, researchers may lack all relevant information, and theoretical support may not be available to formulate the hypotheses.

2.5 Sampling Design

This stage of the research process involves an investigation of the population under study. A complete investigation of the population under study is known as a census inquiry. Usually, in census investigation, all units or items of the population are studied with high accuracy and reliability. However, it is usually not practicable and feasible for the researchers to study the entire population. Researchers usually prefer to investigate small, representative subgroups from the population known as sample. The procedure to select the sub-groups/samples is called sampling design. Sampling entails the process of drawing conclusions based on a subgroup of the population. Hence, the sample is a subset of the population. The first question that needs to be addressed in sampling is “who is to be included in the sample?” and this requires the identification of the target population under study. It is difficult for the researcher to define the population and sampling unit. For example, if a researcher wants to investigate the financial savings and vehicle loan association survey. In that case, individuals with existing accounts will be taken, and this sample unit represents the existing customers and not the potential customers. Hence, it is critical in sampling design to determine the specific target population.

Secondly, the issue that concerns the researchers in sampling design is selecting an appropriate sample size, and the third concern is selecting the sampling units. Researchers need to address these concerns to justify the research. Samples can be selected either using probability sampling techniques or non-probability sampling techniques. There are four types of probability sampling such as simple random, systematic, stratified, and cluster sampling. Non-probability sampling includes convenience, judgmental, quota, and snowball sampling. Depending on the objective, researchers should select the appropriate sampling techniques for their study.

2.6 Fieldwork and Gathering Data

After the formalization of the sampling plan, the fieldwork and data-gathering stage begins. The researcher gathers data after finalizing what to research, among whom, and which method to use. Data gathering involves the process of information collection. Different data collection instruments are available for researchers to collect information or data. Broadly, there are two ways to collect data, such as primary and secondary data collection methods. Primary data include data collected firsthand and are original. Varied methods are available for primary data collection, such as structured and unstructured interviews, focused group discussion, observation, and survey using a structured questionnaire. The data can be collected offline or online. Secondary data included information collected from published or unpublished sources that were already available. Some secondary data collection sources are articles, magazines, company records, expert opinion survey data, feedback of customers, government data, and past research on the subject. For example, to conduct a survey of job satisfaction in an organization, the researcher may circulate a printed questionnaire offline or mail the questionnaire to the selected respondents following an appropriate sampling technique.

Another example could be a study that investigates the purchase preference for luxury cars, and the base model demands primary and empirical information. However, another study that intended to describe the financial investment behavior of existing customers will use secondary data. At this stage, the researchers need to ensure the reliability and validity of the data obtained for the study.

2.7 Data Processing and Analysis

After data gathering, the data needs to be converted or properly coded to answer the research question under study. The information gathered in the data collection phase should be mined from the primary raw data. Data processing starts with data editing, coding, and tabulation. First, it is vital for the researchers to check the data collection forms for missing data, clarity, and consistency in categorization. The editing process involves problems associated with data, such as respondents’ response errors. Editing improves the quality of the data and makes the data usable for tabulation, analysis, and interpretation. Tabulation is a technical process in which classified data are presented in tables. Researchers use computers to feed data to a computer spreadsheet for data analysis. The preparation of a spreadsheet also requires lots of expertise and experience.

After coding the data, the next step is to analyze the data. Data analysis is the utilization of reasoning to make sense of data gathered. Ample statistical techniques are available for the researchers to analyze the data. Based on the research questions, objectives, study types, sampling framework used, data types, and degree of accuracy involved in the research, one can choose from parametric or non-parametric techniques for data analysis. Researchers may adopt univariate, bi-variate or multi-variate methods for data analysis. The analysis may include simple frequency analysis, multiple regression, or structural equation modeling. Different techniques are available for qualitative data, presented in Part 3 of this book.

2.8 Drawing Conclusion and Preparing a Report

After data analysis, the final stage in the research process is the interpretation of the results. The researcher requires analytical skills to interpret the statistical results, link the output with the research objectives, and state the implications of the result.

Research Design:  Research design is the blueprint/systematic steps to carry out research smoothly

Finally, researchers must communicate the result in the form of a report. The preparation of the final report needs to be done with the utmost care. The final report should include the identified research questions, research approach, data collection method, data analysis techniques, study findings, and implications for theory and practice. The structure of the report will be discussed in the last section of this book. The report should be prepared comprehensively to be usable by management or organizations for decision-making.

3. Classification of Research Design

This section highlights the classification of research design. As mentioned in the previous section, research design is the framework for carrying out management and other research. After the identification of a problem, the researchers formulate the research design. A good research design ensures the effectiveness of the research work. The choice of selecting an appropriate design relies on the research objectives. The broad categorization of research design with sub-categorization is detailed in various sub-sections.

3.1 Exploratory Research Design

Methods to Conduct Exploratory Research

• Literature survey
• Secondary sources of data
• Experience survey
• Focused group discussions
• Observations
• Structured and unstructured interviews
• Pilot surveys
• Case Studies

Exploratory research design is the simplest form of research design. The researchers explore the true nature of the problem. When researchers aim to study a new area or examine a new interest, exploratory design is a good option. This research design is flexible and versatile in approach. The information required by the researchers is defined loosely and unstructured. Researchers carrying out qualitative research usually adopt exploratory research design. Exploratory research design serves three purposes (a) it helps the researchers to address their inquisitiveness and quest for better understanding (b) to assess the practicality of carrying out border research (c) and devise methods for further studies.

Methods to Conduct Descriptive Research

• Self-administered survey
• Phone survey
• Mail survey/online survey
• Observation
• Personal interview
• Telephone interview

Exploratory research design has paramount significance in management and social science research. They are crucial for researchers who want to study something new. To cite an example, during the COVID-19 pandemic, physical health, mental health, and safety of school and college-going children were a concern for most people. The online education system was the new normal at that time. Research studying the impact of digitalization, long time spent in online studies on students’ health and mental well-being during the COVID-19 pandemic, is of an exploratory kind. One of the disadvantages of exploratory research design is that researchers rarely get specific answers to the research questions.

3.2 Descriptive Research Design

The prime objective of descriptive research design is to describe certain situations or events. This type of design provides an extensive explanation of the research phenomena under study. In descriptive research, the researchers possess prior knowledge about the problem situations. The information is defined with clarity. This kind of research is preplanned and more structured than exploratory research. Researchers must formulate research questions properly and have clarity regarding the types of data needed and the procedure to be followed to achieve the research objectives. Researchers have the luxury of covering a large representative sample. Researchers must answer five Ws and one H – what, who, when, where, why, and how of research issues. What kind of information is required for the research, who are the target respondents, when the information will be collected, where to interact with the respondents, why information is collected from the respondents and how to collect data from the respondents. Descriptive research studies can be cross-sectional or longitudinal. The major objectives for the following descriptive research are given below.

• To explain the characteristics of certain groups such as the Indian population, employees, students, marketing personnel, organizations, sales persons. For example, a university to design a customized online higher studies course for working professionals needs a holistic profile of the interested population.
• To evaluate the portion of individuals in a specific population portraying a typical behavior. For instance, when a researcher is inclined to know the percentage of employees not interested in an online platform introduced for them in their organization.
• To predict for future. For instance, to know the future of physical retail stores due to the widespread expansion of online stores.
• To examine the extent to which management research variables relate to each other. For example, to what extent does work-life balance, salary, and conducive work environment enhance employee job satisfaction?

3.3 Causal Research Design

Usually, causal research design is adopted by researchers to explain causal relationships among phenomena under study. Causal research examines cause-and-effect relationships among variables. Causal research has certain criteria, as already discussed in Chapter 1. Causal research follows a planned and structured design like descriptive research. Though the magnitude of the relationship among variables is examined in descriptive research, the causal association cannot be explained through such research. Experimentation is one of the methods for carrying out causal research.

In causal research, the researchers usually examine the impact of one variable on another. The researchers try to explore the cause-and-effect relationship (nomothetic explanation). How can the researcher know whether cause and effect are associated? There are three criteria for a nomothetic causal relationship when (1) two or more variables are correlated, (2) the cause precedes the effect and (3) the absence of a plausible alternative explanation for the effect other than the proposed cause (Babbie, 2020). First, without establishing a correlation among two or more variables, causation cannot exist. Second, the cause should happen before the effect in time. For instance, it is more sensible to say that children’s religious affiliation is caused by their parents than to reflect that parents’ religious affiliation is due to children; even in some cases, it is plausible that children may convert to other religions later with their parent’s permission. The third significant condition for a causal relationship is that the effect cannot be attributed to any external third variable for establishing causation.

To cite one classic example, there is a causal association between sales of ice cream and death owing to drowning. Intake of more ice creams in summer does lead to a higher death rate due to drowning. The third intervening variable that causes higher death is season or temperature. In summer, higher deaths occur due to swimming and not because of taking ice-creams. The intervening variable season or temperature causes a higher death rate.

To establish a reliable causal relationship among two or more variables, other influencing variables must be controlled to neutralize their impact on the studied variables. For example, to study the effect of factors influencing training transfer in soft skill training, the other intervening variables such as age, gender, and educational qualification need to be controlled. This kind of research sometimes demands experimentation to establish causality. In most cases, causal research is quantitative and needs statistical hypothesis testing.

3.4 Experimental Research Design

Experimental research aims to examine the cause-effect relationship in a controlled setting by isolating the cause from the effect in time. The three criteria suggested by John Stuart Mill mirror in experimental research. In experimental research, the cause is administered to one group of subjects, known as the treatment group and not to the control group, and the researchers observe the difference in mean effect among the subjects of both groups. Whether variation in the cause is connected to variation in effect is observed. To be more specific, the researcher manipulates the independent variable and examines the change in the dependent variable, keeping other variables constant. Researchers used varied methods during the experiments to reduce the plausible effect of other explanations for the effect, along with ancillary methods to investigate the plausibility of those that cannot be ruled out. It is vital in experimental studies to control the extraneous and confounding variables while carrying out the experiments. Ignorance of such variables may lead to spurious relationships among studied variables. However, bringing many of the variables under experimental control is impossible. For example, personal characteristics of the subject like age, sex, intelligence, beliefs and persona. In such cases, the researchers must observe natural variations in the variables of concern. Then, statistical procedures are used to rule out the plausible impact of uncontrolled factors.

Experimental Research Design:  An experiment is a method of collecting evidence to indicate the effect of one variable on another.

Experimental research design can be conducted in a laboratory setting (laboratory experiment) or in a field setting (field experiments) where the phenomena of research interest happen. As an example, one of the most talked about and controversial experiments conducted on understanding human behavior has been the Stanford Prison Experiments, which took place at Stanford University in 1971. The experiments were funded by the US Office of Naval Research, and the principal investigator for the same was Prof Phillip Zimbardo. The major purpose of these experiments was to understand how norms develop and social expectations about roles shape group behavior. Experimental studies are segregated into four categories such as pre-experimental, true-experimental, quasi-experimental and statistical design.

3.4.1 Correlation, Causation and Cofounds

Correlation cannot be treated as causation, and correlation does not always prove causation. In correlation, it is unclear which variable comes first or whether any alternative explanation exists for the assumed effect. Two variables may be correlated due to chance. Correlation is symmetric, while causation is asymmetric. Two variables may be co-related, but their relationship may be affected by a third variable called cofounds. For example, let’s say that high salary and high educational qualifications are correlated. It is difficult to say with confirmation which comes first. Whether a high educational qualification leads to a high salary, or a high salary leads to a high educational qualification. Both possibilities can hold true and necessitate further investigation. Until researchers conclude through their investigation, a mere correlation among these two variables will not give a clear picture of their causal relationship. There is also the possibility of an alternative explanation for the relationship between high salary and high educational qualifications. The link may be due to a third variable called intellect, which results in high salary and high educational qualifications.

In management research, social science, and natural science, three significant pairs of components are required for experimentation: Experimental and control group, independent and dependent variable, and pre-test and post-test.

3.4.1.1 Experimental and Control Group

The group in which an experimental treatment is administered is known as the experimental or treatment group. In contrast, the group in which no experiment is administered is known as the control group. Using control groups enables the researchers to assess the experiment’s effects. For example, suppose a researcher wants to study the impact of rewards on employee productivity in an organization. In that case, the researcher can experiment with two groups of employees. One group will be given external rewards, known as the experimental group, and the other group (control group) will provide no external rewards. Then, the researcher can investigate the causal association between rewards on employees’ productivity through this experiment. The use of a control group is quite common in medical science research. In social science and management research, the use of control groups and experimental studies became popular with several experiments conducted in the late 1920s and early 1930s by F. J. Roethlisberger and W. J. Dickson (1939) to discover the changes required in working conditions to enhance employee satisfaction and productivity. Their series of experiments resulted in the Hawthorne effect.

3.4.1.2 Independent and Dependent Variables

In experimental research, the researchers study the impact of an independent variable on the dependent variable. Usually, experimental stimuli, whether present or absent, are considered independent variables. Independent variables are manipulated in the study, and their effects are assessed and compared. The researchers compare outcomes when the stimulus is present and not present. Hence, the independent variable is the cause, and the dependent variable is the presumed effect. It is to be noted that the independent variable in one study may serve as a dependent variable in another study. For example, an experiment intends to explore the causality between high salary and job satisfaction, job satisfaction is the dependent variable. However, in another experiment designed to explore the causality between job satisfaction and employee productivity, job satisfaction is the independent variable.

3.4.1.3 Pre- and Post-test

In an experiment, the experimenters measure the variable before conducting the experiment on the group known as the pre-test and measure the variable after conducting the experiments is called as post-test. Hence, subjects are exposed to a stimulus called a dependent variable (pre-testing), then exposed to a stimulus, i.e., independent variable, and again assessed with a dependent variable (post-testing). Any discrepancies between the two measurements of dependent variables are ascribed to the independent variable.

Disclaimer: The content/information published on the website is only for general information of the user and shall not be construed as legal advice. While the Taxmann has exercised reasonable efforts to ensure the veracity of information/content published, Taxmann shall be under no liability in any manner whatsoever for incorrect information, if any.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

PREVIOUS POST

To subscribe to our weekly newsletter please log in/register on Taxmann.com

Latest books.

R.K. Jain's Customs Tariff of India | Set of 2 Volumes

R.K. Jain's Customs Law Manual | 2023-24 | Set of 2 Volumes

R.K. Jain's GST Law Manual | 2023-24

R.K. Jain's GST Tariff of India | 2023-24

Everything on Tax and Corporate Laws of India

Everything you need on Tax & Corporate Laws. Authentic Databases, Books, Journals, Practice Modules, Exam Platforms, and More.

• Express Delivery | Secured Payment
• Free Shipping in India on order(s) above ₹500
• +91 8688939939
• About Company
• Media Coverage
• Union Budget 2024-25

• Business & Support
• Sell with Taxmann
• Locate Dealers
• Locate Representatives
• CD Key Activation
• Privacy Policy
• Return Policy
• Payment Terms

• Open access
• Published: 17 September 2024

Lessons from the COVID-19 pandemic to strengthen NCD care and policy in humanitarian settings: a mixed methods study exploring humanitarian actors’ experiences

• Éimhín Ansbro 1 , 2 ,
• Olivia Heller 3 ,
• Lavanya Vijayasingham 1 , 2 ,
• Caroline Favas 1 , 2 ,
• Jacqueline Rintjema 2 , 4 ,
• Alyssa Chase-Vilchez 5 ,
• Claire Stein 6 , 7 ,
• Rita Issa 8 , 9 ,
• Leah Sanga 1 , 2 ,
• Adrianna Murphy 2 , 10 &
• Pablo Perel 1 , 2  

BMC Health Services Research volume  24 , Article number:  1081 ( 2024 ) Cite this article

Metrics details

The COVID-19 pandemic and response severely impacted people living with non-communicable diseases (PLWNCDs) globally. It exacerbated pre-existing health inequalities, severely disrupted access to care, and worsened clinical outcomes for PLWNCDs, who were at higher risk of morbidity and mortality from the virus. The pandemic’s effects were likely magnified in humanitarian settings, where there were pre-existing gaps in continuity of care for non-communicable diseases (NCDs). We sought to explore factors affecting implementation of NCD care in crisis settings during the COVID-19 pandemic and the adaptations made to support implementation.

Guided by the Consolidated Framework for Implementation Research, we undertook an online survey of 98 humanitarian actors from multiple regions and organization types (March-July 2021), followed by in-depth interviews with 13 purposively selected survey respondents (October-December 2021) . Survey data were analysed using descriptive statistics, while interview data were analysed thematically, using both deductive and inductive approaches.

Initially, humanitarian actors faced challenges influenced by external actors’ priorities, such as de-prioritisation of NCD care by governments, travel restrictions and supply chain interruptions. With each infection wave and lockdown, humanitarian actors were better able to adapt and maintain NCD services. The availability of COVID-19 vaccines was a positive turning point, especially for the risk management of people with NCDs and protection of health workers. Key findings include that, despite pre-existing challenges, humanitarian actors largely continued NCD services during the crisis. Enabling factors that supported continuity of NCD services included the ability to quickly pivot to remote means of communication with PLWNCDs, flexibility in medicine dispensing, and successful advocacy to prioritize NCD management within health systems. Key lessons learned included the importance of partnerships and cooperation with other health actors, and the mobilisation or repurposing of community health workers/volunteer networks.

Conclusions

The COVID-19 experience should prompt national and global health stakeholders to strengthen inclusion of NCDs in emergency preparedness, response, and resilience planning. Key lessons were learned around remote care provision, including adapting to NCD severity, integrating community health workers, providing context-adapted patient information, combating misinformation, and strengthening cross-sectoral partnerships.

Peer Review reports

The SARS-2 Coronavirus (COVID-19) pandemic caused unprecedented challenges worldwide, testing healthcare systems across continents, affecting populations’ health and wellbeing, and highlighting global and national inequities [ 1 , 2 ]. COVID-19 was more likely to cause severe infection and death in people who were older (75 years and above), immunocompromised or living with non-communicable disease (NCD) [ 1 , 3 ]. As early as May 2020, NCDs and COVID-19 were cast as twin epidemics and later as a “syndemic.” They acted synergistically on morbidity and mortality, and shared a common set of underlying risk factors, including socio-economic deprivation, obesity, older age, and ethnicity [ 4 ]. As COVID-19 deaths reached one million worldwide, and the key roles of social inequity and failed political leadership were recognised, there was growing acknowledgement that tackling NCDs would be a “prerequisite for successful containment” of COVID-19 [ 5 ]. This required a broader syndemic approach, encompassing housing, education, employment, health, and the environmental sectors.

For decades before the pandemic, NCDs, notably cardiovascular diseases, cancers, diabetes, and chronic respiratory diseases, were the leading causes of mortality globally. They are responsible for 41 million deaths each year, equating to 75% of total global deaths [ 6 ]. People living in low and middle-income countries (LMICs), where the majority (70%) of global NCD deaths occur, are disproportionally affected by premature NCD mortality (i.e., deaths occurring before the age of 70) [ 6 ]. For best outcomes, people living with NCDs (PLWNCDs) require functioning health systems to deliver a continuum of care. This includes early detection through screening and diagnosis; accessible and continuous care and medications; and supported self-care and education, as well as context-adapted healthy eating and exercise opportunities [ 7 , 8 ].

In parallel, more people than ever are affected by humanitarian crises, which have become more complex and prolonged [ 7 , 8 , 9 ]. Conflict, violence, and socio-economic inequity drive most of these crises, and many are now compounded by climate change. In 2021, COVID-19 overlaid other pre-existing and emerging crisis risks, as humanitarian needs remained at historically high levels. An estimated 306 million people were in need in 2021, 90.4 million more than in 2019, before the COVID-19 pandemic hit [ 10 , 11 ].

Humanitarian emergencies disrupt care for NCDs, through destruction of health infrastructure and supply chains, and by reducing access to care. The continuum from diagnosis and screening services, to medical consultation, provision of regular medicines and equipment, and referral pathways may all be affected. Limited evidence also shows that the rates of acute exacerbations, including heart attacks, strokes, asthma attacks, and amputations are increased by stress, and are higher both during emergencies and in their immediate aftermath [ 7 , 8 ]. Recent World Health Assembly resolutions and the World Health Organization (WHO) NCD Global Action Plan 2013–2030 underlined the importance of ensuring that refugees and internally displaced people can access care for NCDs [ 12 ]. However, until recently, NCDs have not been afforded the same priority as other important health concerns during acute crises, and have often been insufficiently integrated into emergency preparedness and response [ 13 ].

Refugees and other displaced people and those with limited health care access—as well as PLWNCDs—were considered “high burden” populations affected by the pandemic and its response [ 14 ]. Many national response policies to manage COVID-19 infections directly caused disruptions of NCD services along the continuum of care [ 15 , 16 ]. A WHO survey conducted from May to July 2020 indicated that about 75% of global NCD services were disrupted in the early days of the pandemic, with low (65%) and lower- middle income (49%) countries most affected [ 17 ]. In the initial months of the pandemic in 2020, NCD care was commonly disrupted because of the urgent diversion of health care resources towards the COVID-19 response, government-imposed travel restrictions, advice to high-risk people to isolate, and people’s understandable fear of attending health facilities [ 17 , 18 , 19 , 20 ]. Data from high- and middle-income countries demonstrate the consequences of foregone or delayed NCD-related healthcare seeking. These include poorer rates of diabetes diagnosis, control and up-titration of medications, and poorer CVD outcomes due to decreased access to care [ 21 , 22 , 23 , 24 ]. Reduced facility attendance or admission for acute NCD complications, such as heart attacks, often increase out-of-hospital deaths, and worsen long-term complications, including functional impairments and disability [ 20 ].

Some humanitarian actors have signalled their ability to continue NCD services with minimal disruptions during the peak of the COVID-19 pandemic [ 25 ], an ability that was not demonstrated even in stable high- and middle-income settings in the early phases of the response [ 18 ]. However, we know little about how COVID-19 disrupted NCD services in crisis settings more broadly, how actors adapted, and what factors enabled or hindered them to do so.

Though the peak of the COVID-19 pandemic is behind us, it is important that we learn lessons from this experience that may shape future NCD services and policies. Given the likelihood of another pandemic, and the fact that the climate crisis will cause more extreme weather events and compound the vulnerabilities that lead to conflict, WHO and other actors are placing greater emphasis on health system preparedness, response, and resilience. Therefore, factors affecting continuity of care for NCDs and successful adaptations to care delivery in the context of COVID-19 are important for preparing for future health service disruptions, for ongoing crises, and where marginalised or vulnerable communities have limited access to care [ 26 ]. Accordingly, we sought to explore factors affecting implementation of NCD care in crisis settings during the COVID-19 pandemic in LMICs, and the adaptations made to support implementation.

Study team and setting

The Centre for Global Chronic Conditions, in collaboration with the Health in Humanitarian Crises Centre, from the London School of Hygiene and Tropical Medicine (LSHTM), led the study in partnership with the Global Alliance for Chronic Disease (GACD) Humanitarian Crises Working Group. The research design was guided by an advisory committee of experts from key humanitarian organisations and agencies [WHO, United Nations High Commission for Refugees (UNHCR), International Committee of the Red Cross, Médecins sans Frontières, and International Rescue Committee] who work on global policies and programmes delivering NCD care in humanitarian settings. This was a global study, targeting humanitarian actors in all geographical settings, who were involved in direct delivery of NCD care during the COVID-19 pandemic.

Study design

The study used a newly developed online survey in English (Additional file 1 ) targeting humanitarian actors, followed by individual interviews (Additional file 2 ) with selected participants. We focussed on the delivery of care for hypertension, type-1 and type-2 diabetes (“DM/HTN”, implying care for either or all conditions) as these are the most common NCD types currently addressed by humanitarian organisations [ 13 , 27 ]. These conditions are also established tracer conditions, used in the healthcare quality assessment literature to assess health system or service performance [ 28 , 29 , 30 ]. These example conditions tend to be well defined, prevalent, relatively easy to diagnose, and have effective, available treatments.

Conceptual framework and definitions

We used an implementation science framework, the Consolidated Framework for Implementation Research (CFIR – Fig.  1 ) to inform the design and analysis of the survey and interviews [ 31 , 32 ]. CFIR is a practical framework, which provides a list of constructs, organised within domains, that are believed to influence implementation, either positively or negatively. It is intended to help guide the systematic assessment of potential barriers and facilitators and, thus, tailor implementation strategies and adaptations, and/or to explain outcomes. The five major domains of the framework – 1) intervention characteristics, 2) outer setting, 3) inner setting, 4) characteristics of individuals, and 5) process – provided a means to synthesise diverse interventions or adaptations in various contexts in response to a global pandemic.

The Consolidated Framework for Implementation Research framework (2009), Source: [ 31 , 32 ]

For this study, we conceptualised the “intervention” as maintaining access to NCD care while responding to the health risks of the COVID-19 pandemic. “Maintaining access to care” was defined as the continued provision of care to the target population at a minimum acceptable level, compared to the baseline (e.g., before the pandemic), so that the services were available (i.e., with adequate human resources, equipment – including drugs – to safely deliver quality services), physically accessible and affordable, and utilised by the target population. NCD care refers to primary health care level activities for people with hypertension and/or diabetes that we propose are essential to maintain during the COVID-19 pandemic.

Data collection

The online survey (Additional file 1 ) was designed by the LSHTM team, guided by the CFIR framework constructs, reviewed by the advisory committee, and piloted. Questions focussed on the delivery of a specific programme/project, focussing on the characteristics of pre-pandemic NCD services, adaptations made in response to the pandemic, individual and inner and outer setting challenges or facilitators, and decision making. We defined the components of NCD services as: medical consultation, disease monitoring, PLWNCDs’ education and support services, and primary prevention and community screening. The survey was hosted on the BOS Online Survey tool ©. A survey link was shared with all participants via email, and the survey included screening questions to restrict participation to people with relevant profiles. It was launched in March 2021 and closed in June 2021.

For the in-depth interviews, a structured topic guide (Additional file 2 ) was used to direct the flow of conversation, and ensure coherence of discussions with the study’s aims and survey. To facilitate rapid data collection, a team of four female interviewers with a public health background (CS, AC, JS, RI) was trained by EA. Each interviewer invited two to four participants and undertook between one and three interviews. From October to December 2021, thirty participants were contacted by email, of whom 13 took part in an interview. Interviewers probed the participants with follow-up questions based on their unique responses, and at the interviewer’s discretion. Interviews took place from November to 2021 to January 2022, and lasted between 45–60 min. They were conducted online, over the phone, or via Skype or Zoom audio-conferencing platforms. Interviews were conducted in English and were digitally audio-recorded, and transcribed for analysis using MS Word and Excel. Written, informed consent, was transmitted via e-mail. Weekly meetings were held with the study team to debrief on interviews, discuss initial findings and iteratively adapt the topic guide.

Participant sampling

Project managers or medical staff directly involved in NCD care delivery at project/programme level in humanitarian settings during the COVID-19 pandemic were eligible for the online survey. Programming professionals are directly involved in the implementation of NCD programmes and service delivery, and their tacit working knowledge and experience provide invaluable insights into how the COVID-19 pandemic and policies affected NCD programmes, as well as how adaptations were formulated, coordinated, and implemented during this crisis. Using our existing GACD, LSHTM, and advisory committee networks, our partners emailed a convenience sample of their contacts who fit the sampling criteria, sharing information on the study, and inviting them to fill in the online survey. Snowball sampling of the respondents’ contacts was used to extend the sampling frame.

A sub-set of survey participants was invited to participate in in-depth interviews, six months after the survey was administered. The interview cohort was purposively selected to represent voices of participants in a range of roles in NCD programmes, from different organisation types that employed different types of adaptations, across different global regions. With input from the advisory committee, the study team defined the following selection criteria to identify follow-up interview participants: 1) geographical spread, 2) range of adaptations/ adjustments, 3) range of organizations, and 4) range of positions/ roles in NCD care delivery.

Data analysis

Descriptive tabulation of quantitative survey responses was undertaken using the Stata statistical software package [ 33 ]. The survey was conducted as a rapid response to the initial phase of the pandemic, and early findings were shared with the advisory committee.

Qualitative data from a) survey free text responses, and from b) interview transcripts, were analysed jointly, using a combination of Framework Analysis (deductive coding) and inductive open coding approaches [ 34 ]. The Framework Method provides clear steps to follow and produces highly structured outputs of summarised data. It is therefore useful where multiple researchers are working on a project, particularly in multi-disciplinary research teams where not all members have experience of qualitative data analysis. First, an a priori coding template using MS Excel was developed by EA based on the CFIR framework (Fig. 1 ) to guide the deductive coding process (performed by OH, AC, CS). A separate data-driven inductive coding exercise was conducted by EA and LV. Repeated review and the complimentary coding approaches enriched the research team’s interpretive and analytic understanding of the data. The qualitative data is presented as reconstructed narratives using both a descriptive and interpretive stance, by themes, and with direct quotes from the participants.

The survey received 98 responses, from 38 different organisations, operating in 21 different countries. Most survey respondents were working in South-East Asia, Africa, and the Eastern Mediterranean (34%, 33% and 28% respectively), and their programmes were based in protracted conflict areas (32%), and targeted refugees (83%), although 60% targeted mixed populations [i.e., a mix of refugees, internally displaced populations (IDPs), and/or host populations]. Most programmes were in camp settings (70%), and provided DM/HTN care integrated within general primary health care (63%) or with other NCDs (including cardiovascular disease and mental health care) (26%). Table 1 outlines the characteristics of the survey respondents and the NCD programmes they were involved in.

Interviews were conducted with 13 of these survey respondents. Table 2 outlines the interview participants’ characteristics.

Findings from both the survey and interviews are reported below, following the CFIR implementation framework constructs ( intervention characteristics, process, outer setting, inner setting, and characteristics of individuals ) and subconstructs, which are highlighted in italics. As mentioned, we defined the “intervention” as maintaining continuity of NCD services, while mitigating the threat of COVID-19.

Intervention characteristics

Before the pandemic, medical consultation was provided by generalist doctors in 90% of respondents’ NCD programmes; specialist doctors, nurses, and lay- or community-based health workers/volunteers were involved in 27%, 41% and 43% of respondent’s programmes, respectively. Consultations were done individually and face-to-face in most (98%) cases. Groups were utilised for consultation and monitoring, but mainly for education and prevention/screening activities. Most medical consultations were delivered in a primary care centre or health posts (89%), fewer in secondary or tertiary level hospitals (36%), and services included home visits in 25% and mobile clinics in 15% of cases.

During the pandemic response, more than half of the NCD service components provided before the pandemic were partially or fully maintained, including medical consultation (94%), disease monitoring (90%), PLWNCDs’ education and support (88%) and primary prevention and community screening services (61%). As might be expected, face-to-face individual services declined, with more than 50% of these services reduced during the pandemic, and medical consultation via home visits were cut by half. More detail on the characteristics of NCD service components before and during the pandemic are available in Additional file 3 .

Organisations’ implementation processes varied as they experienced different organisational ( inner setting ) and contextual ( outer setting ) barriers and facilitators. Services were adapted iteratively as the pandemic progressed. For example, survey respondents reported outer setting factors that hampered continuity of service delivery, including poor mobile phone coverage (28%), smartphone availability (35%) and internet connectivity (35%). PLWNCDs faced challenges in managing their disease, especially financially (49%) and mentally (42%).

The key CFIR intervention constructs that were generated from interview and survey free text data were source, evidence strength and quality, adaptability, and cost. At the onset of the pandemic, national policies immediately targeted infection prevention and control (IPC) to limit the pandemic’s spread, introducing movement restrictions, and diverting health system policy and resources to the pandemic response. In the early days, interviewees reported initial uncertainty in how to respond to these policies.

The decision to prioritise PLWNCDs and the specific adaptations made to service delivery were perceived as coming strongly from within individual organisations, with recommendations coming from WHO/UNHCR, rather than from national governments. The latter were largely perceived as having “ forgotten ” PLWNCDs in their initial pandemic response plans. The source of IPC guidance, training and equipment was perceived to be national governments, Ministries of Health, and international actors, such as the WHO and UNHCR. The UN sources were considered trustworthy and of good quality, filling essential gaps when information or action was lagging from national resources. The cost of maintaining NCD care was mainly spoken of in terms of the cost and diversion of funds into IPC measures, and the fact that pandemic-related inflation increased costs for governments, organisations, and PLWNCDs, for example, significantly increasing transportation costs. The CFIR constructs complexity, trialability and relative advantage versus other interventions did not feature strongly in the data. There were many unknowns at the beginning of the pandemic response, and there was acknowledgement that organisations did not have time to trial interventions but, instead, needed to act quickly.

In most settings, the process of maintaining NCD care could be summarised as involving the following key components: a) the introduction or enhancement of IPC measures; b) prioritisation of PLWNCDs and maintenance of clinical contact, including through remote means; c) maintenance of medication and equipment supplies; d) maintenance or adaptation of the health workforce; e) information sharing between organisations and with PLWNCDs, and countering misinformation; and iteratively adapting these approaches as the pandemic evolved:

“Adaptations done in NCD service delivery were aimed to address the safety of NCD patients from COVID-19, considering their susceptibility to mortality due to COVID-19, also safety of health care staff, from community level to health facility level” [ID01]

The CFIR constructs planning, engaging, executing, and reviewing were discussed in interviews and survey free text responses. Evaluating was less prominent in the data, given that data were collected relatively early in the pandemic response, and programmes did not have time to formally evaluate their response strategies. However, respondents reported anecdotally that their interventions were successful.

The WHO Health Sector Cluster System or UNHCR-coordination systems, which are used to coordinate multiple agencies during emergency responses, were instrumental in planning and executing the pandemic response in places where it was already established. For example, in these settings, collaboration and information sharing occurred early in the pandemic. Decisions on how to respond were generally made by the organisation’s management, although one interviewee described close engagement of clinical staff in an iterative decision-making process:

“…clinic staff, budget staff and … coordination, all three … were working together to come up with these recommendations of how to overcome the challenges at the clinic level. So, I think the recommendations came mostly from the clinic staff …but it was a collective decision. [ID31]

Infection prevention and control

Interview participants described rapidly introducing COVID-19 risk mitigation measures, including IPC protocols, such as the use of personal protective equipment (PPE), hand hygiene, and social distancing, and training on the clinical management of COVID-19. A number of participants noted there were supply delays in some circumstances. Where organisations initially suspended DM/HTN services, shortages in PPE (14%) was the most commonly reported reason. Masks and PPE were introduced as soon as supplies were available and were often provided by international non-governmental organisations (NGOs) and United Nations (UN) organisations, who stepped in when national supply chains were inadequate or too slow.

Prioritisation of people with NCDs

Respondents consistently reported that their organisations, unlike many national governments, recognised the increased risk PLWNCDS faced, and the need to prioritise their continuity of care. Organisations took varying approaches to social distancing to protect and prioritise PLWNCD and staff. For example, in some contexts, outdoor waiting areas were created, and temperature checks and triage of PLWNCDs were introduced. PLWNCDs were often separated from other primary care patients. In many, although not all, cases, only PLWNCDs with severe or uncontrolled conditions continued to be seen at facilities, by appointment only, while those with stable conditions were advised to remain at home. In a minority of cases, facility-based consultations were maintained for all PLWNCDs, while group-based activities were adapted (Additional file 3 ).

Maintaining NCD consultations

Table 3 outlines the survey response on the change or termination of NCD programmes implemented by the respondents’ organisations. Medical consultations were largely maintained or immediately adapted – only 12% of respondents reported initially suspending and then resuming them in an adapted format. The major reasons reported for suspending consultations were government-mandated movement restrictions (33%) and PLWNCDs’ fear of face-to-face attendance (24%). These factors also reduced the numbers of consultations in the initial months.

Other NCD programme components were also adapted, either immediately or after a period of brief suspension. In most cases, disease monitoring continued unchanged (46%), and the remainder of programmes simplified or reduced monitoring frequency. The few service components that were completely stopped without resumption tended to be at the community level (2% of education and support services, and 6% of primary prevention and community screening services) or involving group-based activities or mobile units (Table  3 and Additional file 3 ).

Reducing facility-based contact

Adaptations were introduced to maintain contact when PLWNCDs could not attend facilities. Face-to-face consultations were either dropped entirely (reducing from 93 to 39%) or decreased in frequency (73%). The principle means used to maintain contact with PLWNCDs remotely were via community health workers or volunteers (CHW), and via use of telemedicine.

CHWs were involved in some aspect of NCD service provision, mainly in education and support and/or NCD prevention and screening activities (Additional file 3 ). They played a role in medical and in disease monitoring in about one fifth and one third of cases, respectively. In response to the pandemic, one fifth of respondents (21%) reported additional task sharing to community-based staff. Their role was expanded to include education around COVID-19, IPC, and vaccination, active follow up of PLWNCDs, home-based clinical and adherence monitoring, and liaison with clinicians, supporting remote management of PLWNCDs. Interview participants from diverse settings highlighted the key role that CHWs played in reaching the community and gaining real-time insights on community needs, disseminating information, and gaining community trust.

In parallel, however, participants emphasised the need for adequate and regularly updated training, communication pathways, and support for CHWs:

“We ensured CHWs (were) kept on their toes in terms of trainings and refresher, information on COVID and NCD and management of NCD within the COVID-19 pandemic. Two, we ensured that CHWs also (were) giving (clinical) information back …It’s also very important to have (a) communication system where CHWs can … share information directly to you and … tell you the situation in the community…. [ID26]

Prior to the pandemic, the survey findings suggest that telemedicine via mobile or landline telephone, WhatsApp, or video consultation, was utilized by a very small proportion of our study respondents’ organisations (Additional file 3 ). The survey results also indicate a higher use of telephones during the pandemic to provide medical consultations, disease monitoring, education and support services, and primary prevention and screening. For example, 2% of respondents reported their organisations using telephone consultations pre-pandemic, which increased to 23% during the pandemic (Fig.  2 ).

Use of technology to support medical consultations before and during the pandemic

Access to and use of blood pressure and blood sugar monitors was variable. Similarly, access to digital devices with internet connectivity such as telephone, smartphones, and tablets, to communicate remotely with health facilities varied significantly. Where there was phone and internet connectivity and access to use of smart devices, programme staff were able to engage with, and monitor PLWNCDs through online platforms. Stable PLWNCDs with controlled disease were supported to self-manage at home via phone consultations or CHW visits, and this was facilitated by PLWNCDs having home monitoring devices (blood pressure machines and glucometers). This was more common in the Middle East and North African region than in Sub Saharan Africa. Lack of available self-care resources in other settings meant that PLWNCDs were not able to monitor and manage their health within their homes. In one setting, PLWNCDs were taught to self-inject insulin rather than having to attend the facility for health workers do it.

In some instances, this change in remote consultation approach was met with initial resistance. As the approach was normalised, PLWNCDs reportedly began to prefer these modes of communication.

Communication via these platforms spanned from health education and awareness, to targeted counselling and psycho-social support, where its wide reach was deemed beneficial in reducing stigma. For example, one programme provided nurse-led psychosocial support via WhatsApp groups. Uptake was increased through the delivery of “ ice breaking ” messages and the service was offered to all PLWNCDs, and therefore engagement with the service was not associated with having a mental illness.

Several examples of the CFIR constructs reviewing and evaluating were offered by interviewees. For example, several organisations realised that their initial attempts to use internet or smart phone-based technology were hampered by PLWNCDs’ lack of or uneven access to digital infrastructure, and they reverted to using telephones or community health workers to maintain contact. One interview respondent also described realising, after a period of implementing phone consultations, that doctors required specific guidance and tools to undertake these safely and consistently.

Maintaining supply of medication and equipment

At the beginning of the pandemic, most interview participants described issues with procurement of medication and IPC equipment, and national level supply chains being diverted to the pandemic response. Supply issues were reported as the main reason some programmes initially stopped or suspended DM/HTN service. In addition, almost half (45%) reported internal supply issues within their organisation which hampered continuity of care, and one third (32%) reported introducing adaptations to medication procurement or supply in response to the pandemic.

Key adaptations to medication supply included increasing the dispensing interval to three months (32%) (following WHO guidance), allowing family and friends to pick up medications from facilities (48%), and in one case, having community health workers deliver medication to people’s homes. The reduced frequency of medication pick-ups was seen as a useful to mitigate exposure to the virus in high-risk populations and to reduce crowding, caseload, and the number of people in health facilities.

Interviewees indicated that supply chain challenges lasted up to about four months and were resolved through national and international interagency collaboration.

Maintaining the health workforce

Survey participants cited staff absence due to COVID-19-related illness or quarantine (60%), and staff burnout (49%) as key internal organisational challenges to maintaining continuity of NCD care during the pandemic. Many health care workers were diverted from their usual roles to the pandemic response, their movements were physically restricted during the “lockdowns”, and interviewees recounted their initial “ panic” and high stress levels.

Strong interorganisational collaboration, particularly within camp settings, allowed organisations to pool their human resources and “cross-cover”, for example, taking on another organisation’s PLWNCDs when they had a COVID-19 outbreak among staff. One organisation reported creating two teams of staff who worked in separate shifts, to minimise burn-out and infection risk. To alleviate these workforce challenges, several reported task-sharing within the facility (25%) and/ or to community-based staff (21%) (Additional file 4 ). Interviewees cited improved supply of PPE and the introduction of COVID-19 vaccines as pivotal changes that protected staff and reduced their fear.

Sharing information and countering misinformation

Themes around use of existing data and data sharing between organisations were generated inductively from the interviews. The importance of patient registries was clearly highlighted, since they allowed staff to track NCD patients, which enabled continuity of care, and information sharing with patients. Where the WHO Health Cluster and UNHCR coordination mechanisms were strong, particularly in camp-based settings, agencies pooled their NCD patient lists and supply data, allowing agencies to share resources and collectively respond.

Communication strategies were key throughout the pandemic response. During the initial phase of the pandemic, programmes focussed on urgently communicating the infection risks and prevention strategies, through public and programme-based communication. Additional messaging on the importance of follow-up care for NCDs was then necessary, to counter people’s fear of attending facilities. Once vaccines were introduced, a new wave of messaging was required and implemented in many of the programmes—this time on the merits and safety of COVID-19 vaccines, and to counter misinformation and vaccine myths.

“At the beginning it was very difficult. You know, the misinformation “oh the COVID-19 vaccine it makes you die.” …we worked in coordination with other health services with the refugee camp and community health volunteers conducting home visits to ensure all NCD patients (got) the vaccine… [ID09]

Community health workers, where they were active, played an important role in delivering these messages, and interviewees also reported using social media, such as Facebook and WhatsApp, SMS messages in some settings, and more traditional loudhailers to spread educational messages, where settings were conducive to this e.g. in camps.

Inner setting

Structural characteristics of the surveyed organisations – most of which were humanitarian actors used to working in volatile settings, assessing acute needs, and rapidly intervening – and their internal networks and communications were important elements in quickly responding, and iteratively adapting to the pandemic. Narratives from the interviews, which were conducted about six months after the survey took place, highlighted that after the initial uncertainty, programme staff felt better equipped to manage the evolving circumstances. Interviewees highlighted their organisations’ resilience, inherent agility, and ability to adapt, and several expressed pride in their organisation’s success in coping, and maintaining continuity of care for PLWNCDs. Furthermore, teamwork and coordination were often strengthened by the pandemic response and several respondents proposed retaining these adaptations after the pandemic.

The physical infrastructure and camp versus urban setting characteristics were highly influential. Movement within camps was less challenging than moving in and out of camps, or within urban areas, and, where host populations used health services within camps, their access was jeopardised.

Strong baseline data collection systems and processes within an organisation enabled assessment of the situation, follow-up of individual patients and data sharing with other organisations:

“In our facility, we have one dedicated register for non-communicable diseases patient… so our dedicated team, continuously (kept) tracking these patients…and we (kept) connection with our community health workers ...” [ID02]

However, other organisations felt hindered by the lack of available data and data infrastructure in planning and rolling out their response.

Generally, interviewees were receptive to the changes that had to be made in response to COVID-19, the idea of protecting PLWNCDs, while maintaining continuity of care fit with individual and organisational norms and values. Interviewees generally felt they had support and feedback from managers. However, many described undertaking additional tasks with a reduced workforce and staff burnout as a prominent theme in both survey and interviews. Some participants also described a lack of “back-up” emergency plans, including alternative workflow plans when staffing was short.

Views on PPE training were mixed; some described it as delayed or improperly carried out. There were also contrasting accounts of CHW training, which was poor in some settings and highly successful with bespoke CHW training packages being developed in other settings. Overall, quick development and dissemination of training programmes, including for non-medical and CHWs, often through online/remote modules from various international and local health actors were recognised as an important enabling factor in continuing NCD care in a safe manner:

“All health workers had training about the IPC measures during COVID-19, and how to deal with patients. This was online training… done at the beginning of the crisis, through the WHO…on their website….” [ID09]

Outer setting

Participants were asked about their awareness of PLWNCDs’ needs and resources and their attempts to prioritise them. Survey respondents cited physical restrictions (88%), social restrictions (60%), fear of attending health services (54%), financial hardship (49%), and poor mental health (42%) as the key challenges faced by PLWNCDs during the pandemic (Additional file 4 ). They attempted to overcome them by introducing remote modalities for consultations and monitoring, and strong, agile messaging campaigns.

As anticipated, respondents highlighted established structural and infrastructural challenges in providing NCD care that existed before the pandemic, including a lack of NCD policy and funding and national economic pressures. More general challenges faced by humanitarians operating within an emergency response, such as fragmented health systems, with pluralistic actors, sometimes operating in vertical programmes with limited integration, were also noted.

The degree to which an organisation was networked with other external organisations ( cosmopolitanism within CFIR) proved a crucial enabler in rapidly adapting and maintaining care for PLWNCDs during the pandemic, and a key theme that was identified from surveys and interview data. Interviewees described utilising pre-existing networks of health actors and WHO-led health cluster meetings, especially in camp-based settings, with a significant strengthening of these relationships, and day-to-day collaboration increasing far beyond pre-pandemic levels. Examples of this included creating a master list of NCD patients within camps, cross covering each other’s operations and borrowing each other’s resources, including health workers, medical supplies, and community volunteer networks. These networks offered key support and a degree of peer pressure or competitive pressure to implement interventions.

One example of a new cross-sectoral collaboration was offered, whereby a health organisation repurposed a CHW network, which was usually involved in protection activities, to engage in active follow up of PLWNCDs. Government stewardship and leadership were also highlighted as key enablers to rapid response and adaptation.

External policies and incentives played a key role as either barriers or enablers. The lack of national-level emergency preparedness plans and mechanisms for coordination between health actors were highlighted by many respondents. Narratives around the early instructions from various Ministries of Health suggest a strong initial focus on infection control, and de-prioritisation of other services, including those for chronic disease:

"COVID took all the, let's say the light and only cases with COVID were prioritized. So no, I think NCDs were pulled back during the pandemic." [ID09]

A lack of pre-existing national-level policies and funding for NCDs, followed by the diversion of funding and staff time in public facilities to infection control measures and COVID-19 treatment hampered the continuity of NCD services and referrals. External policies by partner hospitals or health facilities also influenced the continuity of some NCD programme components. For example, non-emergency referrals to secondary and tertiary care hospitals were often postponed.

Other potential adaptations to reduce facility-based contact for PLWNCDs were hindered by the lack of enabling policies and national infrastructure. For example, policy barriers prevented longer-term dispensing of medicines in some contexts, and the lack of legal mechanisms to enable task sharing or telehealth consultations limited adaptions of service delivery in others. The baseline utility and availability of technology in the local context was a clear influence on the remote care modalities that could be introduced. Respondents reported a lack of national infrastructure to facilitate virtual or remote health activities prior to the pandemic, including for consultations, prescriptions, and medication delivery. Thus, while organisations were initially advised to use social media, smartphones etc., many found that this was unrealistic in their settings.

Persistent advocacy and engagement with Ministries of Health was successful in changing the policy approach towards NCD services and dispensing of medicines. Respondents suggested further advocacy was needed with governments to include NCDs as priority conditions in future emergency response, to allow for longer dispensing intervals to reduce the burden of facility attendance, and to build on technology and infrastructure to allow for remote consultation and dispensing. In Table 4 , below, we summarise our findings around the contextual factors, intervention characteristics and other barriers and enablers that influenced the continuity of NCD care in humanitarian settings during the COVID-19 pandemic. We also note our study participants' recommendations for action to maintain NCD care continuity during future crises (Table 4 ).

To our knowledge, this is one of the first studies to document factors affecting the implementation of NCD care in LMIC humanitarian settings during the COVID-19 pandemic [ 25 ]. A key finding was that NCD services were largely maintained throughout the pandemic response. Respondents’ organisations minimised interruptions to NCD care, while mitigating the risks of COVID-19, by adapting to enable remote care and reduce facility-based contact. Our study respondents highlighted how the pandemic response exacerbated the pre-existing challenges they faced in delivering NCD care in crisis-affected countries. Most humanitarian actors operate in fragile LMIC settings, where health systems are often under-resourced and fragmented, and where national-level emergency preparedness and response mechanisms may be limited. Reflecting the experience in other parts of the world, our data highlighted that initial COVID-19 responses seemed to de-prioritise PLWNCDs, health system resources were diverted away from NCD care and, especially in many LMIC settings, access to pandemic mitigation strategies, PPE and vaccines was frequently delayed [ 11 ]. Maintaining NCD care during the pandemic was also hampered by the lack of pre-existing policy or infrastructure to support remote care modalities, the fear and misinformation around COVID-19, and the initial resistance to remote care expressed by PLWNCDs.

Despite the challenges, humanitarian actors were adept at implementing context-adapted changes to support continuity of NCD services, which is consistent with findings from a similar study [ 25 ]. The humanitarian system’s in-built flexibility and agility, existing humanitarian coordination mechanisms, and strong experience communicating with PLWNCDs and advocating with authorities were all supportive factors. The UN agency coordination mechanisms, including the WHO health cluster approach and UNCHR working groups enabled quick coordination and sharing or repurposing of partner resources. When it was available, strong data collection on NCDs, such as patient registries and supply monitoring, underpinned this effective interagency coordination. Humanitarians’ experience with previous outbreaks, such as cholera and Ebola, while different, may have allowed them to react in a more agile manner than national health systems could. In keeping with this, LMIC countries that were most successful in their pandemic response built on prior outbreak experience and on existing community resources, including community health workers [ 14 ].

The key role of community health workers and volunteers in facilitating continuity of NCD care, sharing key information, and building trust among communities stood out in our data. This is consistent with other studies, which found that, with adequate and timely resources, including adapted protocols, training, and PPE, pre-existing CHW programmes were able to continue with minimal disruption during the pandemic [ 15 , 33 ] . The key part CHWs played in many of the pandemic responses recounted here reflects their pre-existing role in refugee camp settings and within Sub-Saharan African and in Southeast Asian health systems. By contrast, the role is not often utilised in the Middle East and North Africa, and it has been highlighted as a potential area for development [ 35 ]. There is growing evidence for the positive impact of CHWs on NCD management both in stable LMIC settings, and in maintaining services during periods of disruption [ 36 , 37 , 38 , 39 , 40 , 41 ]. However, in expanding this role in future NCD programmes, lessons must be learned around the need to adequately support CHWs with resources, supervision and training [ 42 ].

Telehealth, defined as “the combined use of the internet and information technology for clinical and organisational purposes, both locally and remotely”, has been touted as one innovative approach to maintaining continuity of care for PLWNCDs that should be retained and built upon post-pandemic [ 43 , 44 ]. According to the WHO, telemedicine and patient triage were the most common mitigation strategies used to reduce NCD service disruption in the early days of the pandemic [ 17 ]. However, our study reflects the literature around the introduction of telehealth – its success is highly contingent on national infrastructure, smartphone ownership rates, and internal organisational factors. Moreover, clear guidance, training and culturally-congruent communication all support its successful implementation [ 45 ]. Our data also highlight the need for guidance for clinicians in the use of telemedicine, in keeping with previous calls for specific WHO guidance on the development and use of digital health solutions for NCD care [ 20 ]. Narratives from this study suggest that the wider use of self-care, via home-based monitoring equipment, coupled with tele-health or CHW networks may be beneficial. These modalities may increase access to care, especially in crisis settings, where populations may be cut off from facilities, or where populations are marginalised or hard to reach. However, their cost effectiveness, acceptability and feasibility in different contexts must be tested with robust implementation research [ 46 , 47 ].

Introducing telemedicine may increase health inequalities [ 42 ]. Throughout the pandemic, the use of digital health for NCDs has not been equitable across world regions, disease types, or populations [ 43 ]. Indeed, the COVID-19 pandemic has highlighted and entrenched existing global inequalities - essential health workers, migrants, refugees and other displaced or marginalised populations, and those living with NCDs were among the groups most burdened by its effects [ 14 ]. It shone a spotlight on the global NCD epidemic and the enormous negative health, social and financial effects NCDs bring, the magnitude of which far outweighs that of the pandemic [ 48 ].

Implications and recommendations for practice and policy

Humanitarian actors and health systems continue to learn lessons from the COVID-19 response that may enhance models of NCD care. Our data support calls for more person-centred, community-based care that limits facility-based contact. Developing such models would be useful beyond the pandemic, as they bring care closer to people’s homes and communities and improve access by decreasing transport and time cost burden on vulnerable, resource-limited, and marginalised patients. They also decrease the risk of nosocomial infections, and potentially decrease the burden on health facilities and staff, allowing more time to be spent on quality care. The means of achieving this must be adapted to the context, but may include increased use of community health workers, telephone consultations, home-based disease monitoring and adapted dispensing practices. The potential for social media and CHW networks to spread reliable health messaging was also highlighted in our study. We recommend that new or adapted models of care should be co-developed with PLWNCDs, and evaluated for cost-effectiveness, using implementation research approaches. Training on NCDs and adequate supervision and funding is needed for health care providers – including CHWs – to build and retain their role in supporting communities. Increased funding and advocacy for the inclusion of NCDs in emergency preparedness and response is essential. Finally, we recommend further implementation research to evaluate some of the adaptations described here, for example, CHW- an/or tele-health supported self-care.

The COVID-19 pandemic exposed how underprepared the health systems of many countries were to respond to the global NCD epidemic. For example, only 42% of low-income countries included the continuity of NCD services in their national COVID-19 plan [ 20 ]. WHO has highlighted steps to “build back better” NCD services post-pandemic, such as including NCDs in national emergency response and preparedness plans, and strengthening baseline NCD data collection and NCD supply management systems [ 49 ]. In keeping with the “health for all” paradigm, NCDs should be integrated into strengthened primary health care within a universal health care approach, and access must be extended to people who are forcibly displaced by humanitarian crises.

Strengths and limitations

This study was designed in the early days of the pandemic to gain insights that could be useful to humanitarians as they rolled out their responses. Engagement with an expert advisory committee, and pre-existing relationships with global humanitarian actors, provided access to respondents from multiple global regions. The survey and interviews took place at different time points in the pandemic, enabling the generation of insights relating to different response phases. Analysis was guided by an implementation study framework, which helped synthesise findings from diverse contexts.

However, the survey was not designed to identify the number of unique programmes, nor was it designed to detect differences in service delivery approaches before and during pandemic with statistical power. We cannot comment on the actual level of service use, on how it may have changed, nor on what impact any of the documented adaptations may have had on clinical outcomes, including complication rates and mortality.

We note that our survey’s initial convenience sampling approach, via study partners and existing networks, facilitated reaching major international humanitarian actors, such as UNHCR, but resulted in few local NGOs being included. This sampling frame meant that most survey participants worked in camp settings, despite most refugees and other forcibly displaced populations now living in urban, integrated settings [ 50 ]. The findings around enhanced communication and collaboration may, therefore, be less generalisable to non-camp-based settings. Despite producing a version in Spanish to encourage responses from South America, we had few responses from the Americas and from the Western Pacific. This was presumably because the major relevant NGOs had limited operations in these regions. Offering the survey in French and Arabic, for example, may have increased responses from other regions. Fewer than half of the invited interviewees accepted to participate, possibly because they were still actively involved in the pandemic response. We also acknowledge that PLWNCDs themselves were not included as participants in this study and recommend further research to learn from and respond to their experiences of the pandemic.

The lessons around factors affecting continuity of care for NCDs and successful adaptations to care delivery in the context of COVID-19 are important for preparing for future health service disruptions, including in contexts experiencing ongoing crises or where marginalised or vulnerable communities have limited access to care. Our study findings reenforce global calls for more investment, strengthened partnerships and greater integration of NCDs into emergency preparedness, and building of resilient health systems.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Data availability

The data that support the findings of this study are not openly available due to reasons of sensitivity and are available from the corresponding author upon reasonable request. Data are located in controlled access data storage at the London School of Hygiene & Tropical Medicine.

Abbreviations

Community Health Worker or Volunteer

Consolidated Framework for Implementation Research

SARS CoV-2 Coronavirus

Diabetes Mellitus

Global Alliance for Chronic Diseases

• Hypertension

Infection Prevention and Control

Low- and Middle-Income Country

London School of Hygiene & Tropical Medicine

Non-communicable Diseases

Non-governmental Organisation

People Living with Non-communicable Disease

Personal Protective Equipment

United Nations

United Nations High Commissioner for Refugees

World Health Organization

Al-Oraibi A, Nellums LB, Chattopadhyay K. COVID-19, conflict, and non-communicable diseases among refugees. eClinicalMedicine. 2021;34. Available from: https://www.thelancet.com/journals/eclinm/article/PIIS2589-5370(21)00093-6/fulltext . Cited 14 Feb 2023.

Booth A, Reed AB, Ponzo S, Yassaee A, Aral M, Plans D, et al. Population risk factors for severe disease and mortality in COVID-19: A global systematic review and meta-analysis. PLoS ONE. 2021;16:e0247461.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Kluge HHP, Wickramasinghe K, Rippin HL, Mendes R, Peters DH, Kontsevaya A, et al. Prevention and control of non-communicable diseases in the COVID-19 response. The Lancet. 2020;395:1678–80.

Article   CAS   Google Scholar  

Sheldon TA, Wright J. Twin epidemics of covid-19 and non-communicable disease. BMJ. 2020;369: m2618.

Article   PubMed   Google Scholar  

Horton R. Offline: COVID-19 is not a pandemic. The Lancet. 2020;396:874.

World Health Organization. Non communicable diseases: Key Facts. 2022. Available from: https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases . Cited 31 Oct 2022.

Jawad M, Vamos EP, Najim M, Roberts B, Millett C. Impact of armed conflict on cardiovascular disease risk: a systematic review. Heart Br Card Soc. 2019;105:1388–94.

Google Scholar  

Ngaruiya C, Bernstein R, Leff R, Wallace L, Agrawal P, Selvam A, et al. Systematic review on chronic non-communicable disease in disaster settings. BMC Public Health. 2022;22:1234.

Article   PubMed   PubMed Central   Google Scholar  

UNHCR. UNHCR Global Trends 2023. UNHCR. 2023. Available from: https://www.unhcr.org/ie/global-trends . Cited 17 Feb 2024.

OCHA. OCHA’s Strategic Plan 2023–2026: Transforming Humanitarian Coordination | OCHA. 2023. Available from: https://www.unocha.org/publications/report/world/ochas-strategic-plan-2023-2026-transforming-humanitarian-coordination . Cited 13 Dec 2023.

UNHCR. Global Trends Report 2022. UNHCR. 2022 [cited 2023 Dec 13]. Available from: https://www.unhcr.org/global-trends-report-2022 . Cited 13 Dec 2023.

World Health Organisation. WHO | Global Action Plan for the Prevention and Control of NCDs 2013–2020. WHO. 2015. Available from: https://www.who.int/nmh/events/ncd_action_plan/en/ . Cited 17 Feb 2015.

Ansbro É, Issa R, Willis R, Blanchet K, Perel P, Roberts B. Chronic NCD care in crises: A qualitative study of global experts’ perspectives on models of care for hypertension and diabetes in humanitarian settings. J Migr Health. 2022;5:100094.

Sachs JD, Karim SSA, Aknin L, Allen J, Brosbøl K, Colombo F, et al. The Lancet Commission on lessons for the future from the COVID-19 pandemic. The Lancet. 2022;400:1224–80.

Article   Google Scholar  

Luciani S, Caixeta R, Chavez C, Ondarsuhu D, Hennis A. What is the NCD service capacity and disruptions due to COVID-19? Results from the WHO non-communicable disease country capacity survey in the Americas region. BMJ Open. 2023;13: e070085.

Yaacoub S, Zmeter C, Abbas LA, Leresche E, Kdouh O, Hammoud R, et al. Has the COVID-19 pandemic changed the utilization and provision of essential health care services from 2019 to 2020 in the primary health care network in Lebanon? Results from a nationwide representative cross-sectional survey. PLoS ONE. 2023;18: e0288387.

World Health Organization. Pulse survey on continuity of essential health services during the COVID-19 pandemic: interim report, 27 August 2020. 2020. Available from: https://www.who.int/publications-detail-redirect/WHO-2019-nCoV-EHS_continuity-survey-2020.1 . Cited 31 Oct 2022.

World Health Organization. Non-communicable Diseases Progress Monitor 2022. 2022. Available from: https://www.who.int/publications-detail-redirect/9789240047761 . Cited 12 Nov 2022.

Nikoloski Z, Alqunaibet AM, Alfawaz RA, Almudarra SS, Herbst CH, El-Saharty S, et al. Covid-19 and non-communicable diseases: evidence from a systematic literature review. BMC Public Health. 2021;21:1068.

The World Health Organization. The impact of the COVID-19 pandemic on noncommunicable disease resources and services: results of a rapid assessment. 2020 Sep. Available from: https://www.who.int/publications-detail-redirect/9789240010291

Holland D, Heald AH, Stedman M, Hanna F, Wu P, Duff C, et al. Assessment of the effect of the COVID-19 pandemic on UK HbA1c testing: implications for diabetes management and diagnosis. J Clin Pathol. 2021; Available from: https://jcp.bmj.com/content/early/2021/10/12/jclinpath-2021-207776 . Cited 15 Feb 2023.

Carr MJ, Wright AK, Leelarathna L, Thabit H, Milne N, Kanumilli N, et al. Impact of COVID-19 on diagnoses, monitoring, and mortality in people with type 2 diabetes in the UK. Lancet Diabetes Endocrinol. 2021;9:413.

Del Pinto R, Ferri C, Mammarella L, Abballe S, Dell’Anna S, Cicogna S, et al. Increased cardiovascular death rates in a COVID-19 low prevalence area. J Clin Hypertens. 2020;22:1932–5.

Wright FL, Cheema K, Goldacre R, Hall N, Herz N, Islam N, et al. Effects of the COVID-19 pandemic on secondary care for cardiovascular disease in the UK: an electronic health record analysis across three countries. Eur Heart J - Qual Care Clin Outcomes. 2023;9:377–88.

PubMed   Google Scholar  

Miller L, Alani AH, Avril N, Jingree ML, Atwiine AB, Amire KA, et al. Adaptation of care for non-communicable diseases during the COVID-19 pandemic: a global case study. BMJ Glob Health. 2022;7: e006620.

World Health Organization. Strengthening health emergency prevention, preparedness, response and resilience. 2023. Available from: https://cdn.who.int/media/docs/default-source/emergency-preparedness/who_hepr_wha2023-21051248b.pdf?sfvrsn=a82abdf4_3&download=true . Cited 17 Feb 2024.

Jaung MS, Willis R, Sharma P, Aebischer Perone S, Frederiksen S, Truppa C, et al. Models of care for patients with hypertension and diabetes in humanitarian crises: a systematic review. Health Policy Plan. 2021;36:509–32.

Kessner DM, Kalk CE, Singer J. Assessing Health Quality — The Case for Tracers. N Engl J Med. 1973;288:189–94.

Article   PubMed   CAS   Google Scholar  

Lindström K, Berg L, Rylander B, Hagman A, Olsson L, Bengtsson C. A model for quality assessment in primary health care using the tracer condition technique with insulin treated diabetes as one of the tracers. Scand J Prim Health Care. 1997;15:92–6.

Nolte E, Bain C, McKee M. Diabetes as a Tracer Condition in International Benchmarking of Health Systems. Diabetes Care. 2006;29:1007–11.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

Research Team-Center for Clinical Management Research. The Consolidated Framework for Implementation Research – Technical Assistance for users of the CFIR framework. 2022. Available from: https://cfirguide.org/ . Cited 11 Nov 2022.

StataCorp. Stata Statistical Software: Release 18. College Station, TX: StataCorp LLC.; 2023. https://www.stata.com/support/faqs/resources/citing-softwaredocumentation-faqs/ .

Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117.

Cragg, L, Davies, M, Macdowall W. Health Promotion Theory. 2013. Available from: https://www.mheducation.co.uk/health-promotion-theory-9780335263202-emea-group . Cited 17 Feb 2024.

Washington CH, Tyler FJ, Davis J, Shapiro DR, Richards A, Richard M, et al. Trauma training course: innovative teaching models and methods for training health workers in active conflict zones of Eastern Myanmar. Int J Emerg Med. 2014;7:46.

Perri HB, Ed. National Community Health Worker Programs from Afghanistan to Zimbabwe. 2020. https://chwcentral.org/resources/health-for-the-people%E2%80%8B-national-community-health-worker-programs-from-afghanistan-to-zimbabwe . Cited 17 Feb 2024.

Farzadfar F, Murray CJ, Gakidou E, Bossert T, Namdaritabar H, Alikhani S, et al. Effectiveness of diabetes and hypertension management by rural primary health-care workers (Behvarz workers) in Iran: a nationally representative observational study. The Lancet. 2012;379:47–54.

Newman PM, Franke MF, Arrieta J, Carrasco H, Elliott P, Flores H, et al. Community health workers improve disease control and medication adherence among patients with diabetes and/or hypertension in Chiapas, Mexico: an observational stepped-wedge study. BMJ Glob Health. 2018;3: e000566.

Neupane B. Integrating nutrition in local governance structures: An example from suaahara program Nepal. FASEB J Conf Exp Biol. 2015;29.1. Supplement.741.3.  https://doi.org/10.1096/FASEBJ.29.1_SUPPLEMENT.741.3 .

National Health Mission India. Module for Multi-Purpose Workers (MPW) - Female/Male on Prevention, Screening and Control of Common Non-Communicable Diseases. Available from: https://main.mohfw.gov.in/sites/default/files/Module%20for%20Multi-Purpose%20Workers%20-%20Prevention%2C%20Screening%20and%20Control%20of%20Common%20NCDS_2.pdf . Cited 17 Feb 2024.

Salve S, Raven J, Das P, Srinivasan S, Khaled A, Hayee M, et al. Community health workers and Covid-19: Cross-country evidence on their roles, experiences, challenges and adaptive strategies. PLOS Glob Public Health. 2023;3: e0001447.

Bouabida K, Lebouché B, Pomey M-P. Telehealth and COVID-19 Pandemic: An Overview of the Telehealth Use, Advantages, Challenges, and Opportunities during COVID-19 Pandemic. Healthcare. 2022;10:2293.

Abd-Alrazaq A, Hassan A, Abuelezz I, Ahmed A, Alzubaidi MS, Shah U, et al. Overview of Technologies Implemented During the First Wave of the COVID-19 Pandemic: Scoping Review. J Med Internet Res. 2021;23: e29136.

Favas C, Ansbro É, Eweka E, Agarwal G, Lazo Porras M, Tsiligianni I, et al. Factors Influencing the Implementation of Remote Delivery Strategies for Non-Communicable Disease Care in Low- and Middle-Income Countries: A Narrative Review. Public Health Rev. 2022;0. Available from: https://www.ssph-journal.org/articles/ https://doi.org/10.3389/phrs.2022.1604583/full . Cited 17 Aug 2022.

Slama S, Kim H-J, Roglic G, Boulle P, Hering H, Varghese C, et al. Care of non-communicable diseases in emergencies. The Lancet. 2017;389:326–30.

Remme M, Narasimhan M, Wilson D, Ali M, Vijayasingham L, Ghani F, et al. Self care interventions for sexual and reproductive health and rights: costs, benefits, and financing. BMJ. 2019;365: l1228.

Pan X-F, Yang J, Wen Y, Li N, Chen S, Pan A. Non-Communicable Diseases During the COVID-19 Pandemic and Beyond. Eng Beijing China. 2021;7:899–902.

CAS   Google Scholar  

World Health Organization Regional Office for South-East Asia. Integration of NCD care in emergency response and preparedness. 2018. Available from: https://www.who.int/publications-detail-redirect/9789290226352 . Cited 17 Feb 2024.

Guterres A, Spiegel P. The state of the world’s refugees: adapting health responses to urban environments. JAMA. 2012;308:673–4.

Download references

Acknowledgements

We would like to acknowledge the Global Alliance for Chronic Disease Humanitarian Working Group, Dr James Smith, and our Advisory Board of humanitarian and United Nations actors: Dr Philippa Boulle, Dr Sigiriya Aebischer Perone, Dr Lilian Kiapi, Dr Mike Woodman, and Dr Slim Slama.

EA, CF, AM, PP received LSHTM salary support from Novo Nordisk AAS for this study. The co-authors received no specific funding for their role in the study. The funder had no role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and affiliations.

Department of Epidemiology of Noncommunicable Diseases, Epidemiology & Population Health, London School of Hygiene & Tropical Medicine, London, UK

Éimhín Ansbro, Lavanya Vijayasingham, Caroline Favas, Leah Sanga & Pablo Perel

Centre for Global Chronic Conditions, London, School of Hygiene & Tropical Medicine , London, UK

Éimhín Ansbro, Lavanya Vijayasingham, Caroline Favas, Jacqueline Rintjema, Leah Sanga, Adrianna Murphy & Pablo Perel

Service de Médecine Tropicale Et Humanitaire, Hôpitaux Universitaires de Genève, Geneva, Switzerland

Olivia Heller

Faculty of Law, University of Toronto, Toronto, Canada

Jacqueline Rintjema

Global Alliance for Chronic Diseases, Wellcome, London, UK

Alyssa Chase-Vilchez

Help Age International, Yangon, Myanmar

Claire Stein

Faculty of Economics and Business, University of Groningen, Groningen, The Netherlands

School of Global Development, University of East Anglia, Norwich, UK

T.H. Chan School of Public Health, FXB Center for Health and Human Rights, Harvard University, Boston, USA

Department of Health Services Research and Policy, Public Health & Policy, London School of Hygiene & Tropical Medicine, London, UK

Adrianna Murphy

You can also search for this author in PubMed   Google Scholar

Contributions

EA, CF, AM and PP conceived of and designed the study. EA, CS, ACV, RI collected data. CS, ACV, OH, LV, EA, LS analysed data. EA, OH, LV drafted the manuscript and all authors reviewed drafts.

Corresponding author

Correspondence to Éimhín Ansbro .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval to conduct this study was obtained from the LSHTM Ethical Review Committee (ID 22825). Details of the study focus were shared with participants prior to the survey and interviews. Their written informed consent was obtained before data collection commenced. Participants’ identifying information or the organisations they represent have not been included to ensure their anonymity.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ansbro, É., Heller, O., Vijayasingham, L. et al. Lessons from the COVID-19 pandemic to strengthen NCD care and policy in humanitarian settings: a mixed methods study exploring humanitarian actors’ experiences. BMC Health Serv Res 24 , 1081 (2024). https://doi.org/10.1186/s12913-024-11458-2

Download citation

Received : 29 March 2024

Accepted : 19 August 2024

Published : 17 September 2024

DOI : https://doi.org/10.1186/s12913-024-11458-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Humanitarian
• Noncommunicable
• Implementation
• Service delivery

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Main navigation

• Research & Impact
• Funding Opportunities
• Funded Projects
• Researchers & Partners
• News & Stories

Foundational Projects

• Research in Motion

The Foundational Projects funding program supports high-risk, early-stage projects with the potential to be game-changers in the fields of D2R’s areas of interest.

In the first funding cycle launched in 2024, 47 applications were received of which 17 received awards. View a summary of the review and selection process.

Funded project summaries

Deep mutagenesis approaches to decode t cell receptor-antigen recognition.

To maximize the potential of cancer vaccines, we must enhance the efficacy of T cells, key players in immune responses. Although current vaccines aim to trigger T cells against pathogens or cancer, ensuring they don’t mistakenly attack healthy cells remains a challenge. Moreover, the diversity of T cell receptors and tumor antigens complicates the design of effective and safe cancer vaccines. Our approach combines cutting-edge genome engineering with artificial intelligence to systematically screen for T cell receptors with optimal therapeutic properties. This method aims to crack the T cell receptor recognition code to expedite their optimization for individualized cancer vaccines.'

Principal Investigator: Raquel Cuella Martin (McGill University)

Co-Investigator(s): Heather Melichar (McGill University)

Collaborator(s): Benjamin Haley (Université de Montréal) and Li Xue (Radboud University)

Project duration: Two-year

D2R Axes: Clinical Research, Acceleration, and Implementation (Axis 4) Data Science, Bioinformatics, and Computing in Personalized Medicine (Axis 5)

pHREEDOM: A pH-Responsive probe to study Endosomal Escape for RNA Delivery Optimization and Monitoring

Lipid nanoparticles (LNPs) are used to deliver gene therapies like mRNA and siRNA into cells. One current limitation with LNPs is that only a small amount of the genetic material actually reaches the cell interior. This is due to inefficient "endosomal escape," where LNPs get trapped in compartments inside the cell. To better understand and improve this process, we have developed a probe to track the pH inside LNPs as they move through cells. This tool will help us study how well LNPs release their cargo, leading to better designs for gene therapies and more efficient treatments in the future.

Principal Investigator: Pieter Cullis (University of British Columbia)

Co-Investigator(s): Anna Blakney (University of British Columbia)

Collaborator(s): Sabrina Leslie (UBC)

D2R Axes: RNA Therapeutics (Axis 2) Bioprocessing, Biomanufacturing, and Nanotechnology (Axis 3)

Engineering a Circular RNA Platform for Cap-Dependent Translation

"Messenger RNA, or mRNA, and its translation into protein lies at the heart of the central dogma of molecular biology. This proposal aims to engineer the first synthetic circular RNA platform to facilitate enhanced protein production and therapeutic applications.

Our long-term objectives include exploring the effects of diverse chemical modifications on circular RNA translation and utilizing novel synthetic chemical methods to produce these molecules tailored for human therapeutics. "

Principal Investigator: Masad Damha (McGill University)

Co-Investigator(s): Sidong Huang (McGill University)

D2R Axes: RNA Therapeutics (Axis 2)

Modelling individual differences in patterns of epigenetic regulation

Advances in technologies combined with decreases in sequencing costs make it feasible to obtain multidimensional measures of epigenetic regulation. Since different aspects of regulation are captured from the range of epigenetic datatypes, in this grant we want to develop statistical methods that optimally combine this prolific, rich information to find key epigenetic patterns and how these patterns change with disease. We will develop these new methods using data generated from individuals before and after infection with influenza.

Principal Investigator: Celia Greenwood (Jewish General Hospital)

Co-Investigator(s): Josée Dupuis (McGill University) and Qihuang Zhang (McGill University)

D2R Axes: Data Science, Bioinformatics, and Computing in Personalized Medicine (Axis 5) Population Studies and Genomic Medicine (Axis 1)

A first inclusive study on the ELSI aspects of RNA technologies and therapeutics

Our project will be the first to identify ethical, legal, and social issues raised by research and medical practice involving RNA technologies. We will use a “Delphi study” method, in which experts share their views, respond to each other’s perspectives, and revise their own opinions until the group reaches consensus about the most important issues raised by RNA technology. The Delphi study will include scientists, clinicians, policymakers, bioethicists, legal scholars, and social scientists. We will also conduct a series of interviews to gather the perspectives of people representing vulnerable and marginalized groups who may be affected by these technologies.

Principal Investigator: Yann Joly (McGill University)

Collaborator(s): Thomas Duchaine (McGill University), Silvia Vidal (McGill University) and Charles Dupras (Université de Montréal)

D2R Axes: Ethical, Socioeconomic, and Cultural Dimensions in Genomic Research (Axis 6) RNA Therapeutics (Axis 2)

McGill Brand mRNA Production

This project includes mRNA production technologies and student training aimed at establishing a Quebec-based production platform. Our goal is to provide researchers from diverse communities across Canada the ability to generate novel mRNA compositions and associated intellectual property for vaccines and protein replacement therapies. Two new discovery and production platforms will be evaluated for generating natural and chemically modified mRNAs. The production will combine chemical synthesis of RNA oligonucleotides and enzymatic reactions to generate mRNAs with highly diverse chemical structures aimed at enhancing mRNA potency and stability.

Principal Investigator: Nathan Luedtke (McGill University)

Co-Investigator(s): Masad Damha (McGill University) and Maureen McKeague (McGill University)

B-cell derived extracellular vehicles with enriched miRNA contents that mitigate Th2 inflammation

The cost of care for individuals with severe asthma is alarmingly high, accounting for approximately 50% of the cost of asthma therapy in Canada. Current therapies like steroids have high toxicity, or only target individual inflammatory pathways. Biological therapies that potentially act on multiple pathways, with low toxicity, would be advantageous for severe asthma. Using our laboratory-produced B-Cell-derived extracellular vesicles (B2-EV), we simultaneously target multiple genes via a combination of miRNA. B2-EV will be tested using well-established preclinical mouse asthma models and spatial transcriptomics and proteomics to allow us to perform robust proof-of-concept experiments addressing the molecular basis for B2-EVs as a potential therapy for severe asthma.

Principal Investigator: Bruce Mazer (Research Institute of the McGill University Health Centre)

Co-Investigator(s): Janusz Rak (Research Institute of the McGill University Health Centre)

Collaborator(s): Yasser Riaz Al-Hosseini (McGill University)

Enabling low-cost, massive-scale sequencing of antigen receptors for personalized immunotherapy

T cells identify and eliminate infected and cancer cells. They recognize diseased cells via their antigen receptor. Each T cell has a unique antigen receptor. There are hundreds of millions of T cells that patrol for diseased cells. Identifying antigen receptors is expensive and low throughput. We propose new technology that will allow for marked cost reductions and a massive increase in the number of antigen receptors that can be sequenced at once. This will have a broad impact on fundamental research and clinical applications that include testing the efficacy of emerging vaccines and expediting development of adoptive cell therapies.

Principal Investigator: Heather Melichar (McGill University)

Co-Investigator(s): Judith Mandl (McGill University) and Claudia Kleinman (Lady Davis Institute/McGill University)

Collaborator(s): Benjamin Haley (Université de Montréal)

D2R Axes: Clinical Research, Acceleration, and Implementation (Axis 4) Population Studies and Genomic Medicine (Axis 1)

RNA-targeting therapeutics to address antimicrobial resistance

RNA elements known as riboswitches control genes in bacteria and have not been found in humans. There are ongoing efforts to develop molecules that bind to riboswitches and alter gene expression, thereby impacting growth of pathogenic bacteria. Such molecules could become the next generation of antibiotics. However, there are many unsolved challenges in targeting riboswitches including host toxicity or antimicrobial resistance by accumulating mutations in the riboswitch. To overcome these challenges, we assembled a team to design riboswitch-targeting antibiotics. Using computer simulations and high-throughput assays, we will identify molecules that bind to and alter riboswitch shape, minimizing bacterial resistance

Principal Investigator: Nicolas Moitessier (McGill University)

Co-Investigator(s): Maureen McKeague (McGill University) and Anthony Mittermaier (McGill University)

D2R Axes: RNA Therapeutics (Axis 2) Data Science, Bioinformatics, and Computing in Personalized Medicine (Axis 5)

Targeting an alternative oncogenic Human Epidermal Growth Factor Receptor 2 splice isoform

Breast cancer is one of the most prevalent cancers afflicting human female populations. Despite the development of new therapies, morbidity and mortality from breast cancer remain critical issues due to acquisition of resistance to targeted therapies. Preliminary work from our group has shown that a variant of the HER2 oncogene known as HER2Δ16 which is generated by alternative splicing resulting in skipping of Exon 16 is a potent oncogenic variant of HER2 capable of inducing multifocal metastatic in mammary tumors in Genetic Engineered Mouse Models (GEMMs). This form of HER2 lacks a small portion of its protein sequence in juxta-transmembrane region that facilitates dimerization of receptor leading to constitutive activation of HER2 tyrosine kinase. We have further shown that HER2Δ16-derived tumors are highly resistant to the Trastuzumab-derived T-DM1 conjugated antibody and small molecule kinase inhibitors implicating this isoform as a potential determinant of resistance to HER2 targeted therapies. Finally, we recently showed that expression HER2Δ16 can be detected in over 40% of HER2 positive breast cancer that is further correlated with poor clinical outcome. Despite the pressing need for reagents that specifically target this oncogenic HER2 splice isoform, to date no therapeutic agent is available that specifically targets this isoform. In this D2R proposal, we plan to identify and evaluate whether shRNAs targeting the splice junction could be used to target this oncogenic variant. In a related approach we will also identify key transacting factors regulating exon 16 skipping and assessing whether targeting these factors is a viable approach in preventing formation of the HER2Δ16 oncogenic variant. Results from these studies could alter current clinical practice with HER2-targeted therapies.

Principal Investigator: William Muller (McGill University)

Collaborator(s): Sidong Huang (McGill University)

Defining the role of arginine methyltransferases in the regulation of intronic circular non-coding RNAs in diseases

Circular intronic RNAs (ciRNAs) are single-stranded, covalently closed RNAs derived from intron lariats circularized by 2′–5′ junctions. Although their functions are still largely unexplored, aberrant expression of certain ciRNAs has been recently found in different human diseases. We discovered that the protein arginine methyltransferase 5 (PRMT5), a promising cancer drug target, plays an important role for the regulation of ciRNA expression. We will explore potential applications of PRMT5-regulated ciRNAs as non-invasive diagnostic and prognostic biomarkers for cancers and therapeutic reagents in human diseases such as Amyotropic Lateral Sclerosis (ALS) and cancer.

Principal Investigator: Stéphane Richard (Jewish General Hospital)

D2R Axes: Population Studies and Genomic Medicine (Axis 1) RNA Therapeutics (Axis 2)

Single-cell and spatial profiling of RNA splicing to understand tumour heterogeneity

Abnormalities in how genes are processed, particularly through a process called splicing, play a key role in many diseases, including cancer. This project aims to develop new tools to study gene splicing at the single-cell level, which will help us better understand how it varies between individual cells in a tumor. By combining advanced sequencing technologies with novel computational algorithms, we will create a detailed map of splicing in kidney cancer cells. This approach will provide new insights into how splicing influences cell behavior in cancer and could lead to better understanding and treatments for this disease.

Principal Investigator: Hamed Shateri Najafabadi (McGill University)

Co-Investigator(s): Yasser Riazalhosseini (McGill University) and Ioannis Ragoussis (McGill University)

Collaborator(s): Hani Goodarzi (University of California San Francisco)

D2R Axes: Data Science, Bioinformatics, and Computing in Personalized Medicine (Axis 5) RNA Therapeutics (Axis 2)

DNA Nanoparticles for Targeted Therapy Against Acute Myeloid Leukemia

Acute myeloid leukemia (AML) is an aggressive blood cancer affecting around 1300 Canadians annually, with current therapies only curing 30% of patients. We have developed safe, controllable delivery vehicles for nucleic acid therapeutics, showing enhanced activity in many animal cancer models. We propose to develop nucleic acid therapies to target AML stem cells, which contribute to drug resistance and relapse. These structures will carry cancer therapeutics specifically to AML cells, sparing healthy cells and reducing toxic side effects, improving treatment efficacy and minimizing relapse.

Principal Investigator: Hanadi Sleiman (McGill University)

Co-Investigator(s): Francois Mercier (McGill University, Lady Davis Institute)

D2R Axes: Bioprocessing, Biomanufacturing, and Nanotechnology (Axis 3) RNA Therapeutics (Axis 2)

Live Imaging of mRNA 5′ cap Interaction with translation initiation factors

In eukaryotic cells, the process of mRNA translation (making proteins from mRNA) begins when a specific protein complex called eIF4F interacts with the "cap" at the beginning of mRNA. Our research aims to examine how individual proteins involved in translation interact with the mRNA cap in real-time inside cells. Additionally, we will study how viral proteins, and other factors interfere with this process. The goal is to use this information to improve the design and effectiveness of future mRNA vaccines.

Principal Investigator: Nahum Sonenberg (McGill University)

Co-Investigator(s): Maria Vera Ugalde (McGill University) and Paul Wiseman (McGill University)

Collaborator(s): Jacek Jemielity (University of Warsaw)

Iron/siRNa-Based Dynamic Delivery System Boosting Ferroptototic Cell Death in Metastatic Melanoma

Melanoma has traditionally been resistant to chemotherapy. In addition, acral and uveal melanoma subtypes are typically insensitive to immune checkpoint inhibitor therapeutics, where the 5-year survival rates are <30%. While, the 6.5-year survival rates (57-46%) have improved for patients with advanced cutaneous disease, many still do not respond, thereby necessitating the discovery and development of better therapeutics to significantly improve patient survival. Our goal for this proposal is to develop highly potent RNA-based therapeutics to introduce ferroptosis as a treatment modality for patients with unresectable melanoma for whom surgery is not an option or for whom checkpoint inhibitors are contraindications.

Principal Investigator: Maryam Tabrizian (McGill University)

Co-Investigator(s): Danuta Radzioch (McGill University), Sonia V. del Rincón (McGill University) and David Juncker (McGill University)

Collaborator(s): André Charette (Université de Montréal)

Dominant Negative RNA: A novel approach to CFTR gene therapy

CF is a protein trafficking disease where the mutant, but otherwise functional gene is recognized by the cell’s quality control system. We have identified genes that correct the trafficking defect of mutant CFTR. We have shown that the inactive versions of these genes are the active agents in correcting mutant CFTR The uniqueness of this approach is that that the inactive versions of the two genes correct the trafficking of mutant CFTR trafficking. We have shown that the shRNA in lentivirus versions of these genes can efficiently correct CFTR trafficking. we are currently refining their potential for CF gene therapy in cells from cystic fibrosis patients.

Principal Investigator: David Thomas (McGill University)

Co-Investigator(s): John Hanrahan (McGill University)

Collaborator(s): Michelle Scott (Université de Sherbrooke)

Using native gut bacteria to release RNA therapeutics against intestinal nematode infections

Parasitic helminths inflict neglected tropical diseases upon humans and cause substantial losses in agriculture. Despite efforts, control interventions still face major challenges, exacerbated by the rapid emergence of drug resistance, particularly among livestock parasites. Therefore, it is imperative to explore alternative treatments for these infections. Leveraging emerging evidence of inter-kingdom extracellular vesicle exchange and our recent advancements in engineering native transgenic bacteria capable of engrafting in their original mammalian hosts, I aim to investigate the potential of bacteria-produced noncoding RNAs as anthelmintic agents. The objectives of this research endeavor include: i) Developing native E. coli tools to specifically target helminths both in vitro and in vivo at the site of infection, and ii) Identifying foreign noncoding RNA, originating from the host and the surrounding commensal microbiota, that interacts with worm messenger RNA, potentially regulating parasite gene expression. For modeling purposes in vitro, we will rely on Caenorhabditis elegans, and for in vitro and in vivo experiments, we will employ the mouse intestinal nematode parasite Heligmosomoides polygyrus bakeri. The innovation of this study lies in investigating the efficacy of miRNA targeting against helminth infections and utilizing commensal bacteria as carriers for anthelmintic molecules. This project harbors significant translational potential, aiming to address critical questions hindering progress in RNA therapeutics for parasitic diseases. Upon completion, this research will shed light on: i) the potential utility of antagonizing helminth miRNAs and other targets for treating or preventing helminth infections, thereby elucidating the role of miRNAs in the infection's development and maintenance, and ii) the viability of utilizing live bacteria therapeutics against helminth infections. This data will enrich our qualitative and quantitative understanding of EV and miRNA trafficking in the gut, serving as a cornerstone for future targeted investigations (e.g., on the molecules involved in spatiotemporal EV and RNA trafficking within worm tissue) and facilitating the acquisition of larger funding opportunities, such as CIHR and NIH grants."

Principal Investigator: Lucienne Tritten (McGill University)

Collaborator(s): Amir Zarrinpar (University of California San Diego) and Oliver Rossbach (Justus-Liebig University of Giessen)

D2R Axes: RNA Therapeutics (Axis 2) Data Science, Bioinformatics, and Computing in Personalized Medicine (Axis 5) , Bioprocessing, Biomanufacturing, and Nanotechnology (Axis 3)

Department and University Information

D2r | dna to rna.