critical appraisal research essay

How To Write a Critical Appraisal

A critical appraisal is an academic approach that refers to the systematic identification of strengths and weakness of a research article with the intent of evaluating the usefulness and validity of the work’s research findings. As with all essays, you need to be clear, concise, and logical in your presentation of arguments, analysis, and evaluation. However, in a critical appraisal there are some specific sections which need to be considered which will form the main basis of your work.

Structure of a Critical Appraisal

Introduction.

Your introduction should introduce the work to be appraised, and how you intend to proceed. In other words, you set out how you will be assessing the article and the criteria you will use. Focusing your introduction on these areas will ensure that your readers understand your purpose and are interested to read on. It needs to be clear that you are undertaking a scientific and literary dissection and examination of the indicated work to assess its validity and credibility, expressed in an interesting and motivational way.

Body of the Work

The body of the work should be separated into clear paragraphs that cover each section of the work and sub-sections for each point that is being covered. In all paragraphs your perspectives should be backed up with hard evidence from credible sources (fully cited and referenced at the end), and not be expressed as an opinion or your own personal point of view. Remember this is a critical appraisal and not a presentation of negative parts of the work.

When appraising the introduction of the article, you should ask yourself whether the article answers the main question it poses. Alongside this look at the date of publication, generally you want works to be within the past 5 years, unless they are seminal works which have strongly influenced subsequent developments in the field. Identify whether the journal in which the article was published is peer reviewed and importantly whether a hypothesis has been presented. Be objective, concise, and coherent in your presentation of this information.

Once you have appraised the introduction you can move onto the methods (or the body of the text if the work is not of a scientific or experimental nature). To effectively appraise the methods, you need to examine whether the approaches used to draw conclusions (i.e., the methodology) is appropriate for the research question, or overall topic. If not, indicate why not, in your appraisal, with evidence to back up your reasoning. Examine the sample population (if there is one), or the data gathered and evaluate whether it is appropriate, sufficient, and viable, before considering the data collection methods and survey instruments used. Are they fit for purpose? Do they meet the needs of the paper? Again, your arguments should be backed up by strong, viable sources that have credible foundations and origins.

One of the most significant areas of appraisal is the results and conclusions presented by the authors of the work. In the case of the results, you need to identify whether there are facts and figures presented to confirm findings, assess whether any statistical tests used are viable, reliable, and appropriate to the work conducted. In addition, whether they have been clearly explained and introduced during the work. In regard to the results presented by the authors you need to present evidence that they have been unbiased and objective, and if not, present evidence of how they have been biased. In this section you should also dissect the results and identify whether any statistical significance reported is accurate and whether the results presented and discussed align with any tables or figures presented.

The final element of the body text is the appraisal of the discussion and conclusion sections. In this case there is a need to identify whether the authors have drawn realistic conclusions from their available data, whether they have identified any clear limitations to their work and whether the conclusions they have drawn are the same as those you would have done had you been presented with the findings.

The conclusion of the appraisal should not introduce any new information but should be a concise summing up of the key points identified in the body text. The conclusion should be a condensation (or precis) of all that you have already written. The aim is bringing together the whole paper and state an opinion (based on evaluated evidence) of how valid and reliable the paper being appraised can be considered to be in the subject area. In all cases, you should reference and cite all sources used. To help you achieve a first class critical appraisal we have put together some key phrases that can help lift you work above that of others.

Key Phrases for a Critical Appraisal

• Whilst the title might suggest
• The focus of the work appears to be…
• The author challenges the notion that…
• The author makes the claim that…
• The article makes a strong contribution through…
• The approach provides the opportunity to…
• The authors consider…
• The argument is not entirely convincing because…
• However, whilst it can be agreed that… it should also be noted that…
• Several crucial questions are left unanswered…
• It would have been more appropriate to have stated that…
• This framework extends and increases…
• The authors correctly conclude that…
• The authors efforts can be considered as…
• Less convincing is the generalisation that…
• This appears to mislead readers indicating that…
• This research proves to be timely and particularly significant in the light of…

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Clin Diagn Res
• v.11(5); 2017 May

Critical Appraisal of Clinical Research

Azzam al-jundi.

1 Professor, Department of Orthodontics, King Saud bin Abdul Aziz University for Health Sciences-College of Dentistry, Riyadh, Kingdom of Saudi Arabia.

Salah Sakka

2 Associate Professor, Department of Oral and Maxillofacial Surgery, Al Farabi Dental College, Riyadh, KSA.

Evidence-based practice is the integration of individual clinical expertise with the best available external clinical evidence from systematic research and patient’s values and expectations into the decision making process for patient care. It is a fundamental skill to be able to identify and appraise the best available evidence in order to integrate it with your own clinical experience and patients values. The aim of this article is to provide a robust and simple process for assessing the credibility of articles and their value to your clinical practice.

Introduction

Decisions related to patient value and care is carefully made following an essential process of integration of the best existing evidence, clinical experience and patient preference. Critical appraisal is the course of action for watchfully and systematically examining research to assess its reliability, value and relevance in order to direct professionals in their vital clinical decision making [ 1 ].

Critical appraisal is essential to:

• Combat information overload;
• Identify papers that are clinically relevant;
• Continuing Professional Development (CPD).

Carrying out Critical Appraisal:

Assessing the research methods used in the study is a prime step in its critical appraisal. This is done using checklists which are specific to the study design.

Standard Common Questions:

• What is the research question?
• What is the study type (design)?
• Selection issues.
• What are the outcome factors and how are they measured?
• What are the study factors and how are they measured?
• What important potential confounders are considered?
• What is the statistical method used in the study?
• Statistical results.
• What conclusions did the authors reach about the research question?
• Are ethical issues considered?

The Critical Appraisal starts by double checking the following main sections:

I. Overview of the paper:

• The publishing journal and the year
• The article title: Does it state key trial objectives?
• The author (s) and their institution (s)

The presence of a peer review process in journal acceptance protocols also adds robustness to the assessment criteria for research papers and hence would indicate a reduced likelihood of publication of poor quality research. Other areas to consider may include authors’ declarations of interest and potential market bias. Attention should be paid to any declared funding or the issue of a research grant, in order to check for a conflict of interest [ 2 ].

II. ABSTRACT: Reading the abstract is a quick way of getting to know the article and its purpose, major procedures and methods, main findings, and conclusions.

• Aim of the study: It should be well and clearly written.
• Materials and Methods: The study design and type of groups, type of randomization process, sample size, gender, age, and procedure rendered to each group and measuring tool(s) should be evidently mentioned.
• Results: The measured variables with their statistical analysis and significance.
• Conclusion: It must clearly answer the question of interest.

III. Introduction/Background section:

An excellent introduction will thoroughly include references to earlier work related to the area under discussion and express the importance and limitations of what is previously acknowledged [ 2 ].

-Why this study is considered necessary? What is the purpose of this study? Was the purpose identified before the study or a chance result revealed as part of ‘data searching?’

-What has been already achieved and how does this study be at variance?

-Does the scientific approach outline the advantages along with possible drawbacks associated with the intervention or observations?

IV. Methods and Materials section : Full details on how the study was actually carried out should be mentioned. Precise information is given on the study design, the population, the sample size and the interventions presented. All measurements approaches should be clearly stated [ 3 ].

V. Results section : This section should clearly reveal what actually occur to the subjects. The results might contain raw data and explain the statistical analysis. These can be shown in related tables, diagrams and graphs.

VI. Discussion section : This section should include an absolute comparison of what is already identified in the topic of interest and the clinical relevance of what has been newly established. A discussion on a possible related limitations and necessitation for further studies should also be indicated.

Does it summarize the main findings of the study and relate them to any deficiencies in the study design or problems in the conduct of the study? (This is called intention to treat analysis).

• Does it address any source of potential bias?
• Are interpretations consistent with the results?
• How are null findings interpreted?
• Does it mention how do the findings of this study relate to previous work in the area?
• Can they be generalized (external validity)?
• Does it mention their clinical implications/applicability?
• What are the results/outcomes/findings applicable to and will they affect a clinical practice?
• Does the conclusion answer the study question?
• -Is the conclusion convincing?
• -Does the paper indicate ethics approval?
• -Can you identify potential ethical issues?
• -Do the results apply to the population in which you are interested?
• -Will you use the results of the study?

Once you have answered the preliminary and key questions and identified the research method used, you can incorporate specific questions related to each method into your appraisal process or checklist.

1-What is the research question?

For a study to gain value, it should address a significant problem within the healthcare and provide new or meaningful results. Useful structure for assessing the problem addressed in the article is the Problem Intervention Comparison Outcome (PICO) method [ 3 ].

P = Patient or problem: Patient/Problem/Population:

It involves identifying if the research has a focused question. What is the chief complaint?

E.g.,: Disease status, previous ailments, current medications etc.,

I = Intervention: Appropriately and clearly stated management strategy e.g.,: new diagnostic test, treatment, adjunctive therapy etc.,

C= Comparison: A suitable control or alternative

E.g.,: specific and limited to one alternative choice.

O= Outcomes: The desired results or patient related consequences have to be identified. e.g.,: eliminating symptoms, improving function, esthetics etc.,

The clinical question determines which study designs are appropriate. There are five broad categories of clinical questions, as shown in [ Table/Fig-1 ].

[Table/Fig-1]:

Categories of clinical questions and the related study designs.

2- What is the study type (design)?

The study design of the research is fundamental to the usefulness of the study.

In a clinical paper the methodology employed to generate the results is fully explained. In general, all questions about the related clinical query, the study design, the subjects and the correlated measures to reduce bias and confounding should be adequately and thoroughly explored and answered.

Participants/Sample Population:

Researchers identify the target population they are interested in. A sample population is therefore taken and results from this sample are then generalized to the target population.

The sample should be representative of the target population from which it came. Knowing the baseline characteristics of the sample population is important because this allows researchers to see how closely the subjects match their own patients [ 4 ].

Sample size calculation (Power calculation): A trial should be large enough to have a high chance of detecting a worthwhile effect if it exists. Statisticians can work out before the trial begins how large the sample size should be in order to have a good chance of detecting a true difference between the intervention and control groups [ 5 ].

• Is the sample defined? Human, Animals (type); what population does it represent?
• Does it mention eligibility criteria with reasons?
• Does it mention where and how the sample were recruited, selected and assessed?
• Does it mention where was the study carried out?
• Is the sample size justified? Rightly calculated? Is it adequate to detect statistical and clinical significant results?
• Does it mention a suitable study design/type?
• Is the study type appropriate to the research question?
• Is the study adequately controlled? Does it mention type of randomization process? Does it mention the presence of control group or explain lack of it?
• Are the samples similar at baseline? Is sample attrition mentioned?
• All studies report the number of participants/specimens at the start of a study, together with details of how many of them completed the study and reasons for incomplete follow up if there is any.
• Does it mention who was blinded? Are the assessors and participants blind to the interventions received?
• Is it mentioned how was the data analysed?
• Are any measurements taken likely to be valid?

Researchers use measuring techniques and instruments that have been shown to be valid and reliable.

Validity refers to the extent to which a test measures what it is supposed to measure.

(the extent to which the value obtained represents the object of interest.)

• -Soundness, effectiveness of the measuring instrument;
• -What does the test measure?
• -Does it measure, what it is supposed to be measured?
• -How well, how accurately does it measure?

Reliability: In research, the term reliability means “repeatability” or “consistency”

Reliability refers to how consistent a test is on repeated measurements. It is important especially if assessments are made on different occasions and or by different examiners. Studies should state the method for assessing the reliability of any measurements taken and what the intra –examiner reliability was [ 6 ].

3-Selection issues:

The following questions should be raised:

• - How were subjects chosen or recruited? If not random, are they representative of the population?
• - Types of Blinding (Masking) Single, Double, Triple?
• - Is there a control group? How was it chosen?
• - How are patients followed up? Who are the dropouts? Why and how many are there?
• - Are the independent (predictor) and dependent (outcome) variables in the study clearly identified, defined, and measured?
• - Is there a statement about sample size issues or statistical power (especially important in negative studies)?
• - If a multicenter study, what quality assurance measures were employed to obtain consistency across sites?
• - Are there selection biases?
• • In a case-control study, if exercise habits to be compared:
• - Are the controls appropriate?
• - Were records of cases and controls reviewed blindly?
• - How were possible selection biases controlled (Prevalence bias, Admission Rate bias, Volunteer bias, Recall bias, Lead Time bias, Detection bias, etc.,)?
• • Cross Sectional Studies:
• - Was the sample selected in an appropriate manner (random, convenience, etc.,)?
• - Were efforts made to ensure a good response rate or to minimize the occurrence of missing data?
• - Were reliability (reproducibility) and validity reported?
• • In an intervention study, how were subjects recruited and assigned to groups?
• • In a cohort study, how many reached final follow-up?
• - Are the subject’s representatives of the population to which the findings are applied?
• - Is there evidence of volunteer bias? Was there adequate follow-up time?
• - What was the drop-out rate?
• - Any shortcoming in the methodology can lead to results that do not reflect the truth. If clinical practice is changed on the basis of these results, patients could be harmed.

Researchers employ a variety of techniques to make the methodology more robust, such as matching, restriction, randomization, and blinding [ 7 ].

Bias is the term used to describe an error at any stage of the study that was not due to chance. Bias leads to results in which there are a systematic deviation from the truth. As bias cannot be measured, researchers need to rely on good research design to minimize bias [ 8 ]. To minimize any bias within a study the sample population should be representative of the population. It is also imperative to consider the sample size in the study and identify if the study is adequately powered to produce statistically significant results, i.e., p-values quoted are <0.05 [ 9 ].

4-What are the outcome factors and how are they measured?

• -Are all relevant outcomes assessed?
• -Is measurement error an important source of bias?

5-What are the study factors and how are they measured?

• -Are all the relevant study factors included in the study?
• -Have the factors been measured using appropriate tools?

Data Analysis and Results:

- Were the tests appropriate for the data?

- Are confidence intervals or p-values given?

• How strong is the association between intervention and outcome?
• How precise is the estimate of the risk?
• Does it clearly mention the main finding(s) and does the data support them?
• Does it mention the clinical significance of the result?
• Is adverse event or lack of it mentioned?
• Are all relevant outcomes assessed?
• Was the sample size adequate to detect a clinically/socially significant result?
• Are the results presented in a way to help in health policy decisions?
• Is there measurement error?
• Is measurement error an important source of bias?

Confounding Factors:

A confounder has a triangular relationship with both the exposure and the outcome. However, it is not on the causal pathway. It makes it appear as if there is a direct relationship between the exposure and the outcome or it might even mask an association that would otherwise have been present [ 9 ].

6- What important potential confounders are considered?

• -Are potential confounders examined and controlled for?
• -Is confounding an important source of bias?

7- What is the statistical method in the study?

• -Are the statistical methods described appropriate to compare participants for primary and secondary outcomes?
• -Are statistical methods specified insufficient detail (If I had access to the raw data, could I reproduce the analysis)?
• -Were the tests appropriate for the data?
• -Are confidence intervals or p-values given?
• -Are results presented as absolute risk reduction as well as relative risk reduction?

Interpretation of p-value:

The p-value refers to the probability that any particular outcome would have arisen by chance. A p-value of less than 1 in 20 (p<0.05) is statistically significant.

• When p-value is less than significance level, which is usually 0.05, we often reject the null hypothesis and the result is considered to be statistically significant. Conversely, when p-value is greater than 0.05, we conclude that the result is not statistically significant and the null hypothesis is accepted.

Confidence interval:

Multiple repetition of the same trial would not yield the exact same results every time. However, on average the results would be within a certain range. A 95% confidence interval means that there is a 95% chance that the true size of effect will lie within this range.

8- Statistical results:

• -Do statistical tests answer the research question?

Are statistical tests performed and comparisons made (data searching)?

Correct statistical analysis of results is crucial to the reliability of the conclusions drawn from the research paper. Depending on the study design and sample selection method employed, observational or inferential statistical analysis may be carried out on the results of the study.

It is important to identify if this is appropriate for the study [ 9 ].

• -Was the sample size adequate to detect a clinically/socially significant result?
• -Are the results presented in a way to help in health policy decisions?

Clinical significance:

Statistical significance as shown by p-value is not the same as clinical significance. Statistical significance judges whether treatment effects are explicable as chance findings, whereas clinical significance assesses whether treatment effects are worthwhile in real life. Small improvements that are statistically significant might not result in any meaningful improvement clinically. The following questions should always be on mind:

• -If the results are statistically significant, do they also have clinical significance?
• -If the results are not statistically significant, was the sample size sufficiently large to detect a meaningful difference or effect?

9- What conclusions did the authors reach about the study question?

Conclusions should ensure that recommendations stated are suitable for the results attained within the capacity of the study. The authors should also concentrate on the limitations in the study and their effects on the outcomes and the proposed suggestions for future studies [ 10 ].

• -Are the questions posed in the study adequately addressed?
• -Are the conclusions justified by the data?
• -Do the authors extrapolate beyond the data?
• -Are shortcomings of the study addressed and constructive suggestions given for future research?
• -Bibliography/References:

Do the citations follow one of the Council of Biological Editors’ (CBE) standard formats?

10- Are ethical issues considered?

If a study involves human subjects, human tissues, or animals, was approval from appropriate institutional or governmental entities obtained? [ 10 , 11 ].

Critical appraisal of RCTs: Factors to look for:

• Allocation (randomization, stratification, confounders).
• Follow up of participants (intention to treat).
• Data collection (bias).
• Sample size (power calculation).
• Presentation of results (clear, precise).
• Applicability to local population.

[ Table/Fig-2 ] summarizes the guidelines for Consolidated Standards of Reporting Trials CONSORT [ 12 ].

[Table/Fig-2]:

Summary of the CONSORT guidelines.

Critical appraisal of systematic reviews: provide an overview of all primary studies on a topic and try to obtain an overall picture of the results.

In a systematic review, all the primary studies identified are critically appraised and only the best ones are selected. A meta-analysis (i.e., a statistical analysis) of the results from selected studies may be included. Factors to look for:

• Literature search (did it include published and unpublished materials as well as non-English language studies? Was personal contact with experts sought?).
• Quality-control of studies included (type of study; scoring system used to rate studies; analysis performed by at least two experts).
• Homogeneity of studies.

[ Table/Fig-3 ] summarizes the guidelines for Preferred Reporting Items for Systematic reviews and Meta-Analyses PRISMA [ 13 ].

[Table/Fig-3]:

Summary of PRISMA guidelines.

Critical appraisal is a fundamental skill in modern practice for assessing the value of clinical researches and providing an indication of their relevance to the profession. It is a skills-set developed throughout a professional career that facilitates this and, through integration with clinical experience and patient preference, permits the practice of evidence based medicine and dentistry. By following a systematic approach, such evidence can be considered and applied to clinical practice.

Financial or other Competing Interests

• Literature Searching
• Steps in Conducting a Literature Search
• 1 Reflection on Research Question
• Primary and Secondary Sources
• Hierarchy of Evidence
• Keyword Identification
• Boolean Search Operators
• Subject Headings, Thesaurus and MeSH
• 4 Conducting Search

Critical Appraisal

• Critical Appraisal Tools
• 6 Documenting Search
• Literature Reviews

What is Critical Appraisal?

Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, value and relevance in a particular context. It is an essential skill for evidence-based practice as it allows people to find and use research evidence reliably and efficiently.

Key steps in critical appraisal:

1. Thoroughly understanding the research, including its aims, methodology, results and conclusions, while being aware of any limitations or potential bias.

2. Using a framework or checklist to provide structure and ensure all key points are considered. This allows you to record your reasoning behind decisions based on the research.

3. Identifying the research methods, such as study design, sample size, and data collection and analysis techniques, to assess validity and reliability.

4. Checking the results and conclusions to ensure they are justified by the data and not unduly influenced by bias.

5. Determining the relevance and applicability of the research findings to your specific context or question.

Critical appraisal skills are important as they enable you to systematically and objectively assess published papers, regardless of where they are published or who wrote them. It is crucial to avoid being misled by poor quality research and ensure that any findings used as evidence can reliably improve practice.

What are Critical Appraisal Tools?

Critical appraisal tools are instruments or checklists used to assess the methodological quality, validity, and relevance of published research studies. They provide a structured framework to evaluate various aspects of a study, such as the study design, sampling methods, data collection, statistical analysis, ethical considerations, and applicability of the results.

Key Points About Critical Appraisal Tools

They aim to assess the trustworthiness, relevance, and results of published papers by examining different components of the research process.

The content and criteria assessed by these tools can vary significantly, as there is a lack of consensus on the essential items for critical appraisal.

Many tools are study design-specific, evaluating different aspects for randomized controlled trials, observational studies, qualitative research, systematic reviews, and other study types.

Common elements appraised include sampling methods, internal validity, control of confounding factors, ethical conduct, statistical analysis, and generalizability of results.

Some tools provide an overall quality rating (e.g. high, medium, low) based on the individual item assessments.

The empirical basis for the construction and validation of many critical appraisal tools is often lacking, with limited evidence of their reliability and validity.

In summary, critical appraisal tools are structured instruments that aim to evaluate the methodological rigor and quality of research studies. They assess various aspects of the research process, but their content and criteria can vary widely due to the lack of consensus on essential items and empirical validation.

• << Previous: 5 Screening, Evaluation and Selection of Search Results
• Next: Critical Appraisal Tools >>
• Last Updated: Sep 4, 2024 11:42 AM
• URL: https://library.lsbu.ac.uk/literaturesearching

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Published: 20 January 2009

How to critically appraise an article

• Jane M Young 1 &
• Michael J Solomon 2  

Nature Clinical Practice Gastroenterology & Hepatology volume  6 ,  pages 82–91 ( 2009 ) Cite this article

52k Accesses

99 Citations

449 Altmetric

Metrics details

Critical appraisal is a systematic process used to identify the strengths and weaknesses of a research article in order to assess the usefulness and validity of research findings. The most important components of a critical appraisal are an evaluation of the appropriateness of the study design for the research question and a careful assessment of the key methodological features of this design. Other factors that also should be considered include the suitability of the statistical methods used and their subsequent interpretation, potential conflicts of interest and the relevance of the research to one's own practice. This Review presents a 10-step guide to critical appraisal that aims to assist clinicians to identify the most relevant high-quality studies available to guide their clinical practice.

Critical appraisal is a systematic process used to identify the strengths and weaknesses of a research article

Critical appraisal provides a basis for decisions on whether to use the results of a study in clinical practice

Different study designs are prone to various sources of systematic bias

Design-specific, critical-appraisal checklists are useful tools to help assess study quality

Assessments of other factors, including the importance of the research question, the appropriateness of statistical analysis, the legitimacy of conclusions and potential conflicts of interest are an important part of the critical appraisal process

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 print issues and online access

195,33 € per year

only 16,28 € per issue

Buy this article

• Purchase on SpringerLink
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Making sense of the literature: an introduction to critical appraisal for the primary care practitioner

How to appraise the literature: basic principles for the busy clinician - part 2: systematic reviews and meta-analyses

How to appraise the literature: basic principles for the busy clinician - part 1: randomised controlled trials

Druss BG and Marcus SC (2005) Growth and decentralisation of the medical literature: implications for evidence-based medicine. J Med Libr Assoc 93 : 499–501

PubMed   PubMed Central   Google Scholar  

Glasziou PP (2008) Information overload: what's behind it, what's beyond it? Med J Aust 189 : 84–85

PubMed   Google Scholar  

Last JE (Ed.; 2001) A Dictionary of Epidemiology (4th Edn). New York: Oxford University Press

Google Scholar  

Sackett DL et al . (2000). Evidence-based Medicine. How to Practice and Teach EBM . London: Churchill Livingstone

Guyatt G and Rennie D (Eds; 2002). Users' Guides to the Medical Literature: a Manual for Evidence-based Clinical Practice . Chicago: American Medical Association

Greenhalgh T (2000) How to Read a Paper: the Basics of Evidence-based Medicine . London: Blackwell Medicine Books

MacAuley D (1994) READER: an acronym to aid critical reading by general practitioners. Br J Gen Pract 44 : 83–85

CAS   PubMed   PubMed Central   Google Scholar  

Hill A and Spittlehouse C (2001) What is critical appraisal. Evidence-based Medicine 3 : 1–8 [ http://www.evidence-based-medicine.co.uk ] (accessed 25 November 2008)

Public Health Resource Unit (2008) Critical Appraisal Skills Programme (CASP) . [ http://www.phru.nhs.uk/Pages/PHD/CASP.htm ] (accessed 8 August 2008)

National Health and Medical Research Council (2000) How to Review the Evidence: Systematic Identification and Review of the Scientific Literature . Canberra: NHMRC

Elwood JM (1998) Critical Appraisal of Epidemiological Studies and Clinical Trials (2nd Edn). Oxford: Oxford University Press

Agency for Healthcare Research and Quality (2002) Systems to rate the strength of scientific evidence? Evidence Report/Technology Assessment No 47, Publication No 02-E019 Rockville: Agency for Healthcare Research and Quality

Crombie IK (1996) The Pocket Guide to Critical Appraisal: a Handbook for Health Care Professionals . London: Blackwell Medicine Publishing Group

Heller RF et al . (2008) Critical appraisal for public health: a new checklist. Public Health 122 : 92–98

Article   Google Scholar  

MacAuley D et al . (1998) Randomised controlled trial of the READER method of critical appraisal in general practice. BMJ 316 : 1134–37

Article   CAS   Google Scholar  

Parkes J et al . Teaching critical appraisal skills in health care settings (Review). Cochrane Database of Systematic Reviews 2005, Issue 3. Art. No.: cd001270. 10.1002/14651858.cd001270

Mays N and Pope C (2000) Assessing quality in qualitative research. BMJ 320 : 50–52

Hawking SW (2003) On the Shoulders of Giants: the Great Works of Physics and Astronomy . Philadelphia, PN: Penguin

National Health and Medical Research Council (1999) A Guide to the Development, Implementation and Evaluation of Clinical Practice Guidelines . Canberra: National Health and Medical Research Council

US Preventive Services Taskforce (1996) Guide to clinical preventive services (2nd Edn). Baltimore, MD: Williams & Wilkins

Solomon MJ and McLeod RS (1995) Should we be performing more randomized controlled trials evaluating surgical operations? Surgery 118 : 456–467

Rothman KJ (2002) Epidemiology: an Introduction . Oxford: Oxford University Press

Young JM and Solomon MJ (2003) Improving the evidence-base in surgery: sources of bias in surgical studies. ANZ J Surg 73 : 504–506

Margitic SE et al . (1995) Lessons learned from a prospective meta-analysis. J Am Geriatr Soc 43 : 435–439

Shea B et al . (2001) Assessing the quality of reports of systematic reviews: the QUORUM statement compared to other tools. In Systematic Reviews in Health Care: Meta-analysis in Context 2nd Edition, 122–139 (Eds Egger M. et al .) London: BMJ Books

Chapter   Google Scholar  

Easterbrook PH et al . (1991) Publication bias in clinical research. Lancet 337 : 867–872

Begg CB and Berlin JA (1989) Publication bias and dissemination of clinical research. J Natl Cancer Inst 81 : 107–115

Moher D et al . (2000) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUORUM statement. Br J Surg 87 : 1448–1454

Shea BJ et al . (2007) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology 7 : 10 [10.1186/1471-2288-7-10]

Stroup DF et al . (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 283 : 2008–2012

Young JM and Solomon MJ (2003) Improving the evidence-base in surgery: evaluating surgical effectiveness. ANZ J Surg 73 : 507–510

Schulz KF (1995) Subverting randomization in controlled trials. JAMA 274 : 1456–1458

Schulz KF et al . (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273 : 408–412

Moher D et al . (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. BMC Medical Research Methodology 1 : 2 [ http://www.biomedcentral.com/ 1471-2288/1/2 ] (accessed 25 November 2008)

Rochon PA et al . (2005) Reader's guide to critical appraisal of cohort studies: 1. Role and design. BMJ 330 : 895–897

Mamdani M et al . (2005) Reader's guide to critical appraisal of cohort studies: 2. Assessing potential for confounding. BMJ 330 : 960–962

Normand S et al . (2005) Reader's guide to critical appraisal of cohort studies: 3. Analytical strategies to reduce confounding. BMJ 330 : 1021–1023

von Elm E et al . (2007) Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 335 : 806–808

Sutton-Tyrrell K (1991) Assessing bias in case-control studies: proper selection of cases and controls. Stroke 22 : 938–942

Knottnerus J (2003) Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol 56 : 1118–1128

Furukawa TA and Guyatt GH (2006) Sources of bias in diagnostic accuracy studies and the diagnostic process. CMAJ 174 : 481–482

Bossyut PM et al . (2003)The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 138 : W1–W12

STARD statement (Standards for the Reporting of Diagnostic Accuracy Studies). [ http://www.stard-statement.org/ ] (accessed 10 September 2008)

Raftery J (1998) Economic evaluation: an introduction. BMJ 316 : 1013–1014

Palmer S et al . (1999) Economics notes: types of economic evaluation. BMJ 318 : 1349

Russ S et al . (1999) Barriers to participation in randomized controlled trials: a systematic review. J Clin Epidemiol 52 : 1143–1156

Tinmouth JM et al . (2004) Are claims of equivalency in digestive diseases trials supported by the evidence? Gastroentrology 126 : 1700–1710

Kaul S and Diamond GA (2006) Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann Intern Med 145 : 62–69

Piaggio G et al . (2006) Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA 295 : 1152–1160

Heritier SR et al . (2007) Inclusion of patients in clinical trial analysis: the intention to treat principle. In Interpreting and Reporting Clinical Trials: a Guide to the CONSORT Statement and the Principles of Randomized Controlled Trials , 92–98 (Eds Keech A. et al .) Strawberry Hills, NSW: Australian Medical Publishing Company

National Health and Medical Research Council (2007) National Statement on Ethical Conduct in Human Research 89–90 Canberra: NHMRC

Lo B et al . (2000) Conflict-of-interest policies for investigators in clinical trials. N Engl J Med 343 : 1616–1620

Kim SYH et al . (2004) Potential research participants' views regarding researcher and institutional financial conflicts of interests. J Med Ethics 30 : 73–79

Komesaroff PA and Kerridge IH (2002) Ethical issues concerning the relationships between medical practitioners and the pharmaceutical industry. Med J Aust 176 : 118–121

Little M (1999) Research, ethics and conflicts of interest. J Med Ethics 25 : 259–262

Lemmens T and Singer PA (1998) Bioethics for clinicians: 17. Conflict of interest in research, education and patient care. CMAJ 159 : 960–965

Download references

Author information

Authors and affiliations.

JM Young is an Associate Professor of Public Health and the Executive Director of the Surgical Outcomes Research Centre at the University of Sydney and Sydney South-West Area Health Service, Sydney,

Jane M Young

MJ Solomon is Head of the Surgical Outcomes Research Centre and Director of Colorectal Research at the University of Sydney and Sydney South-West Area Health Service, Sydney, Australia.,

Michael J Solomon

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jane M Young .

Ethics declarations

Competing interests.

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Young, J., Solomon, M. How to critically appraise an article. Nat Rev Gastroenterol Hepatol 6 , 82–91 (2009). https://doi.org/10.1038/ncpgasthep1331

Download citation

Received : 10 August 2008

Accepted : 03 November 2008

Published : 20 January 2009

Issue Date : February 2009

DOI : https://doi.org/10.1038/ncpgasthep1331

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Get new issue alerts Get alerts
• Submit a Manuscript

Secondary Logo

Journal logo.

Colleague's E-mail is Invalid

Your message has been successfully sent to your colleague.

Save my selection

Critical appraisal of a clinical research paper

What one needs to know.

Manjali, Jifmi Jose; Gupta, Tejpal

Department of Radiation Oncology, Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, Maharashtra, India

Address for correspondence: Dr. Tejpal Gupta, ACTREC, Tata Memorial Centre, Homi Bhabha National Institute, Kharghar, Navi Mumbai - 410 210, Maharashtra, India. E-mail: [email protected]

Received May 25, 2020

Received in revised form June 11, 2020

Accepted June 19, 2020

This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.

In the present era of evidence-based medicine (EBM), integrating best research evidence into the clinical practice necessitates developing skills to critically evaluate and analyze the scientific literature. Critical appraisal is the process of systematically examining research evidence to assess its validity, results, and relevance to inform clinical decision-making. All components of a clinical research article need to be appraised as per the study design and conduct. As research bias can be introduced at every step in the flow of a study leading to erroneous conclusions, it is essential that suitable measures are adopted to mitigate bias. Several tools have been developed for the critical appraisal of scientific literature, including grading of evidence to help clinicians in the pursuit of EBM in a systematic manner. In this review, we discuss the broad framework for the critical appraisal of a clinical research paper, along with some of the relevant guidelines and recommendations.

INTRODUCTION

Medical research information is ever growing and branching day by day. Despite the vastness of medical literature, it is necessary that as clinicians we offer the best treatment to our patients as per the current knowledge. Integrating best research evidence with clinical expertise and patient values has led to the concept of evidence-based medicine (EBM).[ 1 ] Although this philosophy originated in the middle of the 19 th century,[ 2 ] it first appeared in its current form in the modern medical literature in 1991.[ 3 ] EBM is defined as the conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of an individual patient.[ 1 ] The essentials of EBM include generating a clinical question, tracking the best available evidence, critically evaluating the evidence for validity and clinical usefulness, further applying the results to clinical practice, and evaluating its performance. Appropriate application of EBM can result in cost-effectiveness and improve health-care efficiency.[ 4 ] Without continual accumulation of new knowledge, existing dogmas and paradigms quickly become outdated and may prove detrimental to the patients. The current growth of medical literature with 1.8 million scientific articles published in the year 2012,[ 5 ] often makes it difficult for the clinicians to keep pace with the vast amount of scientific data, thus making foraging (alerts to new information) and hunting (finding answers to clinical questions) essential skills to help navigate the so-called “jungle” of information.[ 6 ] Therefore, it is essential that health-care professionals read medical literature selectively to effectively utilize their limited time and assiduously imbibe new knowledge to improve decision-making for their patients. To practice EBM in its true sense, a clinician not only needs to devote time to develop the skill of effectively searching the literature, but also needs to learn to evaluate the significance, methodology, outcomes, and transparency of the study.[ 4 ] Along with the evaluation and interpretation of a study, a thorough understanding of its methodology is necessary. It is common knowledge that studies with positive results are relatively easy to publish.[ 7 8 ] However, it is the critical appraisal of any research study (even those with negative results) that helps us to understand the science better and ask relevant questions in future using an appropriate study design and endpoints. Therefore, this review is focused on the framework for the critical appraisal of a clinical research paper. In addition, we have also discussed some of the relevant guidelines and recommendations for the critical appraisal of clinical research papers.

CRITICAL APPRAISAL

Critical appraisal is the process of systematically examining the research evidence to assess its validity, results, and relevance before using it to inform a decision.[ 9 ] It entails the following:

• Balanced assessment of the benefits/strengths and flaws/weaknesses of a study
• Assessment of the research process and results
• Consideration of quantitative and qualitative aspects.

Critical appraisal is performed to assess the following

aspects of a study:

• Validity – Is the methodology robust?
• Reliability – Are the results credible?
• Applicability– Do the results have the potential to change the current practice?

Contrary to the common belief, a critical appraisal is not the negative dismissal of any piece of research or an assessment of the results alone; it is neither solely based on a statistical analysis nor a process undertaken by the experts only. When performing a critical appraisal of a scientific article, it is essential that we know its basic composition and assess every section meticulously.

Initial assessment

This involves taking a generalized look at the details of the article. The journal it was published in holds special value – a peer reviewed, indexed journal with a good impact factor adds robustness to the paper. The setting, timeline, and year of publication of the study also need to be noted, as they provide a better understanding of the evolution of thoughts in that particular subject. Declaration of the conflicts of interest by the authors, the role of the funding source if any, and any potential commercial bias should also be noted.[ 10 ]

COMPONENTS OF A CLINICAL RESEARCH PAPER

The components of any scientific article or clinical research paper remain largely the same. An article begins with a title, abstract, and keywords, which are followed by the main text, which includes the IMRAD – introduction, methods, results and discussion, and ends with the conclusion and references.

It is a brief summary of the research article which helps the readers understand the purpose, methods, and results of the study. Although an abstract may provide a brief overview of the study, the full text of the article needs to be read and evaluated for a thorough understanding. There are two types of abstracts, namely structured and unstructured. A structured abstract comprises different sections typically labelled as background/purpose, methods, results, and conclusion, whereas an unstructured abstract is not divided into these sections.

Introduction

The introduction of a research paper familiarizes the reader with the topic. It refers to the current evidence in the particular subject and the possible lacunae which necessitate the present study. In other words, the introduction puts the study in perspective. The findings of other related studies have to be quoted and referenced, especially their central statements. The introduction also needs to justify the appropriateness of the chosen study.[ 11 ]

This section highlights the procedure followed while conducting the study. It provides all the data necessary for the study's appraisal and lays out the study design which is paramount. For clinical research articles, this section should describe the participant or patient/population/problem (P), intervention (I), comparison (C), outcome (O), and study design (S) PICO(S), generally referred to as the PICO(S) framework [ Table 1 ].

Study designs and levels of evidence

Study designs are broadly divided into descriptive and interventional studies,[ 12 ] which can be further subdivided as shown in Figure 1 . Each study design has its own characteristics and should be used in the appropriate setting. The various study designs form the building blocks of evidence. This in turn justifies the need for a hierarchical classification of evidence, referred to as “Levels of Evidence,” as it forms the cornerstone of EBM [ Table 2 ]. Most medical journals now mandate that the submitted manuscript conform to and comply with the clinical research reporting statements and guidelines as applicable to the study design [ Table 3 ] to maintain clarity, transparency, and reproducibility and ensure comparability across different studies asking the same research question. As per the study design, the appropriate descriptive and inferential statistical analyses should be specified in the statistical plan. For prospective studies, a clear mention of sample size calculation (depending on the type of study, power, alpha error, meaningful difference, and variance) is mandatory, so as to identify whether the study was adequately powered.[ 13 ] The endpoints (primary, secondary, and exploratory, if any) should be mentioned clearly along with the exact methods used for the measurement of the variables.

Statistical testing

The statistical framework of any research study is commonly based on testing the null hypothesis, wherein the results are deemed significant by comparing P values obtained from an experimental dataset to a predefined significance level (0.05 being the most popular choice). By definition, P value is the probability under the specified statistical model to obtain a statistical summary equal to or more extreme than the one computed from the data and can range from 0 to 1. P < 0.05 indicates that results are unlikely to be due to chance alone. Unfortunately, P value does not indicate the magnitude of the observed difference, which may also be desirable. An alternative and complementary approach is the use of confidence intervals (CI), which is a range of values calculated from the observed data, that is likely to contain the true value at a specified probability. The probability is chosen by the investigator, and it is set customarily at 95% (1– alpha error of 0.05). CI provides information that may be used to test hypotheses; additionally, they provide information related to the precision, power, sample size, and effect size.

This section contains the findings of the study, presented clearly and objectively. The results obtained using the descriptive and inferential statistical analyses (as mentioned in the methods section) should be described. The use of tables and figures, including graphical representation [ Table 4 ], is encouraged to improve the clarity;[ 14 ] however, the duplication of these data in the text should be avoided.

The discussion section presents the authors' interpretations of the obtained results. This section includes:

• A comparison of the study results with what is currently known, drawing similarities and differences
• Novel findings of the study that have added to the existing body of knowledge
• Caveats and limitations.

It is imperative that the key relevant references are cited in any research paper in the appropriate format which allows the readers to access the original source of the specified statement or evidence. A brief look at the reference list gives an overview of how well the indexed medical literature was searched for the purpose of writing the manuscript.

Overall assessment

After a careful assessment of the various sections of a research article, it is necessary to assess the relevance of the study findings to the present scenario and weigh the potential benefits and drawbacks of its application to the population. In this context, it is necessary that the integrity of the intervention be noted. This can be verified by assessing the factors such as adherence to the specified program, the exposure needed, quality of delivery, participant responsiveness, and potential contamination. This relates to the feasibility of applying the intervention to the community.

BIAS IN CLINICAL RESEARCH

Research articles are the media through which science is communicated, and it is necessary that we adhere to the basic principles of transparency and accuracy when communicating our findings. Any such trend or deviation from the truth in data collection, analysis, interpretation, or publication is called bias.[ 15 ] This may lead to erroneous conclusions, and hence, all scientists and clinicians must be aware of the bias and employ all possible measures to mitigate it.

The extent to which a study is free from bias defines its internal validity. Internal validity is different from the external validity and precision. The external validity of a study is about its generalizability or applicability (depends on the purpose of the study), while precision is the extent to which a study is free from random errors (depends on the number of participants). A study is irrelevant without internal validity even if it is applicable and precise.[ 16 ] A bias can be introduced at every step in the flow of a study [ Figure 2 ].

The various types of biases in clinical research include:

• Selection bias: This happens while recruiting patients. This may lead to the differences in the way patients are accepted or rejected for a trial and the way in which interventions are assigned to the individuals. We need to assess whether the study population is a true representative of the target population. Furthermore, when there is no or an inadequate sequence generation, it can result in the over-estimation of treatment effects compared to randomized trials.[ 14 ] This can be mitigated by using a process called randomization. Randomization is the process of assigning clinical trial participants to treatment groups, such that each participant has an equal chance of being assigned to a particular group. This process should be completely random (e.g., tossing a coin, using a computer program, and throwing dice). When the process is not exactly random (e.g., randomization by date of birth, odd-even numbers, alternation, registration date, etc.), there is a significant potential for a selection bias
• Allocation bias: This is a bias that sets in when the person responsible for the study also allocates the treatment. It is known that inadequate or unclear concealment of allocation can lead to an overestimation of the treatment effects.[ 17 ] Adequate allocation concealment helps in mitigating this bias. This can be done by sequentially numbering identical drug containers or through central allocation by a person not involved in study enrollment
• Confounding bias: Having an effect on the dependent and independent variables through a spurious association, confounding factors can introduce a significant bias. Hence, the baseline characteristics need to be similar in the groups being compared. Known confounders can be managed during the selection process by stratified randomization (in randomized trials) and matching (in observational studies) or during analysis by meta-regression.[ 18 ] However, the unknown confounders can be minimized only through randomization
• Performance bias: This is a bias that is introduced because of the knowledge about the intervention allocation in the patient, investigator, or outcome assessor. This results in ascertainment or recall bias (patient), reporting bias (investigator), and detection bias (outcome assessor), all of which can lead to an overestimation of the treatment effects.[ 17 ] This can be mitigated by blinding – a process in which the treatment allocation is hidden from the patient, investigator, and/or outcome assessor. However, it has to be noted that blinding may not be practical or possible in all kinds of clinical trials
• Method bias: In clinical trials, it is necessary that the outcomes be assessed and recorded using valid and reliable tools, the lack of which can introduce a method bias[ 19 ]
• Attrition bias: This is a bias that is introduced because of the systematic differences between the groups in the loss of participants from the study. It is necessary to describe the completeness of the outcomes including the exclusions (along with the reasons), loss to follow-up, and drop-outs from the analysis
• Other bias: This includes any important concerns about biases not covered in the other domains.

Trial registration

In the recent times, it has become an ethical as well as a regulatory requirement in most countries to register the clinical trials prospectively before the enrollment of the first subject. Registration of a clinical trial is defined as the publication of an internationally agreed upon set of information about the design, conduct, and administration of any clinical trial on a publicly accessible website managed by a registry conforming to international standards. Apart from improving the awareness and visibility of the study, registration ensures transparency in the conduct and reduces publication bias and selective reporting. Some of the common sites are the ClinicalTrials. gov run by the National Library of Medicine of the National Institutes of Health (), Clinical Trials Registry-India () run by the Indian Council of Medical Research, and the International Clinical Trials Registry Platform () run by the World Health Organization.

Tools for critical appraisal

Several tools have been developed to assess the transparency of the scientific research papers and the degree of congruence of the research question with the study in the context of the various sections listed above [ Table 5 ].

Ethical considerations

Bad ethics cannot produce good science. Therefore, all scientific research must follow the ethical principles laid out in the declaration of Helsinki. For clinical research, it is mandatory that team members be trained in good clinical practice, familiarize themselves with clinical research methodology, and follow standard operating procedures as prescribed. Although the regulatory framework and landscape may vary to a certain extent depending upon the country where the research work is conducted, it is the responsibility of the Institutional Review Boards/Institutional Ethics Committees to provide study oversight such that the safety, well-being, and rights of the participants are adequately protected.

CONCLUSIONS

Critical appraisal is the systematic examination of the research evidence reported in the scientific articles to assess their validity, reliability, and applicability before using their findings to inform decision-making. It should be considered as the first step to grade the quality of evidence.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

• Cited Here |
• Google Scholar

Appraisal; bias; clinical study; evidence-based medicine; guidelines; tools

• + Favorites
• View in Gallery

Readers Of this Article Also Read

Epidemiological studies of risk factors could aid in designing risk..., do you have a standard way of interpreting the standard deviation a narrative....

Please enter both an email address and a password.

Account login

• Show/Hide Password Show password Hide password
• Reset Password

Need to reset your password?  Enter the email address which you used to register on this site (or your membership/contact number) and we'll email you a link to reset it. You must complete the process within 2hrs of receiving the link.

We've sent you an email.

An email has been sent to Simply follow the link provided in the email to reset your password. If you can't find the email please check your junk or spam folder and add [email protected] to your address book.

• About RCS England

• Dissecting the literature: the importance of critical appraisal

08 Dec 2017

Kirsty Morrison

This post was updated  in 2023.

Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context.

Amanda Burls, What is Critical Appraisal?

Why is critical appraisal needed?

Literature searches using databases like Medline or EMBASE often result in an overwhelming volume of results which can vary in quality. Similarly, those who browse medical literature for the purposes of CPD or in response to a clinical query will know that there are vast amounts of content available. Critical appraisal helps to reduce the burden and allow you to focus on articles that are relevant to the research question, and that can reliably support or refute its claims with high-quality evidence, or identify high-level research relevant to your practice.

Critical appraisal allows us to:

• reduce information overload by eliminating irrelevant or weak studies
• identify the most relevant papers
• distinguish evidence from opinion, assumptions, misreporting, and belief
• assess the validity of the study
• assess the usefulness and clinical applicability of the study
• recognise any potential for bias.

Critical appraisal helps to separate what is significant from what is not. One way we use critical appraisal in the Library is to prioritise the most clinically relevant content for our Current Awareness Updates .

How to critically appraise a paper

There are some general rules to help you, including a range of checklists highlighted at the end of this blog. Some key questions to consider when critically appraising a paper:

• Is the study question relevant to my field?
• Does the study add anything new to the evidence in my field?
• What type of research question is being asked? A well-developed research question usually identifies three components: the group or population of patients, the studied parameter (e.g. a therapy or clinical intervention) and outcomes of interest.
• Was the study design appropriate for the research question? You can learn more about different study types and the hierarchy of evidence here .
• Did the methodology address important potential sources of bias? Bias can be attributed to chance (e.g. random error) or to the study methods (systematic bias).
• Was the study performed according to the original protocol? Deviations from the planned protocol can affect the validity or relevance of a study, e.g. a decrease in the studied population over the course of a randomised controlled trial .
• Does the study test a stated hypothesis? Is there a clear statement of what the investigators expect the study to find which can be tested, and confirmed or refuted.
• Were the statistical analyses performed correctly? The approach to dealing with missing data, and the statistical techniques that have been applied should be specified. Original data should be presented clearly so that readers can check the statistical accuracy of the paper.
• Do the data justify the conclusions? Watch out for definite conclusions based on statistically insignificant results, generalised findings from a small sample size, and statistically significant associations being misinterpreted to imply a cause and effect.
• Are there any conflicts of interest? Who has funded the study and can we trust their objectivity? Do the authors have any potential conflicts of interest, and have these been declared?

And an important consideration for surgeons:

• Will the results help me manage my patients?

At the end of the appraisal process you should have a better appreciation of how strong the evidence is, and ultimately whether or not you should apply it to your patients.

Further resources:

• How to Read a Paper by Trisha Greenhalgh
• The Doctor’s Guide to Critical Appraisal by Narinder Kaur Gosall
• CASP checklists
• CEBM Critical Appraisal Tools
• Critical Appraisal: a checklist
• Critical Appraisal of a Journal Article (PDF)
• Introduction to...Critical appraisal of literature
• Reporting guidelines for the main study types

Kirsty Morrison, Information Specialist

Share this page:

• Library Blog

Medicine: A Brief Guide to Critical Appraisal

• About this guide
• First Year Library Essentials
• Literature Reviews and Data Management
• Systematic Search for Health This link opens in a new window
• Guide to Using EndNote This link opens in a new window
• A Brief Guide to Critical Appraisal
• Manage Research Data This link opens in a new window
• Articles & Databases
• Point of Care Tools
• Anatomy & Radiology
• Drugs and Medicines
• Diagnostic Tests & Calculators
• Health Statistics
• Multimedia Sources
• News & Public Opinion

Have you ever seen a news piece about a scientific breakthrough and wondered how accurate the reporting is? Or wondered about the research behind the headlines? This is the beginning of critical appraisal: thinking critically about what you see and hear, and asking questions to determine how much of a 'breakthrough' something really is.

The article " Is this study legit? 5 questions to ask when reading news stories of medical research " is a succinct introduction to the sorts of questions you should ask in these situations, but there's more than that when it comes to critical appraisal. Read on to learn more about this practical and crucial aspect of evidence-based practice.

What is Critical Appraisal?

Critical appraisal forms part of the process of evidence-based practice. “ Evidence-based practice across the health professions ” outlines the fives steps of this process. Critical appraisal is step three:

• Ask a question
• Access the information
• Appraise the articles found
• Apply the information

Critical appraisal is the examination of evidence to determine applicability to clinical practice. It considers (1) :

• Are the results of the study believable?
• Was the study methodologically sound?  
• What is the clinical importance of the study’s results?
• Are the findings sufficiently important? That is, are they practice-changing?  
• Are the results of the study applicable to your patient?
• Is your patient comparable to the population in the study?

Why Critically Appraise?

If practitioners hope to ‘stand on the shoulders of giants’, practicing in a manner that is responsive to the discoveries of the research community, then it makes sense for the responsible, critically thinking practitioner to consider the reliability, influence, and relevance of the evidence presented to them.

While critical thinking is valuable, it is also important to avoid treading too much into cynicism; in the words of Hoffman et al. (1):

… keep in mind that no research is perfect and that it is important not to be overly critical of research articles. An article just needs to be good enough to assist you to make a clinical decision.

How do I Critically Appraise?

Evidence-based practice is intended to be practical . To enable this, critical appraisal checklists have been developed to guide practitioners through the process in an efficient yet comprehensive manner.

Critical appraisal checklists guide the reader through the appraisal process by prompting the reader to ask certain questions of the paper they are appraising. There are many different critical appraisal checklists but the best apply certain questions based on what type of study the paper is describing. This allows for a more nuanced and appropriate appraisal. Wherever possible, choose the appraisal tool that best fits the study you are appraising.

Like many things in life, repetition builds confidence and the more you apply critical appraisal tools (like checklists) to the literature the more the process will become second nature for you and the more effective you will be.

How do I Identify Study Types?

Identifying the study type described in the paper is sometimes a harder job than it should be. Helpful papers spell out the study type in the title or abstract, but not all papers are helpful in this way. As such, the critical appraiser may need to do a little work to identify what type of study they are about to critique. Again, experience builds confidence but having an understanding of the typical features of common study types certainly helps.

To assist with this, the Library has produced a guide to study designs in health research .

The following selected references will help also with understanding study types but there are also other resources in the Library’s collection and freely available online:

• The “ How to read a paper ” article series from The BMJ is a well-known source for establishing an understanding of the features of different study types; this series was subsequently adapted into a book (“ How to read a paper: the basics of evidence-based medicine ”) which offers more depth and currency than that found in the articles. (2)  
• Chapter two of “ Evidence-based practice across the health professions ” briefly outlines some study types and their application; subsequent chapters go into more detail about different study types depending on what type of question they are exploring (intervention, diagnosis, prognosis, qualitative) along with systematic reviews.  
• “ Clinical evidence made easy ” contains several chapters on different study designs and also includes critical appraisal tools. (3)  
• “ Translational research and clinical practice: basic tools for medical decision making and self-learning ” unpacks the components of a paper, explaining their purpose along with key features of different study designs. (4)  
• The BMJ website contains the contents of the fourth edition of the book “ Epidemiology for the uninitiated ”. This eBook contains chapters exploring ecological studies, longitudinal studies, case-control and cross-sectional studies, and experimental studies.

Reporting Guidelines

In order to encourage consistency and quality, authors of reports on research should follow reporting guidelines when writing their papers. The EQUATOR Network is a good source of reporting guidelines for the main study types.

While these guidelines aren't critical appraisal tools as such, they can assist by prompting you to consider whether the reporting of the research is missing important elements.

Once you've identified the study type at hand, visit EQUATOR to find the associated reporting guidelines and ask yourself: does this paper meet the guideline for its study type?

Which Checklist Should I Use?

Determining which checklist to use ultimately comes down to finding an appraisal tool that:

• Fits best with the study you are appraising
• Is reliable, well-known or otherwise validated
• You understand and are comfortable using

Below are some sources of critical appraisal tools. These have been selected as they are known to be widely accepted, easily applicable, and relevant to appraisal of a typical journal article. You may find another tool that you prefer, which is acceptable as long as it is defensible:

• CASP (Critical Appraisal Skills Programme)
• JBI (Joanna Briggs Institute)
• CEBM (Centre for Evidence-Based Medicine)
• SIGN (Scottish Intercollegiate Guidelines Network)
• STROBE (Strengthing the Reporting of Observational Studies in Epidemiology)
• BMJ Best Practice

The information on this page has been compiled by the Medical Librarian. Please contact the Library's Health Team ( [email protected] ) for further assistance.

Reference list

1. Hoffmann T, Bennett S, Del Mar C. Evidence-based practice across the health professions. 2nd ed. Chatswood, N.S.W., Australia: Elsevier Churchill Livingston; 2013.

2. Greenhalgh T. How to read a paper : the basics of evidence-based medicine. 5th ed. Chichester, West Sussex: Wiley; 2014.

3. Harris M, Jackson D, Taylor G. Clinical evidence made easy. Oxfordshire, England: Scion Publishing; 2014.

4. Aronoff SC. Translational research and clinical practice: basic tools for medical decision making and self-learning. New York: Oxford University Press; 2011.

• << Previous: Guide to Using EndNote
• Next: Manage Research Data >>
• Last Updated: Aug 27, 2024 9:00 AM
• URL: https://deakin.libguides.com/medicine

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Volume 25, Issue 1
• Critical appraisal of qualitative research: necessity, partialities and the issue of bias
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-5660-8224 Veronika Williams ,
• Anne-Marie Boylan ,
• http://orcid.org/0000-0003-4597-1276 David Nunan
• Nuffield Department of Primary Care Health Sciences , University of Oxford, Radcliffe Observatory Quarter , Oxford , UK
• Correspondence to Dr Veronika Williams, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford OX2 6GG, UK; veronika.williams{at}phc.ox.ac.uk

https://doi.org/10.1136/bmjebm-2018-111132

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• qualitative research

Introduction

Qualitative evidence allows researchers to analyse human experience and provides useful exploratory insights into experiential matters and meaning, often explaining the ‘how’ and ‘why’. As we have argued previously 1 , qualitative research has an important place within evidence-based healthcare, contributing to among other things policy on patient safety, 2 prescribing, 3 4 and understanding chronic illness. 5 Equally, it offers additional insight into quantitative studies, explaining contextual factors surrounding a successful intervention or why an intervention might have ‘failed’ or ‘succeeded’ where effect sizes cannot. It is for these reasons that the MRC strongly recommends including qualitative evaluations when developing and evaluating complex interventions. 6

Critical appraisal of qualitative research

Is it necessary.

Although the importance of qualitative research to improve health services and care is now increasingly widely supported (discussed in paper 1), the role of appraising the quality of qualitative health research is still debated. 8 10 Despite a large body of literature focusing on appraisal and rigour, 9 11–15 often referred to as ‘trustworthiness’ 16 in qualitative research, there remains debate about how to —and even whether to—critically appraise qualitative research. 8–10 17–19 However, if we are to make a case for qualitative research as integral to evidence-based healthcare, then any argument to omit a crucial element of evidence-based practice is difficult to justify. That being said, simply applying the standards of rigour used to appraise studies based on the positivist paradigm (Positivism depends on quantifiable observations to test hypotheses and assumes that the researcher is independent of the study. Research situated within a positivist paradigm isbased purely on facts and consider the world to be external and objective and is concerned with validity, reliability and generalisability as measures of rigour.) would be misplaced given the different epistemological underpinnings of the two types of data.

Given its scope and its place within health research, the robust and systematic appraisal of qualitative research to assess its trustworthiness is as paramount to its implementation in clinical practice as any other type of research. It is important to appraise different qualitative studies in relation to the specific methodology used because the methodological approach is linked to the ‘outcome’ of the research (eg, theory development, phenomenological understandings and credibility of findings). Moreover, appraisal needs to go beyond merely describing the specific details of the methods used (eg, how data were collected and analysed), with additional focus needed on the overarching research design and its appropriateness in accordance with the study remit and objectives.

Poorly conducted qualitative research has been described as ‘worthless, becomes fiction and loses its utility’. 20 However, without a deep understanding of concepts of quality in qualitative research or at least an appropriate means to assess its quality, good qualitative research also risks being dismissed, particularly in the context of evidence-based healthcare where end users may not be well versed in this paradigm.

How is appraisal currently performed?

Appraising the quality of qualitative research is not a new concept—there are a number of published appraisal tools, frameworks and checklists in existence. 21–23  An important and often overlooked point is the confusion between tools designed for appraising methodological quality and reporting guidelines designed to assess the quality of methods reporting. An example is the Consolidate Criteria for Reporting Qualitative Research (COREQ) 24 checklist, which was designed to provide standards for authors when reporting qualitative research but is often mistaken for a methods appraisal tool. 10

Broadly speaking there are two types of critical appraisal approaches for qualitative research: checklists and frameworks. Checklists have often been criticised for confusing quality in qualitative research with ‘technical fixes’ 21 25 , resulting in the erroneous prioritisation of particular aspects of methodological processes over others (eg, multiple coding and triangulation). It could be argued that a checklist approach adopts the positivist paradigm, where the focus is on objectively assessing ‘quality’ where the assumptions is that the researcher is independent of the research conducted. This may result in the application of quantitative understandings of bias in order to judge aspects of recruitment, sampling, data collection and analysis in qualitative research papers. One of the most widely used appraisal tools is the Critical Appraisal Skills Programme (CASP) 26 and along with the JBI QARI (Joanna Briggs Institute Qualitative Assessment and Assessment Instrument) 27 presents examples which tend to mimic the quantitative approach to appraisal. The CASP qualitative tool follows that of other CASP appraisal tools for quantitative research designs developed in the 1990s. The similarities are therefore unsurprising given the status of qualitative research at that time.

Frameworks focus on the overarching concepts of quality in qualitative research, including transparency, reflexivity, dependability and transferability (see box 1 ). 11–13 15 16 20 28 However, unless the reader is familiar with these concepts—their meaning and impact, and how to interpret them—they will have difficulty applying them when critically appraising a paper.

The main issue concerning currently available checklist and framework appraisal methods is that they take a broad brush approach to ‘qualitative’ research as whole, with few, if any, sufficiently differentiating between the different methodological approaches (eg, Grounded Theory, Interpretative Phenomenology, Discourse Analysis) nor different methods of data collection (interviewing, focus groups and observations). In this sense, it is akin to taking the entire field of ‘quantitative’ study designs and applying a single method or tool for their quality appraisal. In the case of qualitative research, checklists, therefore, offer only a blunt and arguably ineffective tool and potentially promote an incomplete understanding of good ‘quality’ in qualitative research. Likewise, current framework methods do not take into account how concepts differ in their application across the variety of qualitative approaches and, like checklists, they also do not differentiate between different qualitative methodologies.

On the need for specific appraisal tools

Current approaches to the appraisal of the methodological rigour of the differing types of qualitative research converge towards checklists or frameworks. More importantly, the current tools do not explicitly acknowledge the prejudices that may be present in the different types of qualitative research.

Concepts of rigour or trustworthiness within qualitative research 31

Transferability: the extent to which the presented study allows readers to make connections between the study’s data and wider community settings, ie, transfer conceptual findings to other contexts.

Credibility: extent to which a research account is believable and appropriate, particularly in relation to the stories told by participants and the interpretations made by the researcher.

Reflexivity: refers to the researchers’ engagement of continuous examination and explanation of how they have influenced a research project from choosing a research question to sampling, data collection, analysis and interpretation of data.

Transparency: making explicit the whole research process from sampling strategies, data collection to analysis. The rationale for decisions made is as important as the decisions themselves.

However, we often talk about these concepts in general terms, and it might be helpful to give some explicit examples of how the ‘technical processes’ affect these, for example, partialities related to:

Selection: recruiting participants via gatekeepers, such as healthcare professionals or clinicians, who may select them based on whether they believe them to be ‘good’ participants for interviews/focus groups.

Data collection: poor interview guide with closed questions which encourage yes/no answers and/leading questions.

Reflexivity and transparency: where researchers may focus their analysis on preconceived ideas rather than ground their analysis in the data and do not reflect on the impact of this in a transparent way.

The lack of tailored, method-specific appraisal tools has potentially contributed to the poor uptake and use of qualitative research for informing evidence-based decision making. To improve this situation, we propose the need for more robust quality appraisal tools that explicitly encompass both the core design aspects of all qualitative research (sampling/data collection/analysis) but also considered the specific partialities that can be presented with different methodological approaches. Such tools might draw on the strengths of current frameworks and checklists while providing users with sufficient understanding of concepts of rigour in relation to the different types of qualitative methods. We provide an outline of such tools in the third and final paper in this series.

As qualitative research becomes ever more embedded in health science research, and in order for that research to have better impact on healthcare decisions, we need to rethink critical appraisal and develop tools that allow differentiated evaluations of the myriad of qualitative methodological approaches rather than continuing to treat qualitative research as a single unified approach.

• Williams V ,
• Boylan AM ,
• Lingard L ,
• Orser B , et al
• Brawn R , et al
• Van Royen P ,
• Vermeire E , et al
• Barker M , et al
• McGannon KR
• Dixon-Woods M ,
• Agarwal S , et al
• Greenhalgh T ,
• Dennison L ,
• Morrison L ,
• Conway G , et al
• Barrett M ,
• Mayan M , et al
• Lockwood C ,
• Santiago-Delefosse M ,
• Bruchez C , et al
• Sainsbury P ,
• ↵ CASP (Critical Appraisal Skills Programme). date unknown . http://www.phru.nhs.uk/Pages/PHD/CASP.htm .
• ↵ The Joanna Briggs Institute . JBI QARI Critical appraisal checklist for interpretive & critical research . Adelaide : The Joanna Briggs Institute , 2014 .
• Stephens J ,

Contributors VW and DN: conceived the idea for this article. VW: wrote the first draft. AMB and DN: contributed to the final draft. All authors approve the submitted article.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Correction notice This article has been updated since its original publication to include a new reference (reference 1.)

Read the full text or download the PDF:

• Mayo Clinic Libraries
• Evidence Synthesis Guide
• Risk of Bias by Study Design

Evidence Synthesis Guide : Risk of Bias by Study Design

• Review Types & Decision Tree
• Standards & Reporting Results
• Materials in the Mayo Clinic Libraries
• Training Resources
• Review Teams
• Develop & Refine Your Research Question
• Develop a Timeline
• Project Management
• Communication
• PRISMA-P Checklist
• Eligibility Criteria
• Register your Protocol
• Other Resources
• Other Screening Tools
• Grey Literature Searching
• Citation Searching
• Minimize Bias
• GRADE & GRADE-CERQual
• Data Extraction Tools
• Synthesis & Meta-Analysis
• Publishing your Review

Risk of Bias of Individual Studies

““Assessment of risk of bias is a key step that informs many other steps and decisions made in conducting systematic reviews. It plays an important role in the final assessment of the strength of the evidence.” 1  

Risk of Bias by Study Design (featured tools)

• Systematic Reviews
• Non-RCTs or Observational Studies
• Diagnostic Accuracy
• Animal Studies
• Qualitative Research
• Tool Repository
• AMSTAR 2 The original AMSTAR was developed to assess the risk of bias in systematic reviews that included only randomized controlled trials. AMSTAR 2 was published in 2017 and allows researchers to identify high quality systematic reviews, including those based on non-randomised studies of healthcare interventions. more... less... AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews)
• ROBIS ROBIS is a tool designed specifically to assess the risk of bias in systematic reviews. The tool is completed in three phases: (1) assess relevance(optional), (2) identify concerns with the review process, and (3) judge risk of bias in the review. Signaling questions are included to help assess specific concerns about potential biases with the review. more... less... ROBIS (Risk of Bias in Systematic Reviews)
• BMJ Framework for Assessing Systematic Reviews This framework provides a checklist that is used to evaluate the quality of a systematic review.
• CASP Checklist for Systematic Reviews This CASP checklist is not a scoring system, but rather a method of appraising systematic reviews by considering: 1. Are the results of the study valid? 2. What are the results? 3. Will the results help locally? more... less... CASP (Critical Appraisal Skills Programme)
• CEBM Systematic Reviews Critical Appraisal Sheet The CEBM’s critical appraisal sheets are designed to help you appraise the reliability, importance, and applicability of clinical evidence. more... less... CEBM (Centre for Evidence-Based Medicine)
• JBI Critical Appraisal Tools, Checklist for Systematic Reviews JBI Critical Appraisal Tools help you assess the methodological quality of a study and to determine the extent to which study has addressed the possibility of bias in its design, conduct and analysis.
• NHLBI Study Quality Assessment of Systematic Reviews and Meta-Analyses The NHLBI’s quality assessment tools were designed to assist reviewers in focusing on concepts that are key for critical appraisal of the internal validity of a study. more... less... NHLBI (National Heart, Lung, and Blood Institute)
• RoB 2 RoB 2 provides a framework for assessing the risk of bias in a single estimate of an intervention effect reported from a randomized trial, rather than the entire trial. more... less... RoB 2 (revised tool to assess Risk of Bias in randomized trials)
• CASP Randomised Controlled Trials Checklist This CASP checklist considers various aspects of an RCT that require critical appraisal: 1. Is the basic study design valid for a randomized controlled trial? 2. Was the study methodologically sound? 3. What are the results? 4. Will the results help locally? more... less... CASP (Critical Appraisal Skills Programme)
• CONSORT Statement The CONSORT checklist includes 25 items to determine the quality of randomized controlled trials. Critical appraisal of the quality of clinical trials is possible only if the design, conduct, and analysis of RCTs are thoroughly and accurately described in the report. more... less... CONSORT (Consolidated Standards of Reporting Trials)
• NHLBI Study Quality Assessment of Controlled Intervention Studies The NHLBI’s quality assessment tools were designed to assist reviewers in focusing on concepts that are key for critical appraisal of the internal validity of a study. more... less... NHLBI (National Heart, Lung, and Blood Institute)
• JBI Critical Appraisal Tools Checklist for Randomized Controlled Trials JBI Critical Appraisal Tools help you assess the methodological quality of a study and to determine the extent to which study has addressed the possibility of bias in its design, conduct and analysis.
• ROBINS-I ROBINS-I is a tool for evaluating risk of bias in estimates of the comparative effectiveness… of interventions from studies that did not use randomization to allocate units to comparison groups. more... less... ROBINS-I (Risk Of Bias in Non-randomized Studies – of Interventions)
• NOS This tool is used primarily to evaluate and appraise case-control or cohort studies. more... less... NOS (Newcastle-Ottawa Scale)
• AXIS Cross-sectional studies are frequently used as an evidence base for diagnostic testing, risk factors for disease, and prevalence studies. The AXIS tool focuses mainly on the presented study methods and results. more... less... AXIS (Appraisal tool for Cross-Sectional Studies)
• NHLBI Study Quality Assessment Tools for Non-Randomized Studies The NHLBI’s quality assessment tools were designed to assist reviewers in focusing on concepts that are key for critical appraisal of the internal validity of a study. • Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies • Quality Assessment of Case-Control Studies • Quality Assessment Tool for Before-After (Pre-Post) Studies With No Control Group • Quality Assessment Tool for Case Series Studies more... less... NHLBI (National Heart, Lung, and Blood Institute)
• Case Series Studies Quality Appraisal Checklist Developed by the Institute of Health Economics (Canada), the checklist is comprised of 20 questions to assess the robustness of the evidence of uncontrolled case series studies.
• Methodological Quality and Synthesis of Case Series and Case Reports In this paper, Dr. Murad and colleagues present a framework for appraisal, synthesis and application of evidence derived from case reports and case series.
• MINORS The MINORS instrument contains 12 items and was developed for evaluating the quality of observational or non-randomized studies. This tool may be of particular interest to researchers who would like to critically appraise surgical studies. more... less... MINORS (Methodological Index for Non-Randomized Studies)
• JBI Critical Appraisal Tools for Non-Randomized Trials JBI Critical Appraisal Tools help you assess the methodological quality of a study and to determine the extent to which study has addressed the possibility of bias in its design, conduct and analysis. • Checklist for Analytical Cross Sectional Studies • Checklist for Case Control Studies • Checklist for Case Reports • Checklist for Case Series • Checklist for Cohort Studies
• QUADAS-2 The QUADAS-2 tool is designed to assess the quality of primary diagnostic accuracy studies it consists of 4 key domains that discuss patient selection, index test, reference standard, and flow of patients through the study and timing of the index tests and reference standard. more... less... QUADAS-2 (a revised tool for the Quality Assessment of Diagnostic Accuracy Studies)
• JBI Critical Appraisal Tools Checklist for Diagnostic Test Accuracy Studies JBI Critical Appraisal Tools help you assess the methodological quality of a study and to determine the extent to which study has addressed the possibility of bias in its design, conduct and analysis.
• STARD 2015 The authors of the standards note that essential elements of diagnostic accuracy study methods are often poorly described and sometimes completely omitted, making both critical appraisal and replication difficult, if not impossible. The Standards for the Reporting of Diagnostic Accuracy Studies was developed to help improve completeness and transparency in reporting of diagnostic accuracy studies. more... less... STARD 2015 (Standards for the Reporting of Diagnostic Accuracy Studies)
• CASP Diagnostic Study Checklist This CASP checklist considers various aspects of diagnostic test studies including: 1. Are the results of the study valid? 2. What were the results? 3. Will the results help locally? more... less... CASP (Critical Appraisal Skills Programme)
• CEBM Diagnostic Critical Appraisal Sheet The CEBM’s critical appraisal sheets are designed to help you appraise the reliability, importance, and applicability of clinical evidence. more... less... CEBM (Centre for Evidence-Based Medicine)
• SYRCLE’s RoB Implementation of SYRCLE’s RoB tool will facilitate and improve critical appraisal of evidence from animal studies. This may enhance the efficiency of translating animal research into clinical practice and increase awareness of the necessity of improving the methodological quality of animal studies. more... less... SYRCLE’s RoB (SYstematic Review Center for Laboratory animal Experimentation’s Risk of Bias)
• ARRIVE 2.0 The ARRIVE 2.0 guidelines are a checklist of information to include in a manuscript to ensure that publications on in vivo animal studies contain enough information to add to the knowledge base. more... less... ARRIVE 2.0 (Animal Research: Reporting of In Vivo Experiments)
• Critical Appraisal of Studies Using Laboratory Animal Models This article provides an approach to critically appraising papers based on the results of laboratory animal experiments, and discusses various bias domains in the literature that critical appraisal can identify.
• CEBM Critical Appraisal of Qualitative Studies Sheet The CEBM’s critical appraisal sheets are designed to help you appraise the reliability, importance and applicability of clinical evidence. more... less... CEBM (Centre for Evidence-Based Medicine)
• CASP Qualitative Studies Checklist This CASP checklist considers various aspects of qualitative research studies including: 1. Are the results of the study valid? 2. What were the results? 3. Will the results help locally? more... less... CASP (Critical Appraisal Skills Programme)
• Quality Assessment and Risk of Bias Tool Repository Created by librarians at Duke University, this extensive listing contains over 100 commonly used risk of bias tools that may be sorted by study type.
• Latitudes Network A library of risk of bias tools for use in evidence syntheses that provides selection help and training videos.

References & Recommended Reading

1.    Viswanathan, M., Patnode, C. D., Berkman, N. D., Bass, E. B., Chang, S., Hartling, L., ... & Kane, R. L. (2018). Recommendations for assessing the risk of bias in systematic reviews of health-care interventions .  Journal of clinical epidemiology ,  97 , 26-34.

2.     Kolaski, K., Logan, L. R., & Ioannidis, J. P. (2024). Guidance to best tools and practices for systematic reviews .  British Journal of Pharmacology ,  181 (1), 180-210

3.     Fowkes FG, Fulton PM.  Critical appraisal of published research: introductory guidelines.   BMJ (Clinical research ed).  1991;302(6785):1136-1140.

4.     Shea BJ, Reeves BC, Wells G, et al.  AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.   BMJ (Clinical research ed).  2017;358:j4008.

5..     Whiting P, Savovic J, Higgins JPT, et al.  ROBIS: A new tool to assess risk of bias in systematic reviews was developed.   Journal of clinical epidemiology.  2016;69:225-234.

6.     Sterne JAC, Savovic J, Page MJ, et al.  RoB 2: a revised tool for assessing risk of bias in randomised trials.  BMJ (Clinical research ed).  2019;366:l4898.

7.     Moher D, Hopewell S, Schulz KF, et al.  CONSORT 2010 Explanation and Elaboration: Updated guidelines for reporting parallel group randomised trials.  Journal of clinical epidemiology.  2010;63(8):e1-37.

8..    Sterne JA, Hernan MA, Reeves BC, et al.  ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions.  BMJ (Clinical research ed).  2016;355:i4919.

9.    Downes MJ, Brennan ML, Williams HC, Dean RS.  Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS).   BMJ open.  2016;6(12):e011458.

10.   Guo B, Moga C, Harstall C, Schopflocher D.  A principal component analysis is conducted for a case series quality appraisal checklist.   Journal of clinical epidemiology.  2016;69:199-207.e192.

11.   Murad MH, Sultan S, Haffar S, Bazerbachi F.  Methodological quality and synthesis of case series and case reports.  BMJ evidence-based medicine.  2018;23(2):60-63.

12.   Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J.  Methodological index for non-randomized studies (MINORS): development and validation of a new instrument.   ANZ journal of surgery.  2003;73(9):712-716.

13.   Whiting PF, Rutjes AWS, Westwood ME, et al.  QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies.   Annals of internal medicine.  2011;155(8):529-536.

14.   Bossuyt PM, Reitsma JB, Bruns DE, et al.  STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies.   BMJ (Clinical research ed).  2015;351:h5527.

15.   Hooijmans CR, Rovers MM, de Vries RBM, Leenaars M, Ritskes-Hoitinga M, Langendam MW.  SYRCLE's risk of bias tool for animal studies.   BMC medical research methodology.  2014;14:43.

16.   Percie du Sert N, Ahluwalia A, Alam S, et al.  Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0.  PLoS biology.  2020;18(7):e3000411.

17.   O'Connor AM, Sargeant JM.  Critical appraisal of studies using laboratory animal models.   ILAR journal.  2014;55(3):405-417.

• << Previous: Minimize Bias
• Next: GRADE & GRADE-CERQual >>
• Last Updated: Aug 30, 2024 2:14 PM
• URL: https://libraryguides.mayo.edu/systematicreviewprocess

Evaluating Sources

• Introduction
• Test Yourself

Introduction to critical appraisal

Critical appraisal checklists, video: using the casp checklist for appraisal of qualitative research, critical thinking.

• Other useful resources

The second stage to the evaluative process is critical appraisal. This involves a careful examination of the author's arguments and the evidence they provide to support their claims. Even if you are reading an article published in a top academic journal or a book by a leading authority in your field of study, you should start from a position of neutrality and take the approach that the author (whoever they are) must make a case to persuade you of the validity of their arguments.

Ask questions...

• Do I find the author’s arguments persuasive?​
• Are the aims of the study clear?​
• How strong is the evidence supporting the claims?​
• What are the strengths and weaknesses of the study? ​
• What are the limitations of study design? Have these been acknowledged? ​
• What has research by other authors shown?​
• Do any factors cause me to doubt the validity or reliability of the research? ​

In addition to the facts and analysis offered by the authors, think about the context of their research, the appropriateness and limitations of study design, and any external factors that may impact the relevance or the importance of their approach.   

Look at the bigger picture

When consulting the literature, seek out the different points of view, set them against each other, test them by questioning them, and see how well they stand up when you raise objections and expose them to opposing views by other authors.

There are a number of critical appraisal checklists which can help you evaluate particular kinds of studies. Though primarily intended for healthcare clinicians, these checklists are excellent tools for researchers in any discipline  Here  are a few examples:

CASP  

CONSORT  

TREND 

Keep in mind that most studies have imitations and will rarely tick every box. When a study fails to satisfy one or more criteria on a checklist, consider the  extent to which this impacts the evidence (if at all).These considerations can inform the critical discussion in your essay. 

Critical thinking is the foundation for good academic writing. It should inform every stage of the journey from planning your essay to embarking on your research project to writing the final draft of your paper. Evaluating and critically appraising sources is a key stage in this process,

• Critical engagement with the idea or topic 
• Critical appraisal of each piece of evidence we encounter
• Comparing competing sources of evidence​
• Developing an argument on the idea or topic informed by our research
• << Previous: Test Yourself
• Next: Other useful resources >>
• Last Updated: Sep 3, 2024 1:46 PM
• URL: https://uws-uk.libguides.com/evaluating_sources

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A critical appraisal of reporting in randomized controlled trials investigating osteopathic manipulative treatment: a meta-research study.

1. Introduction

2. materials and methods, 2.1. eligibility, 2.2. study selection, 2.3. data collection process, 2.4. evaluating the completeness of reporting, 2.5. evaluating the risk of bias, 2.6. statistical analysis, 3.1. adherence to consort checklist (completeness of reporting), 3.2. relationship between completeness of reporting and risk of bias, 3.3. relationship between completeness of reporting and protocol registration, journals characteristics and date of publication, 4. discussion, 4.1. summary of results, 4.2. implications for future research, 4.3. limitations, 5. conclusions, supplementary materials, author contributions, data availability statement, conflicts of interest.

• Available online: https://oialliance.org/wp-content/uploads/2024/02/OIA_Report_2020_FINAL_Summary.pdf (accessed on 1 June 2024).
• Available online: https://www.who.int/publications/i/item/9789241599665 (accessed on 1 June 2024).
• Morin, C.; Gaboury, I. Osteopathic empirical research: A bibliometric analysis from 1966 to 2018. BMC Complement Med. Ther. 2021 , 21 , 196. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Dal Farra, F.; Aquino, A.; Tarantino, A.G.; Origo, D. Effectiveness of Myofascial Manual Therapies in Chronic Pelvic Pain Syndrome: A Systematic Review and Meta-Analysis. Int. Urogynecol. J. 2022 , 33 , 2963–2976. [ Google Scholar ] [ CrossRef ]
• Buffone, F.; Monacis, D.; Tarantino, A.G.; Dal Farra, F.; Bergna, A.; Agosti, M.; Vismara, L. Osteopathic Treatment for Gastrointestinal Disorders in Term and Preterm Infants: A Systematic Review and Meta-Analysis. Healthcare 2022 , 10 , 1525. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Dal Farra, F.; Buffone, F.; Risio, R.G.; Tarantino, A.G.; Vismara, L.; Bergna, A. Effectiveness of osteopathic interventions in patients with non-specific neck pain: A systematic review and meta-analysis. Complement Ther. Clin. Pract. 2022 , 49 , 101655. [ Google Scholar ] [ CrossRef ]
• Cerritelli, F.; Ruffini, N.; Lacorte, E.; Vanacore, N. Osteopathic manipulative treatment in neurological diseases: Systematic review of the literature. J. Neurol. Sci. 2016 , 369 , 333–341. [ Google Scholar ] [ CrossRef ]
• Bucher, H.C.; Guyatt, G.H.; Cook, D.J.; Holbrook, A.; McAlister, F.A. Users’ guides to the medical literature: XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. Evidence-Based Medicine Working Group. JAMA 1999 , 282 , 771–778. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Moher, D.; Schulz, K.F.; Altman, D.G. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001 , 357 , 1191–1194. [ Google Scholar ] [ CrossRef ]
• Cochrane. Cochrane Handbook for Systematic Reviews of Interventions, Version 6.4 ; Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A., Eds.; (Updated August 2023); The Cochrane Collaboration: London, UK; John Wiley & Sons Ltd.: Hoboken, NJ, USA, 2023. [ Google Scholar ]
• Innocenti, T.; Salvioli, S.; Giagio, S.; Feller, D.; Cartabellotta, N.; Chiarotto, A. Declaration of use and appropriate use of reporting guidelines in high-impact rehabilitation journals is limited: A meta-research study. J. Clin. Epidemiol. 2021 , 131 , 43–50. [ Google Scholar ] [ CrossRef ]
• Ioannidis, J.P.A. Meta-research: Why research on research matters. PLoS Biol. 2018 , 16 , e2005468. [ Google Scholar ] [ CrossRef ]
• Ioannidis, J.P.; Fanelli, D.; Dunne, D.D.; Goodman, S.N. Meta-research: Evaluation and Improvement of Research Methods and Practices. PLoS Biol. 2015 , 13 , e1002264. [ Google Scholar ] [ CrossRef ]
• Higgins, J.P.; Altman, D.G.; Gøtzsche, P.C.; Jüni, P.; Moher, D.; Oxman, A.D.; Savović, J.; Schulz, K.F.; Weeks, L.; Sterne, J.A.C.; et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011 , 343 , d5928. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Butcher, N.J.; Monsour, A.; Mew, E.J.; Chan, A.-W.; Moher, D.; Mayo-Wilson, E.; Terwee, C.B.; Chee-A-Tow, A.; Baba, A.; Gavin, F.; et al. Guidelines for Reporting Outcomes in Trial Reports: The CONSORT-Outcomes 2022 Extension. JAMA 2022 , 328 , 2252–2264. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Innocenti, T.; Giagio, S.; Salvioli, S.; Feller, D.; Minnucci, S.; Brindisino, F.; IJzelenberg, W.; Ostelo, R.; Chiarotto, A. Completeness of Reporting Is Suboptimal in Randomized Controlled Trials Published in Rehabilitation Journals, with Trials with Low Risk of Bias Displaying Better Reporting: A Meta-research Study. Arch. Phys. Med. Rehabil. 2022 , 103 , 1839–1847. [ Google Scholar ] [ CrossRef ]
• Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021 , 372 , n71. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Ouzzani, M.; Hammady, H.; Fedorowicz, Z.; Elmagarmid, A. Rayyan-a web and mobile app for systematic reviews. Syst. Rev. 2016 , 5 , 210. [ Google Scholar ] [ CrossRef ]
• Flemyng, E.; Moore, T.H.; Boutron, I.; Higgins, J.P.T.; Hróbjartsson, A.; Nejstgaard, C.H.; Dwan, C. Using Risk of Bias 2 to assess results from randomised controlled trials: Guidance from Cochrane. BMJ Evid. Based Med. 2023 , 28 , 260–266. [ Google Scholar ] [ CrossRef ]
• Lehane, E.; Leahy-Warren, P.; O'Riordan, C.; Savage, E.; Drennan, J.; O’Tuathaigh, C.; O’Connor, M.; Corrigan, M.; Burke, F.; Hayes, M. Evidence-based practice education for healthcare professions: An expert view. BMJ Evid. Based Med. 2019 , 24 , 103–108. [ Google Scholar ] [ CrossRef ]
• Thomas, E.T.; Heneghan, C. Catalogue of bias: Selective outcome reporting bias. BMJ Evid. Based Med. 2022 , 27 , 370–372. [ Google Scholar ] [ CrossRef ]
• Riley, S.P.; Swanson, B.T.; Cook, C.E. “Trustworthiness”, confidence in estimated effects, and confidently translating research into clinical practice. Arch. Physiother. 2023 , 13 , 8. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Murad, M.H.; Katabi, A.; Benkhadra, R.; Montori, V.M. External validity, generalisability, applicability and directness: A brief primer. BMJ Evid. Based Med. 2018 , 23 , 17–19. [ Google Scholar ] [ CrossRef ]
• De Angelis, C.; Drazen, J.M.; Frizelle, F.A.; Haug, C.; Hoey, J.; Horton, R.; Kotzin, S.; Laine, C.; Marusic, A.; Overbeke, J.P.M.; et al. Clinical trial registration: A statement from the International Committee of Medical Journal Editors. N. Engl. J. Med. 2004 , 351 , 1250–1251. [ Google Scholar ] [ CrossRef ]
• Lindsley, K.; Fusco, N.; Teeuw, H.; Mooij, E.; Scholten, R.; Hooft, L. Poor compliance of clinical trial registration among trials included in systematic reviews: A cohort study. J. Clin. Epidemiol. 2021 , 132 , 79–87. [ Google Scholar ] [ CrossRef ]
• Viergever, R.F.; Li, K. Trends in global clinical trial registration: An analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open. 2015 , 5 , e008932. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Leech, J.B.; Owen, W.E.; Young, J.L.; Rhon, D.I. Incomplete reporting of manual therapy interventions and a lack of clinician and setting diversity in clinical trials for neck pain limits replication and real-world translation. A scoping review. J. Man. Manip. Ther. 2023 , 31 , 153–161, Erratum in J. Man. Manip. Ther. 2023 , 31 , 214. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Alvarez, G.; Solà, I.; Sitjà-Rabert, M.; Fort-Vanmeerhaeghe, A.; Gich, I.; Fernándezd, C.; Urrútia, G. A methodological review revealed that reporting of trials in manual therapy has not improved over time. J. Clin. Epidemiol. 2020 , 121 , 32–44. [ Google Scholar ] [ CrossRef ]
• Gonzalez, G.Z.; Moseley, A.M.; Maher, C.G.; Nascimento, D.P.; Costa, L.D.C.M.; Costa, L.O. Methodologic Quality and Statistical Reporting of Physical Therapy Randomized Controlled Trials Relevant to Musculoskeletal Conditions. Arch. Phys. Med. Rehabil. 2018 , 99 , 129–136. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Gianola, S.; Castellini, G.; Agostini, M.; Bolotta, R.; Corbetta, D.; Frigerio, P.; Gasparini, M.; Gozzer, P.; Guariento, E.; Li, L.C.; et al. Reporting of Rehabilitation Intervention for Low Back Pain in Randomized Controlled Trials: Is the Treatment Fully Replicable? Spine 2016 , 41 , 412–418. [ Google Scholar ] [ CrossRef ]
• Candy, B.; Vickerstaff, V.; Jones, L.; King, M. Description of complex interventions: Analysis of changes in reporting in randomised trials since 2002. Trials 2018 , 19 , 110. [ Google Scholar ] [ CrossRef ]
• Dal Farra, F.; Risio, R.G.; Vismara, L.; Bergna, A. Effectiveness of osteopathic interventions in chronic non-specific low back pain: A systematic review and meta-analysis. Complement Ther Med. 2021 , 56 , 102616. [ Google Scholar ] [ CrossRef ]
• Hoffmann, T.C.; Glasziou, P.P.; Boutron, I.; Milne, R.; Perera, R.; Moher, D.; Altman, D.G.; Barbour, V.; Macdonald, H.; Johnston, M.; et al. Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014 , 348 , g1687. [ Google Scholar ] [ CrossRef ]
• Yamato, T.; Maher, C.; Saragiotto, B.; Moseley, A.; Hoffmann, T.; Elkins, M.; Brooks, D. The TIDieR Checklist Will Benefit the Physiotherapy Profession. Physiother. Can. 2016 , 68 , 311–314. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Alvarez, G.; Cerritelli, F.; Urrutia, G. Using the template for intervention description and replication (TIDieR) as a tool for improving the design and reporting of manual therapy interventions. Man. Ther. 2016 , 24 , 85–89. [ Google Scholar ] [ CrossRef ]
• Nascimento, D.P.; Gonzalez, G.Z.; Araujo, A.C.; Moseley, A.; Maher, C.; Costa, L.O.P. Factors associated with the reporting quality of low back pain systematic review abstracts in physical therapy: A methodological study. Braz. J. Phys. Ther. 2021 , 25 , 233–241. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mrowinski, M.J.; Fronczak, P.; Fronczak, A.; Ausloos, M.; Nedic, O. Artificial intelligence in peer review: How can evolutionary computation support journal editors? PLoS ONE 2017 , 12 , e0184711. [ Google Scholar ] [ CrossRef ]
• Chubb, J.; Cowling, P.; Reed, D. Speeding up to keep up: Exploring the use of AI in the research process. AI Soc. 2022 , 37 , 1439–1457. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gøtzsche, P.C.; Ioannidis, J.P.A.; Clarke, M.; Devereaux, P.J.; Kleijnen, J.; Moher, D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: Explanation and elaboration. BMJ 2009 , 339 , b2700. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Click here to enlarge figure

Share and Cite

Zambonin Mazzoleni, G.; Bergna, A.; Buffone, F.; Sacchi, A.; Misseroni, S.; Tramontano, M.; Dal Farra, F. A Critical Appraisal of Reporting in Randomized Controlled Trials Investigating Osteopathic Manipulative Treatment: A Meta-Research Study. J. Clin. Med. 2024 , 13 , 5181. https://doi.org/10.3390/jcm13175181

Zambonin Mazzoleni G, Bergna A, Buffone F, Sacchi A, Misseroni S, Tramontano M, Dal Farra F. A Critical Appraisal of Reporting in Randomized Controlled Trials Investigating Osteopathic Manipulative Treatment: A Meta-Research Study. Journal of Clinical Medicine . 2024; 13(17):5181. https://doi.org/10.3390/jcm13175181

Zambonin Mazzoleni, Gabriele, Andrea Bergna, Francesca Buffone, Andrea Sacchi, Serena Misseroni, Marco Tramontano, and Fulvio Dal Farra. 2024. "A Critical Appraisal of Reporting in Randomized Controlled Trials Investigating Osteopathic Manipulative Treatment: A Meta-Research Study" Journal of Clinical Medicine 13, no. 17: 5181. https://doi.org/10.3390/jcm13175181

Article Metrics

Article access statistics, supplementary material.

ZIP-Document (ZIP, 448 KiB)

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Bolshoy Kamen' Tourism

• Things To Do in Bolshoy Kamen'
• Bolshoy Kamen' Itineraries
• Bolshoy Kamen' Hotels

Bolshoy Kamen' Trip Planner

Plan your customized day by day trip plan for Bolshoy Kamen'. Choose from various experinces categories as adventure, romantic and family and kids friendly for your trip using using Bolshoy Kamen' trip planner.

• THINGS TO DO
• HOW TO REACH
• TOP RESTAURANTS

• Systematic Review
• Open access
• Published: 02 September 2024

Common data quality elements for health information systems: a systematic review

• Hossein Ghalavand 1 ,
• Saied Shirshahi 2 ,
• Alireza Rahimi 2 ,
• Zarrin Zarrinabadi 1 &
• Fatemeh Amani 3  

BMC Medical Informatics and Decision Making volume  24 , Article number:  243 ( 2024 ) Cite this article

Metrics details

Data quality in health information systems has a complex structure and consists of several dimensions. This research conducted for identify Common data quality elements for health information systems.

A literature review was conducted and search strategies run in Web of Knowledge, Science Direct, Emerald, PubMed, Scopus and Google Scholar search engine as an additional source for tracing references. We found 760 papers, excluded 314 duplicates, 339 on abstract review and 167 on full-text review; leaving 58 papers for critical appraisal.

Current review shown that 14 criteria are categorized as the main dimensions for data quality for health information system include: Accuracy, Consistency, Security, Timeliness, Completeness, Reliability, Accessibility, Objectivity, Relevancy, Understandability, Navigation, Reputation, Efficiency and Value- added. Accuracy, Completeness, and Timeliness, were the three most-used dimensions in literature.

Conclusions

At present, there is a lack of uniformity and potential applicability in the dimensions employed to evaluate the data quality of health information system. Typically, different approaches (qualitative, quantitative and mixed methods) were utilized to evaluate data quality for health information system in the publications that were reviewed. Consequently, due to the inconsistency in defining dimensions and assessing methods, it became imperative to categorize the dimensions of data quality into a limited set of primary dimensions.

Peer Review reports

Appropriate planning in the health sector relies on the existence of accurate data and the quality of the data must be continuously controlled. The World Health Organization has tried to ensure the quality of health data by providing a toolkit. This toolkit supports countries to assess and improve the quality of health data [ 1 , 2 ].

The existence of accurate, complete, and timely data plays an important role in health care management [ 3 , 4 , 5 ]. Data quality is often only considered a component of the effectiveness of health information systems, and hiding the value of data quality in other parts of the health field can lead to incorrect decision-making [ 6 , 7 , 8 , 9 ]. Previous studies have confirmed that data quality is a multidimensional concept. Data quality assessment requires familiarity with different subjective and objective criteria and both subjective perceptions of people and objective measurements of information must be addressed [ 10 , 11 ]. Qualitative evaluations of subjective data reflect the needs and experiences of stakeholders, and objective evaluations reflect the needs of managers and stakeholders [ 12 ].

Adverse effects on the quality of care, increasing costs, creating liability risks, and reducing the benefits of investing in health information systems can be identified as the negative effects of poor-quality data [ 13 , 14 , 15 , 16 ]. Defects in data quality can lead to incorrect diagnosis and intervention in health care [ 4 , 13 , 17 , 18 ]. The quality of healthcare depends on the existence of quality data, which ultimately leads to a significant impact on customer satisfaction [ 13 , 19 ].

Data quality in health information systems has a complex structure and consists of several dimensions and some critical factors performance such as environmental and organizational, technical and behavioral affected on data quality in health information system [ 20 , 21 , 22 ]. As we mentioned later, previous studies have sporadically reported some data quality elements in health information systems. There is no comprehensive agreement on its dimensions and there is no unique accepted definition of data quality among researchers for health information systems. However, there is still a lack of a review compiling and synthesizing all elements introduced in the literature. In this study, a more comprehensive understanding of the elements for quality of data in health information systems has been done using a systematic review method. The findings of this study can provide opportunities for health policy maker to become familiar with various data quality elements in health information. This systematic review specifically answered the following research questions:

1- What are the common data quality elements for health information systems?

2- What are the roles of common data quality elements to improve the performance of health information systems?

In this review, we used a systematic approach to retrieve the relevant research studies. Our reporting strategy follows the PRISMA guidelines [ 23 ].

Eligibility criteria

In this study the inclusion criteria were: (1) Data quality components were showcased within a health information system; (2) published from the year 2003 to 2024; (3) empirical studies that answered the research questions or tested the hypothesis and conducted on specific health system The exclusion criteria were: (1) Research that did not outline data quality dimensions in health management systems; (2) Content presented in a format other than a scientific article such as Conference papers, book sections, and …; (4) Methodologies deemed to be deficient in terms of quality; (5) Publication language not in English; and (7) The full text was unavailable.

Information sources

The literature search was conducted between September and October 2023, using the following five electronic scientific databases: Web of Knowledge, Science Direct, Emerald, PubMed, Scopus and Google Scholar search engine as an additional source for tracing references.

Search strategy

This study used a systematized review approach to identify common data quality elements for health information systems. The following keywords were used in the search strategy: Data quality, Health, clinic, Hospital, Medical, Information system. The keywords chosen were searched using various combinations and in the fields of title, abstract, subject, and keyword. We considered the search features in each database and used the Boolean operators (AND, OR) to combine and search selected keywords. An example of the search strategy was given in Table  1 .

Study selection

All the results were imported into EndNote reference management software. The duplicate and non-journal papers were removed. Next, the title and abstract of the remaining articles were screened to detect subject relevance with the research objectives. The selected articles were analyzed based on the inclusion and exclusion criteria. Finally, the reference lists of all identified articles were searched for additional studies. Two researchers undertook the screening of titles and abstracts obtained through the searches. A sample of just over 20% of articles was double screened in order to assess the level of agreement between the researchers. Disagreements were resolved through discussion or consultation with a third researcher.

Data collection process

Data extraction was completed independently by two assessors. The data were extracted from including four sections: bibliographic information, methodology, and the data quality elements investigated, and key findings. Each study was treated as a single unit of analysis and the relevant information in each study was extracted using a designated data extraction form.

Information was extracted from each included study (including first author, title, publication date, type of study, methodology, processes of knowledge management that were studied and selected results). We emphasize the results of selected papers that have reported elements for assessment data quality in health information systems.

Risk of bias in individual studies

In this study, we used the Joanna Briggs Institute (JBI) checklist [ 24 ] for quality assessment. The authors assessed the included studies with a further random examination by two independent reviewers. The results of the quality assessment were compared any disagreements between the reviewers were addressed through discussion or by involving a third reviewer.

Synthesis of results

In this review, by adopting similar identifies elements as broader themes, the results of the included studies were analyzed and categorized. Finally, the homogeneous data quality elements in health information systems were synthesized and described.

Risk of bias within studies

The JBI checklist was applied to all 58 studies; none were excluded based on quality assessment and all studies were rated as unclear or high risk of bias. In 16% of studies, we cannot find “statement locating the researcher culturally or theoretically” and in 37%, “influence of the researcher on the research” is not addressed.

The search for systematic reviews identified 734 references published between 2003 and 2024. Title and abstract review selected 167 references for full text review. In the analysis, it was found that 68 papers did not address research questions or test hypotheses, 32 papers lacked discussion on data quality dimensions in health management systems, and nine documents presented content in a format other than a scientific article.

Out of the 58 selected paper for final review, 42 were released between 2013 and 2024 [ 1 , 4 , 5 , 7 , 8 , 9 , 10 , 11 , 14 , 15 , 16 , 17 , 18 , 21 , 22 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 ]. Thirteen papers looked at information quality [ 7 , 11 , 14 , 27 , 28 , 29 , 31 , 37 , 52 , 54 , 55 , 56 ], five at content quality [ 7 , 15 , 21 , 43 , 50 ], and thirty-six at data quality [ 4 , 5 , 10 , 14 , 17 , 20 , 21 , 27 , 28 , 29 , 31 , 32 , 33 , 36 , 37 , 42 , 43 , 44 , 47 , 49 , 50 , 51 , 52 , 53 , 55 , 57 , 58 , 59 , 60 ]. None of the publications, however, made a distinction between “data” and “information,” or between “data quality” and “information quality.” As a result, “information quality” and “data quality” were used synonymously [ 21 ]. The search results and the study selection process are presented in Fig.  1 .

Flow diagram of study selection process

Evaluating the quality of the data was the primary goal of the reviewed studies [ 4 , 5 , 10 , 13 , 14 , 15 , 17 , 18 , 19 , 20 , 21 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 35 , 36 , 37 , 38 , 39 , 41 , 42 , 43 , 44 , 45 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 55 , 56 , 57 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 ].Two paper focused on information quality in health systems [ 11 , 52 ]. Methods for evaluating the quality of data were presented in eight publications [ 10 , 20 , 21 , 35 , 38 , 41 , 51 , 52 ], 19 publications tended to conduct on the health information [ 5 , 8 , 10 , 11 , 16 , 17 , 20 , 21 , 22 , 26 , 31 , 37 , 42 , 47 , 49 , 50 , 51 , 55 , 57 , 60 , 66 ] and eight paper focus on health or medical records as an information system in health context [ 13 , 19 , 25 , 38 , 44 , 45 , 64 , 67 ].

To describe data quality, the studies employed a total of 57 dimensions. The first data quality attribute for health information system that was most often used was accuracy [ 4 , 5 , 15 , 17 , 19 , 28 , 29 , 32 , 33 , 34 , 37 , 41 , 43 , 45 , 46 , 49 , 51 , 53 , 59 ], second is completeness [ 4 , 5 , 20 , 28 , 29 , 30 , 41 , 44 , 45 , 46 , 48 , 49 , 51 , 52 , 53 , 56 ], and third most-frequently criterion is timeliness [ 5 , 28 , 41 , 44 , 45 , 51 ]. Table  2 displays the common dimensions of data quality in health information systems that derived from existing literature.

Data accuracy measures the extent to which information accurately represents the objects or events. The accuracy of the information that is gathered, utilized, and stored is assessed through data accuracy. It is imperative for records to serve as a dependable source of information and to facilitate the generation of valuable insights through analysis. Maintaining high data accuracy guarantees that records and datasets meet the standards for reliability and trustworthiness, allowing for their use in decision-making and various applications [ 4 , 5 , 17 , 28 , 29 , 32 , 34 ]. Correctness, precision, free of error, validity, believability and integrity are common terms that use for describe data accuracy [ 21 ]. Data believability relates to whether the data is regarded as being true, real, and credible. Data believability is based on user’s perceptions [ 1 , 36 , 40 ].

Data consistency is the state in which all copies or instances of data are identical across various information systems. This uniformity is crucial in maintaining the accuracy, currency, and coherence of data across different platforms and applications. It is essential for instilling trust in users accessing the data. Implementing data validation rules, employing data standardization techniques, and utilizing data synchronization processes are some strategies to uphold data consistency. By ensuring data consistency, organizations can provide users with reliable information for making informed decisions, streamline operations, minimize errors, and enhance efficiency [ 9 , 45 , 48 , 51 , 52 , 65 ].

Data security is the practice of protecting information from corruption, theft, or unauthorized access throughout its life cycle. This involves safeguarding hardware, software, storage devices, and user devices, as well as implementing access controls, administrative controls, and organizational policies. By utilizing tools and technologies that enhance visibility of data usage, such as data masking, encryption, and redaction, organizations can ensure the security of their data. Moreover, data security assists organizations in streamlining auditing procedures and complying with data protection regulations, ultimately reducing the risk of cyber-attacks, human error, and insider threats [ 5 , 48 , 56 ]. Secure access, safe, confidentiality and privacy are common terms that use for describe data security [ 21 ].

Data timeliness denotes the currency and availability of data at the required time for its intended use. This is critical for enabling health organizations to make swift and accurate decisions based on the most up-to-date information. The timeliness of data has an impact on data quality as it determines the reliability and usefulness of information systems. Moreover, timely data can lead to cost savings as organizations can utilize real-time data to effectively manage inventories, optimize delivery routes, and coordinate with suppliers, thus reducing the risk of stock outs, minimizing delivery delays, and ensuring smooth operations [ 5 , 25 , 28 , 41 , 44 , 45 , 51 ].

Completeness of data refers to the extent to which information includes all necessary elements and observations for a specific purpose. This factor enhances the integrity and reliability of analyses, preventing gaps in understanding and supporting more robust decision-making processes. In a complete dataset, all variables relevant to the presentation of information should be present and fully populated with valid data values. Any missing, incorrect, or incomplete entries in the dataset can compromise the quality of analyses, interpretations, and decisions based on that data [ 4 , 5 , 9 , 28 , 29 , 30 , 41 , 44 , 45 , 52 ]. Coverage, comprehensiveness, appropriate amount, adequate, appropriate amount of data and integrity are common terms that use for describe data completeness [ 21 ]. The amount of data indicates the extent of data sets obtained for analysis and processing. In present-day information systems, these sets of data are frequently observed to be escalating in size, reaching capacities such as terabytes and petabytes [ 4 , 29 , 50 , 57 ].

Data reliability pertains to the uniformity of data across various records, programs, or platforms, as well as the credibility of the data source. Reliable data remains consistently accurate, while unreliable data may not always be valid, making it challenging to ascertain its accuracy. Consequently, organizations cannot depend on unreliable data for decision-making. Data reliability, also referred to as data observability, represents the trustworthiness of data and the insights derived from it for enabling sound decision-making. Reliability is characterized by two other fundamental elements of data quality include accuracy and consistency [ 9 , 49 , 53 , 57 , 59 , 65 ].

Data accessibility refers to the ease with which users can locate, retrieve, comprehend, and utilize data within an organization’s information systems. This is crucial in the modern digital landscape, where data is valuable for decision-making, strategic planning, and operational efficiency. Ensuring data accessibility involves creating an environment where data is available, understandable, and usable by individuals with varying levels of technical expertise. This approach is closely tied to data democratization, which aims to break down silos and make data available across different levels and departments of an organization. A well-implemented data accessibility strategy ensures that data is not locked away in isolated information systems but is integrated and accessible, contributing to a more informed and agile organizational structure. The ultimate goal is to empower users to leverage data in their daily tasks and decision-making processes, thus fostering a data-driven culture [ 4 , 26 , 29 , 33 , 50 , 57 ].

Data Objectivity refers to the extent to which data is free from personal biases, emotions, and subjective interpretations. Objective data is verifiable, reliable, and accurate, meaning that it can be verified independently by multiple parties. In other words, objective data is based on facts rather than opinions or judgments. In the context of information systems, data objectivity is crucial because it enables organizations to make informed decisions based on accurate and reliable information. Objective data helps to reduce errors, inconsistencies, and uncertainties, ensuring that business processes are efficient, effective, and compliant with regulatory requirements. Data objectivity in information systems is often hindered by biases in data collection, data quality issues, information overload, and lack of standardization. Biases may arise from human error, sampling errors, or deliberate data manipulation during the collection process. Inaccuracies, inconsistencies, and incompleteness resulting from poor data quality can compromise the objectivity of the information. The overwhelming amount of data available can make it challenging to differentiate between objective and subjective information. Inconsistencies in data representation and interpretation may occur due to the use of different systems or formats [ 36 , 41 , 44 , 45 , 46 ].

Data relevancy is an aspect of data quality that determines whether the data used or generated are relevant to add to the new target system and how usable it is for users [ 9 , 29 , 45 , 48 , 51 ]. Ease of operation, Usability, applicable, utility, Usefulness, Perceived usefulness and importance are common terms that use for describe data relevancy [ 21 ]. The concept of data usability revolves around a user’s ability to obtain meaningful information from various systems. When data is stored in text files that demand prolonged and intricate processing before it can be analyzed, its usability is limited. Conversely, data that is conveniently displayed on a performance dashboard for immediate interpretation is classified as highly usable [ 4 , 25 , 29 , 45 , 48 , 50 ]. The concept of data usefulness denotes the level at which data, post-analysis, aligns with the intended purpose within a given context for its user or consumer. In most cases, data usefulness is attained when all criteria related to data quality, such as dependability, thoroughness, uniformity, and others, are fulfilled [ 43 , 50 , 52 ].

Data Understandability refer to the level at which data exhibits qualities that facilitate understanding and analysis by users, and are presented in relevant languages, symbols, and measurements within a defined context of utilization [ 22 , 34 , 37 , 46 ]. Interpretability, ease of understanding, granularity and transparency are common terms that use for describe data understandability [ 21 ].

Data navigation refers to the process of searching, locating, and extracting relevant data from a vast pool of information to support decision-making, problem-solving, or analysis. It involves the utilization of different techniques and tools to navigate through extensive data, identify patterns, trends, and correlations, and present the information in a meaningful and actionable way. The success of data navigation is contingent upon several dimensions, including technical, domain knowledge, systems, methodological, and human dimensions. The technical dimension involves mastering programming languages like SQL and Python, utilizing data visualization software such as Tableau and Power BI, and implementing data mining techniques like machine learning algorithms. Domain knowledge dimension stresses the importance of expertise in specific fields. Information system dimension highlights the role of databases, data warehouses, cloud storage platforms, and other technologies in facilitating data navigation by storing, managing, and providing access to data. Methodological dimension focuses on statistical analysis, data mining techniques, and data visualization methods as key approaches to navigating data. Lastly, human dimension recognizes the significance of communication skills, collaboration, and critical thinking in the process of data navigation [ 4 , 50 , 65 , 68 ].

Data reputation is the evaluation of the trustworthiness, reliability, and credibility of data in an information system. It signifies the extent to which stakeholders, such as users, decision-makers, and other systems, perceive the data as accurate, reliable, and complete. Within an information system, data reputation plays a crucial role in decision-making, trust, system performance, and data sharing [ 42 , 60 , 61 ].

The concept of data efficiency revolves around an organization’s effectiveness in maximizing the value obtained from its data, while simultaneously minimizing the resources essential for processing, storing, and up keeping that data. Put simply, data efficiency focuses on streamlining the collection, storage, analysis, and utilization of data to meet objectives. When considering an information system, data efficiency can be examined from various angles, such as efficiency in data acquisition, storage, processing, analysis, visualization, security, retention, and archiving [ 7 , 28 , 29 , 48 ].

Data value-added pertains to the process of refining raw data into more useful, meaningful, and valuable information that can support decision-making, drive business outcomes, and create a competitive advantage. This process involves extracting insights, patterns, or trends from large datasets and presenting them in a manner that is easy to understand and act upon. By prioritizing these dimensions of data value-added within an information system, organizations can ensure that their data is transformed into valuable insights that support informed decision-making and drive business outcomes [ 5 , 22 , 25 , 45 ].

In a few papers, the concept of “fitness for use” was applied to data quality [ 6 , 55 , 69 ]. Two viewpoints can be used to characterize data quality: (1) the inherent quality of the data elements and set, and (2) how the set satisfies the needs of the user. The definition provided by the International Standards Organization best captures the accepted meaning of data quality, which is “the totality of features and characteristics of an entity that bears on its ability to satisfy stated and implied needs” [ 4 , 15 , 28 , 33 , 53 ].

Current review study identified 14 common dimensions for data quality in health information system. In related research data quality dimensions classified on four dimensions include: intrinsic (accuracy, objectivity, reputation), contextual timeliness, completeness, and relevancy), representational (representational format, understandability, consistency), and accessibility (accessibility, security) categories [ 53 , 60 , 69 , 70 , 71 ]. There exists a certain level of intersection between the aspects of data quality recognized in this review and those research in prior classifications of data quality.

Previous literature has often discussed intrinsic data quality in terms of the absence of defects, as indicated by various dimensions such as accuracy, perfection, freshness, and uniformity [ 72 ]. and “completeness, unambiguity, meaningless and correctness” [ 54 , 73 , 74 ]. The Canadian Institute for Health Information put forth a set of 69 quality criteria, organized into 24 quality characteristics, and further classified into 6 quality dimensions: accuracy, timeliness, comparability, usability, relevance, and privacy & security [ 58 , 71 ]. Research on data quality has primarily concentrated on recognizing general quality traits like accuracy, currency, completeness, correctness, consistency, and timeliness as fundamental aspects of data quality applicable across different fields. Nevertheless, existing reviews reveal a lack of consensus regarding the conceptual framework and definition of data quality [ 70 , 73 ]. However, our pervious review shows there is a lack of consensus conceptual framework and definition for data quality [ 1 , 71 ].

In this study, the three most-frequently used dimensions of data quality were accuracy, completeness and timeliness, respectively. This arrangement is somewhat different from previous literature in which the three most-frequently used dimensions were arranged in the order of completeness, accuracy, and timeliness, respectively [ 43 , 51 , 53 ]. Furthermore, the absence of a precise definition of the data quality dimensions led to complexities in evaluating them. The definitions of dimensions and their associated metrics were occasionally based on intuition, past experiences, or the underlying goals. These results indicate that data quality is a multi-faceted phenomenon. Likewise, other scholars argue that data quality is a multi-dimensional notion [ 5 , 28 , 38 , 52 , 61 ].

The Health Information Systems heavily rely on data, as they perform essential functions like generation, compilation, analysis, synthesis, communication, and data application to support decision-making. The literature frequently evaluates the dimensions of data quality, but there is currently a lack of consistency and potential generalizability in using these dimensions and methods to assess data quality in Health Information Systems. In this review of the literature, the data quality for health information system were examined and identified 14 common dimension include: Accuracy, Consistency, Security, Timeliness, Completeness, Reliability, Accessibility, Objectivity, Relevancy, Understandability, Navigation, Reputation, Efficiency and Value- added.

The quality of data in health information systems is indispensable for healthcare institutions to make well-informed decisions and provide patients with optimal care. Accurate and timely data assists healthcare organizations and professionals in identifying patterns, predicting outcomes, and enhancing patient results. Conversely, inadequate data quality in healthcare or other data-related issues can lead to inaccurate diagnoses, inappropriate treatments, and harm to patients. To ensure data quality in healthcare, organizations must prioritize investments in data governance, data management, and data analysis tools, while also maintaining a continuous process of monitoring and improving data quality in health information systems.

It is essential to have high-quality data in order to ensure the safe and dependable delivery of healthcare services. Health facility data plays a crucial role in monitoring performance. While various organizations may prioritize different aspects of data quality, it is important to acknowledge that no health data, regardless of its source, can be deemed flawless. All data are susceptible to various limitations related to data quality, including missing values, bias, measurement error, and human errors in data entry and computation. These limitations are associated with technical, behavioral, and organizational factors [ 75 ].

This study has limitations. Firstly, the number of articles with complete data was relatively small. Secondly, assessing the quality of some studies were difficult because the quality assessment criteria were not clearly identified. We have proposed four fundamental implications to inspire future research. Firstly, it is crucial for researchers to give equal attention to all dimensions of data quality, as these dimensions can have both direct and indirect effects on data quality outcomes. Secondly, researchers should aim to evaluate the existing data quality models and frameworks through a combination of mixed methods and case study designs. Thirdly, it is important to identify the underlying causes of data quality issues in health information systems. Lastly, efforts should be made to develop interventions that can effectively address and prevent data quality issues from occurring.

Data availability

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Joanna Briggs Institute

Liaw S-T, et al. Quality assessment of real-world data repositories across the data life cycle: a literature review. J Am Med Inform Assoc. 2021;28(7):1591–9.

Article   PubMed   PubMed Central   Google Scholar  

WHO. Data Quality Assurance (DQA) . Health Service Data 2022 [cited 2022 2022]; https://www.who.int/data/data-collection-tools/health-service-data/data-quality-assurance-dqa#:~:text=WHO%20has%20produced%20the%20Data,annual%20data%20quality%20desk%20review

FMoH E. Health sector transformation plan . 2015, Addis Ababa, Ethiopia.

Rumisha SF, et al. Data quality of the routine health management information system at the primary healthcare facility and district levels in Tanzania. BMC Med Inf Decis Mak. 2020;20(1):340.

Article   Google Scholar  

Chekol A, et al. Data quality and associated factors of routine health information system among health centers of West Gojjam Zone, northwest Ethiopia, 2021. Front Health Serv. 2023;3:1059611.

Pipino LL, Lee YW, Wang RY. Data quality assessment. Commun ACM. 2002;45(4):211–8.

Ouedraogo M, et al. A quality assessment of Health Management Information System (HMIS) data for maternal and child health in Jimma Zone, Ethiopia. PLoS ONE. 2019;14(3):e0213600.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lemma S, et al. Improving quality and use of routine health information system data in low-and middle-income countries: a scoping review. PLoS ONE. 2020;15(10):e0239683.

Bammidi TR, et al. The crucial role of Data Quality in Automated decision-making systems. Int J Manage Educ Sustainable Dev. 2024;7(7):22.

Google Scholar  

Adane A, et al. Exploring data quality and use of the routine health information system in Ethiopia: a mixed-methods study. BMJ open. 2021;11(12):e050356.

Mohammed SA, Yusof MM. Towards an evaluation framework for information quality management (IQM) practices for health information systems–evaluation criteria for effective IQM practices. J Eval Clin Pract. 2013;19(2):379–87.

Article   PubMed   Google Scholar  

Long J, Seko C. A New Method for Database Data Quality Evaluation at the Canadian Institute for Health Information (CIHI) . in ICIQ . 2002. Citeseer.

Adeleke IT, et al. Data quality assessment in healthcare: a 365-day chart review of inpatients’ health records at a Nigerian tertiary hospital. J Am Med Inform Assoc. 2012;19(6):1039–42.

Singh M, et al. Health management information system data quality under NRHM in District Sonipat, Haryana. Int J Health Sci Res (IJHSR). 2016;6(9):11–4.

CAS   Google Scholar  

Harrison K, Rahimi N. Carolina Danovaro-Holliday, factors limiting data quality in the expanded programme on immunization in low and middle-income countries: a scoping review . Vaccine. 2020;38(30):4652–63.

Shama AT, et al. Assessment of quality of routine health information system data and associated factors among departments in public health facilities of Harari region, Ethiopia. BMC Med Inf Decis Mak. 2021;21(1):1–12.

Bosch-Capblanch X, et al. Does an innovative paper-based health information system (PHISICC) improve data quality and use in primary healthcare? Protocol of a multicountry, cluster randomised controlled trial in sub-saharan African rural settings. BMJ Open. 2021;11(7):e051823.

Ehsani-Moghaddam B, Martin K, Queenan JA. Data quality in healthcare: a report of practical experience with the Canadian Primary Care Sentinel Surveillance Network data. Health Inform Manage J. 2021;50(1–2):88–92.

Brown PJB, Warmington V. Data quality probes—exploiting and improving the quality of electronic patient record data and patient care. Int J Med Informatics. 2002;68(1):91–8.

Lima CR, et al. [Review of data quality dimensions and applied methods in the evaluation of health information systems]. Cad Saude Publica. 2009;25(10):2095–109.

Alipour J, Ahmadi M. Dimensions and assessment methods of data quality in health information systems. Acta Med Mediterranea. 2017;33(2):313–20.

Tolera A et al. Barriers to healthcare data quality and recommendations in public health facilities in dire Dawa city administration, eastern Ethiopia: a qualitative study. Front Digit Health, 2024. 6.

Vrabel M. M. Preferred reporting items for systematic reviews and meta-analyses . In Oncology nursing forum . Oncology Nursing Society; 2015.

JBI QARI Critical appraisal checklist for interpretive & critical research . The Joanna Briggs Institute, Adelaide 2018; http://joannabriggs.org/research/critical-appraisal-tools.html

Fraser HSF, et al. Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the role of clinical Alerts: cross-sectional observational study. JMIR Public Health Surveill. 2024;10:e49127.

Madandola OO, et al. The relationship between electronic health records user interface features and data quality of patient clinical information: an integrative review. J Am Med Inform Assoc. 2023;31(1):240–55.

Getachew N, Erkalo B, Garedew MG. Data quality and associated factors in the health management information system at health centers in Shashogo district, Hadiya Zone, southern Ethiopia, 2021. Volume 22. BMC Medical Informatics and Decision Making; 2022. pp. 1–9. 1.

Solomon M, et al. Data quality assessment and associated factors in the health management information system among health centers of Southern Ethiopia. PLoS ONE. 2021;16(10):e0255949.

Moukénet A, et al. Health management information system (HMIS) data quality and associated factors in Massaguet district, Chad. BMC Med Inf Decis Mak. 2021;21(1):326.

do Einloft N. Data quality and arbovirus infection associated factors in pregnant and non-pregnant women of childbearing age in Brazil: a surveillance database analysis. One Health. 2021;12:100244.

Ayele W et al. Data quality and it’s correlation with routine health information system structure and input at public health centers in Addis Ababa, Ethiopia. Ethiop J Health Dev, 2021. 35(1).

Mulissa Z, et al. Effect of data quality improvement intervention on health management information system data accuracy: an interrupted time series analysis. PLoS ONE. 2020;15(8):e0237703.

Yourkavitch J, Prosnitz D, Herrera S. Data quality assessments stimulate improvements to health management information systems: evidence from five African countries. J Glob Health. 2019;9(1):010806.

Endriyas M, et al. Understanding performance data: health management information system data accuracy in Southern Nations nationalities and people’s Region, Ethiopia. BMC Health Serv Res. 2019;19(1):1–6.

Biancone P, et al. Data quality methods and applications in health care system: a systematic literature review. Int J Bus Manage. 2019;14(4):35–47.

Liu Y, et al. [Designing and implementation of the data quality control in the information system of air pollution and health impact monitoring]. Wei Sheng Yan Jiu. 2018;47(2):277–80.

PubMed   Google Scholar  

Kumar M, et al. Research gaps in routine health information system design barriers to data quality and use in low- and middle-income countries: a literature review. Int J Health Plann Manage. 2018;33(1):e1–9.

Feder SL. Data quality in electronic health records research: quality domains and assessment methods. West J Nurs Res. 2018;40(5):753–66.

Watson NL, et al. Data management and data quality in PERCH, a large international case-control study of severe childhood pneumonia. Clin Infect Dis. 2017;64(suppl3):S238–44.

Wagenaar BH, et al. Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia. BMC Health Serv Res. 2017;17:65–75.

Puttkammer N, et al. Identifying priorities for data quality improvement within Haiti׳s iSanté EMR system: comparing two methods. Health Policy Technol. 2017;6(1):93–104.

Finnegan K, et al. Barriers and facilitators of Data Quality and Use in Malawi’s Health Information System. Annals Global Health. 2017;83(1):36–7.

Chen H, et al. Data Quality of the Chinese National AIDS Information System: a critical review. Stud Health Technol Inf. 2017;245:1352.

Woinarowicz M, Howell M. The impact of electronic health record (EHR) interoperability on immunization information system (IIS) data quality. Online J Public Health Inf. 2016;8(2):e184.

Puttkammer N, et al. An assessment of data quality in a multi-site electronic medical record system in Haiti. Int J Med Informatics. 2016;86:104–16.

Article   CAS   Google Scholar  

Nicol E, Dudley L, Bradshaw D. Assessing the quality of routine data for the prevention of mother-to-child transmission of HIV: an analytical observational study in two health districts with high HIV prevalence in South Africa. Int J Med Informatics. 2016;95:60–70.

Wagenaar BH, et al. Effects of a health information system data quality intervention on concordance in Mozambique: time-series analyses from 2009–2012. Popul Health Metr. 2015;13:9.

Taggart J, Liaw S-T, Yu H. Structured data quality reports to improve EHR data quality. Int J Med Informatics. 2015;84(12):1094–8.

Glèlè Ahanhanzo Y, et al. Data quality assessment in the routine health information system: an application of the Lot Quality Assurance Sampling in Benin. Health Policy Plan. 2015;30(7):837–43.

Glèlè Ahanhanzo Y, et al. Factors associated with data quality in the routine health information system of Benin. Arch Public Health. 2014;72(1):25.

Chen H, et al. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014;11(5):5170–207.

Hahn D, Wanjala P, Marx M. Where is information quality lost at clinical level? A mixed-method study on information systems and data quality in three urban Kenyan ANC clinics. Glob Health Action. 2013;6:21424.

Chen H, Yu P, Wang N. Do we have the reliable data? An exploration of data quality for AIDS information system in China. Stud Health Technol Inf. 2013;192:1042.

Choquet R, et al. The Information Quality Triangle: a methodology to assess clinical information quality , in MEDINFO 2010 . IOS; 2010. pp. 699–703.

Mettler T, Rohner P, Baacke L. Improving data quality of health information systems: a holistic design-oriented approach. 2008.

Sørensen HT, et al. Identification of cases of meningococcal disease: data quality in two Danish population-based information systems during a 14-year period. Int J Risk Saf Med. 1995;7(3):179–89.

Gimbel S, et al. An assessment of routine primary care health information system data quality in Sofala Province, Mozambique. Popul Health Metr. 2011;9:12.

Kerr K, Norris T, Stockdale R. Data quality information and decision making: a healthcare case study. ACIS 2007 proceedings, 2007: p. 98.

Ben Saïd M, et al. A multi-source information System via the internet for end-stage renal disease: Scalability and Data Quality. Stud Health Technol Inf. 2005;116:994–9.

Fletcher DM. Achieving data quality. How data from a pediatric health information system earns the trust of its users. J Ahima. 2004;75(10):22–6.

Bean KP. Data quality in hospital strategic information systems: a summary of survey findings. Top Health Inf Manage. 1994;15(2):13–25.

CAS   PubMed   Google Scholar  

Kelly A, Becker W. Nutrition information systems and data quality requirements. WHO Reg Publ Eur Ser. 1991;34:15–24.

Leitheiser RL. Data quality in health care data warehouse environments . in Proceedings of the 34th annual Hawaii international conference on system sciences . 2001. IEEE.

Ndira S, Rosenberger K, Wetter T. Assessment of data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda). Methods Inf Med. 2008;47(06):489–98.

Article   CAS   PubMed   Google Scholar  

Silva AA, et al. [Evaluation of data quality from the information system on live births in 1997–1998]. Rev Saude Publica. 2001;35(6):508–14.

Woelk GB, Moyo IM, Ray CS. A health information system revised. Part II: improving data quality and utilization. Cent Afr J Med. 1987;33(7):170–3.

Abbasi R, Khajouei R, Sadeqi M, Jabali. Timeliness and accuracy of information sharing from hospital information systems to electronic health record in Iran. J Health Adm. 2019;22(2):28–40.

Elavsky F, Nadolskis L, Moritz D. Data navigator: an accessibility-centered data navigation toolkit. IEEE Trans Vis Comput Graph. 2023;20(1):16–25.

Wang RY. A product perspective on total data quality management. Commun ACM. 1998;41(2):58–65.

Liaw S-T et al. Data quality and fitness for purpose of routinely collected data–a general practice case study from an electronic practice-based research network (ePBRN) . in AMIA Annual Symposium Proceedings . 2011. American Medical Informatics Association.

Rahimi A, et al. Ontological specification of quality of chronic disease data in EHRs to support decision analytics: a realist review. Decis Analytics. 2014;1:1–31.

Redman TC. Measuring data accuracy: A framework and review. Information quality, 2014: pp. 21–36.

Orme AM, Yao H, Etzkorn LH. Indicating ontology data quality, stability, and completeness throughout ontology evolution. J Softw Maintenance Evolution: Res Pract. 2007;19(1):49–75.

Yao H, Orme AM, Etzkorn L. Cohesion metrics for ontology design and application. J Comput Sci. 2005;1(1):107–13.

Endriyas M, et al. Understanding performance data: health management information system data accuracy in Southern Nations nationalities and people’s Region, Ethiopia. BMC Health Serv Res. 2019;19(175):1–6.

Download references

Acknowledgements

Not applicable.

This study was supported by Abadan University of medical sciences, Research code: 1557.

Author information

Authors and affiliations.

Department of Medical library and Information Science, Abadan University of Medical Sciences, Abadan, Iran

Hossein Ghalavand & Zarrin Zarrinabadi

Department of Medical library and Information Science, School of Health Management and Information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran

Saied Shirshahi & Alireza Rahimi

Student Research Committee, Abadan University of Medical Sciences, Abadan, Iran

Fatemeh Amani

You can also search for this author in PubMed   Google Scholar

Contributions

Hossein Ghalavand and Saied Shirshahi Conceived the study, prepared the analysis plan, conducted the analysis, and prepared the draft manuscript. Alireza Rahimi, Zarrin Zarrinabadi and Fatemeh Amani Conceived the study, prepared the analysis plan, performed the literature search, screening for study inclusion/exclusion, and risk of bias assessment, conducted the analysis, and prepared the draft manuscript. All authors contributed to the final version of the manuscript.

Corresponding author

Correspondence to Hossein Ghalavand .

Ethics declarations

Ethics approval and consent to participate.

This research study was approved by Ethics Committee in Biomedical Research at Abadan University of Medical Sciences (Ethical code: IR.ABADANUMS.REC.1401.122).

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ghalavand, H., Shirshahi, S., Rahimi, A. et al. Common data quality elements for health information systems: a systematic review. BMC Med Inform Decis Mak 24 , 243 (2024). https://doi.org/10.1186/s12911-024-02644-7

Download citation

Received : 17 April 2024

Accepted : 21 August 2024

Published : 02 September 2024

DOI : https://doi.org/10.1186/s12911-024-02644-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Data quality
• Health Information System
• Systematic review

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

• General enquiries: [email protected]

• Open access
• Published: 30 August 2024

Barriers in providing quality end-of-life care as perceived by nurses working in critical care units: an integrative review

• Yousef Saleh Rubbai 1 , 2 ,
• Mei Chan Chong 1 ,
• Li Yoong Tang 1 ,
• Khatijah Lim Abdullah 3 ,
• Walid Theib Mohammad 6 ,
• Samira Mohajer   ORCID: orcid.org/0000-0001-7118-1783 1 , 4 &
• Mohammad Namazinia   ORCID: orcid.org/0000-0003-2198-7556 5  

BMC Palliative Care volume  23 , Article number:  217 ( 2024 ) Cite this article

Metrics details

Despite increasing interest in quality end-of-life care (EOLC), critically ill patients often receive suboptimal care. Critical care nurses play a crucial role in EOLC, but face numerous barriers that hinder their ability to provide compassionate and effective care.

An integrative literature review was conducted to investigate barriers impacting the quality of end-of-life care. This review process involved searching database like MEDLINE, Cochrane Central Register of Controlled Trials, CINAHL, EBSCO, and ScienceDirect up to November 2023. Search strategies focused on keywords related to barriers in end-of-life care and critical care nurses from October 30th to November 10th, 2023. The inclusion criteria specified full-text English articles published between 2010 and 2023 that addressed barriers perceived by critical care nurses. This integrative review employs an integrated thematic analysis approach, which combines elements of deductive and inductive analysis, to explore the identified barriers, with coding and theme development overseen by the primary and secondary authors.

Out of 103 articles published, 11 articles were included in the review. There were eight cross-sectional descriptive studies and three qualitative studies, which demonstrated barriers affecting end-of-life care quality. Quality appraisal using the Mixed Method Appraisal Tool was completed by two authors confirmed the high credibility of the selected studies, indicating the presence of high-quality evidence across the reviewed articles. Thematic analysis led to the three main themes (1) barriers related to patients and their families, (2) barriers related to nurses and their demographic characteristics, and (3) barriers related to health care environment and institutions.

This review highlights barriers influencing the quality of end of life care perceived by critical care nurses and the gaps that need attention to improve the quality of care provided for patients in their final stages and their fsmilies within the context of critical care. This review also notes the need for additional research to investigate the uncover patterns and insights that have not been fully explored in the existing literature to enhance understanding of these barriers. This can help to inform future research, care provision, and policy-making. Specifically, this review examines how these barriers interact, their cumulative impact on care quality, and potential strategies to overcome.

Peer Review reports

Introduction

It was estimated that 56.8 million people, including 25.7 million at the end of life, need palliative care; however, only about 14% of people who need palliative care currently receive it [ 1 ]. The need for acute care settings increased in response to life-threatening emergencies and the acute exacerbation of diseases [ 2 , 3 ]. These settings were developed to meet the need for providing optimal health care, saving patient lives and decreasing the rate of mortality using advanced technology [ 2 , 4 ]. Caring in intensive care units sometimes involves withholding or withdrawing treatments that have lasted a lifetime, and in these cases, the role of ICU nurses goes from providing life-saving measures to end-of-life care [ 5 ]. Care at the end of a life is a special kind of health care for individuals and families who are living with a life-limiting illness [ 6 ]. End-of-life care (EOLC) includes a crucial component of intensive care nurses’ work; nurses are in a unique position to cooperate with families to provide care for patients at the end of their lives [ 7 , 8 , 9 , 10 43 ].

Advanced technology in critical care units has led to improved nursing care in many areas, such as End-Of-Life-Care (EOLC) [ 11 ]. This type of care has moved towards enhancing comfort and reducing patients’ suffering [ 12 ]. As EOLC involves enhancing the physical, emotional, and spiritual quality of life for critically ill patients, traditional measures are now challenged as advanced technology has revolutionized nursing care through innovations such as adjustable beds and pressure-relieving mattresses, which help optimize patient comfort, and advanced communication technologies, for example, video conferencing facilitating communication between patients, families, and healthcare providers, allowing for ongoing support, counseling, and decision-making discussions throughout the end-of-life journey. Therefore, quality EOLC has become a significant concern for healthcare decision-makers, healthcare providers, researchers, patients, and families [ 13 ]. Despite the increased interest and demand in providing good EOLC, this care is still limited In the critical care and does not meet the recommended standards [ 14 ]. Critical Care Nurses spend more time with patients compared to other members of the multidisciplinary team. They serve as implementers, educators, and coordinators in end-of-life care. Their role in delivering EOLC is essential as they are presumably prepared to provide this care and meet patients and their family’s needs, including pain control, management of physical, emotional, spiritual, and social needs, and communication with patients and their families [ 15 ]. Therefore, it is important to look into the factors that impede the provision of quality end-of-life care from their perspectives. Many barriers affecting the provision of EOLC in critical care areas have been reported in the literature [ 13 , 16 , 17 ].

End-of-life care (EOLC) involves caring for and managing terminally ill patients and families. The quality of EOLC in critical care units has been evaluated based on factors such as patient/family involvement in decision-making, professional communication between health professionals and patients/families, care quality, support types, illness and symptom management, spirituality, and organizational support for critical care nurses [ 18 ]. Furthermore, working in a critical care unit environment is stressful and emotionally taxing for health professionals such as nurses. Carers of terminally ill patients may experience distressing emotions such as helplessness, loss of power, sadness, and hopelessness [ 18 ]. These feelings make it difficult to provide optimal end-of-life care. Additionally, nurses focus on managing symptoms, disease prognosis, treatment options, and physical aspects, but in fact, caring in critical care units follows a universal and holistic model. Previous research has shown that patients and families are not receiving adequate care at the end of life.

Researchers categorized factors that affect EOLC into barriers and challenges [ 13 ]. Barriers have been classified into three categories: patient and family-related, nurses and other health care workers’ related, and health care institutions’ related [ 16 , 17 ].

Barriers related to communication between health care providers and patients and families and characteristics of critical care nurses, including nurses’ age, gender, educational level, and end-of-life care training, significantly affect providing good EOLC [ 19 , 20 , 21 , 22 , 23 ]. this integrative review aims to go beyond merely identifying and categorizing barriers. By synthesizing results from a wide range of studies, the review seeks to uncover patterns and insights that have not been fully explored in the existing literature to enhance understanding of these barriers. This can help to inform future research, care provision, and policy-making. Specifically, this review will examine how these barriers interact, their cumulative impact on care quality, and potential strategies to overcome Despite the fact that EOLC is decisive to patient care, appropriate provision of this service is still lacking in several aspects. In the ICUs, EOLC must be considered an essential factor. However, owing to the existing practices of nurses, the adequate delivery of EOLC tends to bear various inefficiencies.

Nurses and other healthcare staff seem to come across multiple barriers that hinder their ability to offer effective care to critically ill patients. Considering the given dearth of research in this context, we intend to present a comprehensive insight into the issue. In this review, we focused on EOLC provided by critical care nurses, who were defined as nurses dealing with patients suffering from acute health problems due to injury, surgery, or exacerbated chronic diseases and need close monitoring in units such as intensive care units (surgical, medical, and pediatric) and cardiac care units. Due to the importance of exploring these barriers in determining the quality of EOLC, this integrative review paper was conducted to examine and highlight evidence from the literature on these barriers that affect the provision of quality EOLC. This paper explores and identifies current published peer-reviewed studies addressing barriers that affect the quality of EOLC as perceived by critical care nurses. This integrative review seeks to answer the following question: What barriers affect the quality of end-of-life care perceived by nurses working in critical care units?

An integrative review design was the most suitable method to explore and produce a new understanding from various types of literature (experimental, non-experimental, and theoretical) to enhance understanding of the phenomenon under investigation (i.e., EOLC). This method also facilitated nursing science by informing further research, care provision, and policy-making. It also highlights strengths, weaknesses, limitations, and gaps in knowledge, and supports what is already known about theories relevant to our topic [ 24 ]. Therefore, this design helps meet this review’s purposes.

Search strategies

The search process involved four phases which were developed by the first author (YR) and validated by two expert authors (MCC and KLA) as follows: (1) identifying the problems related to the research question, (2) conducting a systematic literature search, (3) screening the articles to develop themes, and (4) performing critical analysis to develop the themes.

From October 30, 2023, to November 10, 2023, electronic literature searches were conducted using major databases such as MEDLINE, Cochrane, CINAHL, EBSCO, and ScienceDirect.

Search methods were defined using the MeSH (Medical Subject Headings) descriptors of the keywords “end-of-life care,” “barriers,” and “critical care nurses.” Additionally, the reference lists of all identified articles were manually searched for additional studies. The operators used in this search included “AND” and “OR,” as well as the truncation tools of each database. A refined search was performed with terms such as “critical care nurses’ perceptions” OR “opinions” AND “quality end-of-life care” OR “quality of death and dying.” Subsequently, terms like “barriers” OR “obstacles” OR “challenges” AND “quality end-of-life care” OR “quality of death and dying” were employed. Finally, the descriptors “critical care nurses’ perceptions,” “barriers,” and “quality end-of-life care” were used (Fig.  1 ).

PRISMA search flow diagram

Inclusion and exclusion criteria

The inclusion criteria for this search to select relevant articles were as follows: (1) Full-text articles, (2) Papers published in the English language from 2010 to 2023, and (3) Articles that specifically describe the barriers perceived by critical care nurses that affect the quality of end-of-life care.

Intervention studies and studies that describe barriers to providing quality end-of-life care from other perspectives, such as physicians and patients’ families were excluded. For the studies who included nurses and other health care workers within the context of critical care, the researchers included the results that relevant to nurses and excluded the others.

Data extraction

The data extraction and analysis were carried out to collect and consolidate the data from the selected studies into a standard format relevant to the research field. The extracted data included specific descriptions of the settings, populations, study methods, and outcome measures (Table  31 ). Two authors (YSR and KLA) independently extracted the data and reached an agreement after discussion with the third author (MCC).

Included and excluded studies

Following the review process, the authors made the final decision on studies that met the study criteria. Out of a total of 103 articles, 9 duplicates were removed. The abstracts of the remaining 94 articles were initially found to be somewhat relevant to the research topic. However, after examining the articles in terms of research methodology and results, 36 articles that matched the selection criteria for this study were ultimately chosen. The full text of the 36 articles was reexamined based on the title first for suitability. Subsequently, the abstracts of the studies were reviewed, leading to the exclusion of 23 articles for various reasons, leaving 13 studies for further consideration in this study. However, two articles were disqualified as they did not contain a specific research methodology or reviewed literature papers; they relied solely on theoretical information. This step resulted in the inclusion of 11 research articles in this integrative review of the literature (Table  1 ).

Quality appraisal

To ensure the methodology’s quality and avoid bias in the design, highly credible and respected search engines were adopted to select peer-reviewed studies according to the inclusion criteria in this review. The articles chosen in this review were categorized into two sections based on study design and research methodology: quantitative and qualitative studies. These were evaluated manually and independently for each study, with any disagreements resolved by two experts (KLA, Professor, and MCC, Associate Professor) who have experience in research methodology, using the Mixed Methods Appraisal Tool (MMAT) version 2018 [ 25 ]. This tool includes specific criteria for evaluating the quality of quantitative, qualitative, and mixed-method studies. The MMAT consists of a checklist of five research components for each type of study with a rating scale including “Yes,” “No,” and “Can’t tell.” The overall results suggest that the evidence quality across the ten studies was high (Table  2 ).

Data synthesis

Thematic analysis in this review involves a systematic process of coding and theme development, using both inductive and deductive approaches. This method ensures a comprehensive synthesis of diverse data sources, providing valuable insights into the research topic [ 24 , 26 ]. Thematic analysis was employed for all studies to investigate the subject of interest. The coding for the themes in this review followed the six recommended phases: Familiarizing with the data; making initial codes; searching for themes; reviewing themes and making a thematic plan; defining and naming themes; generating the final picture of the report [ 24 ]. The coding was conducted by the primary author (YSR) and confirmed by the three secondary authors (LH, SM, and LY). Any discrepancies were discussed and resolved through consensus.

Search outcomes

The search process yielded a total of 103 articles. All articles resulting from the search process were independently reviewed by all authors in this study for the research process, purpose, methodology, tools, main findings, recommendations, and limitations.

Characteristics of included studies

Eight cross-sectional descriptive studies and three qualitative studies were selected, which were conducted in the following countries: two from the USA [ 27 , 28 ] and a single study from each of the following countries: Saudi Arabia [ 22 ], Jordan [ 29 ], Egypt [ 12 ], Malaysia [ 13 ], Scotland [ 30 ], Poland [ 31 ], Hong Kong [ 32 ], South Africa [ 33 ], and China [ 34 ].

In this comprehensive analysis of 11 studies, a diverse range of methodologies and findings were examined across different countries and healthcare settings. The studies included a mix of quantitative and qualitative approaches, with sample sizes varying from small convenience samples to larger cohorts. Key barriers to providing End of Life Care (EOLC) were identified, such as challenges in communication with families, lack of support from managers, and insufficient training in EOLC. The studies highlighted the importance of addressing these barriers to improve the quality of care provided by nurses in critical care settings. Notably, demographic characteristics and their impact on EOLC provision were not consistently addressed across the studies, indicating a potential area for further research and exploration in this field (Table  31 ).

The thematic analysis of included studies revealed several key themes and sub-themes related to barriers in End of Life Care (EOLC). These themes encompassed various aspects, including challenges related to patients and their families, healthcare institutions and the environment, as well as barriers specific to nurses. Communication and collaboration between patients, nurses, and families included issues such as seeking updates about patient status, misunderstandings about life-saving measures, misunderstanding poor prognosis, troubled family dynamics, and conflicts within families regarding life support decisions [ 22 , 34 ]. Additionally, barriers related to Institution Policy and procedures highlighted concerns such as insufficient standard procedures, communication challenges in decision-making, inadequate ICU design, inappropriate staffing policies, and deficiencies in rooms, supplies, and noise control. Furthermore, barriers associated with nurses encompassed their emotional experiences and socio-demographic characteristics [ 12 ] (Table  4 ).

Among the results of the selected articles on nurses’ perceptions of barriers affecting quality EOLC, three main themes were identified: (1) Communication and collaboration between patients, nurses, and families (2) Institution Policy and procedural barriers, and (3) barriers related to nurses and their demographics. An overlap in some of these areas, such as the themes addressing barriers related to patients and their families, was identified [ 11 , 22 , 35 ]. This overlap indicates a high level of consensus between the authors in identifying the barriers affecting the quality of end-of-life care.

Communication and collaboration between patients, nurses, and families

After reviewing the existing body of literature in this domain, it was observed that some familial factors had been largely perceived as prominent barriers to providing EOLC by the nurses. Although some authors concluded family issues as the highest-ranking concern for nurses in providing quality EOLC, there were variations in the type of barriers they encountered [ 11 , 28 , 35 ]. For example, continuous requests for updates on patients’ status from their families were identified as the top-rated barrier affecting the quality of EOLC from the perspective of critical care nurses. In addition, family misunderstandings about life-saving measures, as well as doubts and uncertainties regarding prognosis, resulted in a lack of time for nurses to provide quality EOLC, as they spent significant time explaining these matters [ 29 ]. Similarly, continuous phone calls from family members seeking updates on patients’ conditions were ranked highest (M = 4.23) among barriers affecting EOLC [ 28 ]. Additionally, dealing with distressed family members also received the highest total mean score (M = 3.3) [ 13 ]. On the contrary, another study found that out of 70 nurses, the practice of calling nurses for updates on patients’ conditions had the lowest impact on EOLC practice (62.2%), while misunderstanding about life-saving measures (65.7%) played a crucial role in determining the quality of EOLC [ 36 ]. The study concluded that the primary barrier related to patients and their families was the lack of understanding among family members about what life-saving measures entailed. Similarly, another source also reported consistent findings indicating that families often did not accept poor prognoses for patients and struggled to grasp the significance of life-saving measures [ 22 ].

Furthermore, previous studies have indicated that barriers affecting EOLC and thereby the quality of care include the presence of family members with patients, inadequate communication with patients’ families, lack of involvement in discussions about patient care decisions, conflicts among family members regarding decisions to cease or continue life support treatment, and unrealistic expectations regarding prognosis [ 22 , 30 , 37 ].

Communication and collaboration among doctors and nurses are vital in designing an effective healthcare plan for patients. However, inadequate and inappropriate collaboration and support, such as conflicting opinions, disagreements, and insufficient cooperation between them, can lead to various difficulties that may result in poor patient care [ 22 ]. Research scholars who have conducted studies in this area have acknowledged that agreement between nurses and physicians regarding care directions for patients at the end of life is one of the most critical barriers to enhancing the quality of EOLC [ 29 ].

Similarly, another study found that poor communication between nurses and physicians resulted in inappropriate decision-making and disagreement about care plans, which subsequently impacted the quality of care [ 13 ]. Additionally, inadequate and poor communication between nurses and other healthcare teams diverted attention from the goal of care [ 28 ].

Failures in communication between nurses and other healthcare providers can lead to misunderstandings of care messages, which can affect EOLC practices [ 30 ]. It also highlighted the lack of communication and cooperation between doctors and other healthcare team members; nurses emphasized the need for a communication training course [ 11 ].

Good communication between nurses and physicians and consideration of nurses’ opinions were found to enhance the quality of EOLC [ 12 ]. Furthermore, educating critical care nurses about communication and collaboration skills was reported as crucial for improving the quality of EOLC [ 13 ].

Barriers related to nurses

The given three sub-themes were identified regarding the impact of nurses-related barriers and the influence of some of their demographic factors on the quality of EOLC:

Lack of opportunities for training and education.

Emotional and psychological issue.

Nurses’ socio -demographic factors.

Lack of opportunities for training and education

It was reported that critical care nurses were not adequately prepared to provide EOLC; nurses needed to increase their knowledge about cultural aspects, ethical issues, skills, communication, and training regarding the continuity of care and the management of physical and psychosocial symptoms [ 11 , 13 , 28 ]. Furthermore, nurses who did not participate in any EOLC training course perceived more barriers to delivering quality EOLC than those who had participated in introductory training courses [ 13 , 28 ]. Attia et al. [ 12 ]. reported that 60% of critical care nurses perceived that they had received poor education and training concerning family grieving, symptom management, and quality EOLC. Furthermore, Holms et al. [ 30 ]. found that all participants acknowledged that they had received very little formal education and training on EOLC, particularly those who worked in intensive care. In a study by Jordan et al. [ 37 ], nurses emphasized that EOLC education is essential during the orientation period before starting their ICU jobs.

Emotional and psychological issue

Five articles in this review have studied the effect of nurses’ feelings and emotions as barriers to providing quality EOLC [ 11 , 13 , 28 , 30 , 37 ]. Nurses stated that they feel sad when they cannot help the patients to die peacefully, and they lack emotional support, considering this one of the main barriers to providing EOLC [ 11 ]. Staff morale distress was reported repeatedly during interviews with ICU nurses about their experience of EOLC. This feeling of despair is accompanied by many causes, such as lack of staff experience, poor communication, inadequate training about EOLC, lack of a suitable environment, and lack of support from senior staff [ 30 ]. Nurses acknowledged that they felt like they were participating in decisions to withdraw or withhold life-sustaining treatment, resulting in conflicting emotions and feeling helpless in advocating for the patients with mixed feelings of sadness, grief, anger, and frustration [ 37 ]. Lastly, Crump et al. [ 28 ] and Omar Daw Hussin et al. [ 13 ] observed that critical care nurses received inadequate emotional support from managers and experts within healthcare institutions, which affects the quality of EOLC they provide.

Nurses’ socio -demographic factors

It has been identified that some socio-demographic characteristics of nurses also play a significant role in shaping their opinions regarding perceived barriers. For example, age, education, experience in the field, and other similar factors profoundly impact their perceptions of the barriers to providing EOLC. A study by Omar Daw Hussin et al. [ 13 ] revealed that nurses ( n  = 553) aged 21–30 years old had the highest mean total score for barrier factors to provide quality EOLC compared to other age groups. This was also higher in diploma holders than in nurses with certificates and bachelor’s degrees. Regarding years of experience as critical care nurses, they found that nurses with minimal years of experience (1–10 years) had the highest mean total score for difficulties. Similarly, Chan et al. [ 38 ] found that nurses’ age, qualifications, and experience in caring for patients at EOL were significantly associated with their perceived barriers. Nurses’ distress in intensive care units was linked to various factors, one of which is the lack of experience in providing EOLC, as reported by Holms et al. [ 30 ].

Institution Policy and procedural barriers

Healthcare facilities and the surrounding environment where patients stay have a significant influence on their quick recovery, mental and physical health, as well as health progress [ 11 ]. Therefore, healthcare institutions ought to establish a healthy environment for patients’ well-being. However, in the current review, it was understood that nurses identified a group of barriers related to hospital settings, such as the insufficiency of standard procedures pertaining to EOLC in place at the institution, inappropriate staffing policies in the ICU, lack of rooms prepared for EOLC, insufficient supplies to assist families in EOLC, and a noisy environment with bright lights in patients’ rooms [ 11 ]. Likewise, researchers concluded that intensive care unit nurses face time constraints due to heavy workloads; they also reported that intensive care units have poor designs that interrupt patients’ privacy and affect the provision of quality EOLC [ 12 , 28 ]. Previous studies identified a lack of EOLC rules and guidelines governing the provision of quality EOLC in critical care units, such as limited visiting hours, guiding preferred care pathways, and excessive paperwork burdens [ 12 , 13 , 30 ].

In this section, we discuss the results of this review on the barriers to providing quality end-of-life care derived from the literature and compare them with the results of previous studies.

The themes emerging from the data helped us understand that some familial factors play a decisive role in hindering timely and effective EOLC provision to patients. Our findings are consistent with Beckstrand et al. [ 36 ] and Friedenberg et al. [ 39 ], who also found that families’ lack of understanding or insufficient understanding of the life-saving measures performed for patients often contributes to delayed EOLC provision, due to their ambiguous opinions and uncertainty about the treatment given. Additionally, before taking any action, barriers related to other factors such as cultural aspects, not covered in this paper, should not be disregarded as they may have a significant influence on the outcomes.

There was agreement among all the authors in this review that communication and collaboration issues were at the forefront of factors that affect the quality of EOLC.in critical care setings, poor communication and collaboration between nurses and physicians makes nurses perceive their roles as secondary in the decision-making process. Additionally, critical care nurses also noted that interrupted communication leads to misunderstandings and conflicts in decision-making, diverting them from the goal of EOLC. It was also agreed that communication breakdown and conflicts in decision-making among healthcare teams impact the quality of care for patients with chronic end-stage diseases [ 40 ].

Reviewing the selected studies made us aware that nurses perceived inadequate training and education about EOLC significantly impacts their practice in delivering quality EOLC. The nurses also acknowledged the importance of receiving training and education regarding EOLC, such as symptom management, dealing with grieving families, and communication skills during the orientation period before starting their work in critical care units. Therefore, critical care nurses need to enhance their knowledge about cultural aspects, ethical issues, communication skills, and training related to the continuity of care and the management of physical and psychosocial symptoms [ 36 ].

Apart from training issues, we found that the feeling of not being able to provide proper care to some patients, consistent distress due to increased workload, or managing patients with critical conditions such as prolonging unavoidable death could be attributed to their deteriorating mental health, which they perceive as a barrier to offering EOLC. These results were also supported by Calvin et al. [ 41 ], who found that novice cardiac care unit nurses expressed more fear and discomfort while caring for dying patients and communicating with their families.

This review further shows that healthcare organizations lack policies and guidelines that govern EOLC, such as staffing policies and scheduling visiting hours, leading to a shortage of nurses, increased workload, and decreased presence of family members with their patients. This lack of policies was also indicated in their study [ 36 ]. Critical care units in this review have a poor design that challenges nurses when providing EOLC and interrupts patient privacy. This is consistent with Sheward et al. [ 42 ], who found that the poor design of critical care units may compromise patients’ confidentiality and affect the provision of quality EOLC.

In summary, our findings revealed that some familial factors play a decisive role in hindering timely and effective EOLC provision to patients. Moreover, nurses perceived that inadequate training and education about EOLC significantly impact their practice in providing good EOLC. Therefore, these aspects of our results are confirmed by broader literature, as evidenced before. The ceuurent review highlights the importance of enhancing family communication throught the needs for conducting education and training programs among health care profesionals in crirical care settings about communication skills. Additionally, healthcare organizations lack policies and guidelines that lead to a shortage of nurses, increased workload, and decreased family members’ presence with their patients, governing EOLC. Thus, this integrative review addresses the question of what barriers affect the quality of end-of-life care as perceived by nurses working in critical care units. Combining diverse methodologies can lead to inadequate rigor, imprecision, bias, flawed analysis, synthesis, and deductions. Therefore, there is a need for future studies to further refine the key indicators.

Strengths and limitations

The selected studies were conducted in several countries, which may enhance the generalizability of the study findings. The limitations of this review study are that it focused mainly on descriptive and non-experimental studies. Additionally, the assessment of quality appraisal for selected studies was subjective to the authors according to MMAT, which could affect the studies’ appraisal. The selection of only English articles may introduce bias regarding barriers beyond EOLC in countries where English is not commonly spoken.

The review indicated that healthcare organizations must provide critical care nurses with evidence-based pathways and guidelines to guide them in providing EOLC, increase emotional support from nursing managers and supervisors, and improve critical care settings design. Further studies need to be conducted on the barriers that affect the quality of EOLC and suggestions to overcome these barriers at the level of patients and families, nurses, physicians, other healthcare providers, and healthcare organizations to enhance teamwork and collaboration and improve the quality of EOLC.

This review also calls for additional research to be conducted to explore the barriers that affect the quality of end-of-life care. These studies should investigate barriers at multiple levels, including those affecting patients and families, nurses, physicians, other healthcare providers, and healthcare organizations. By identifying and understanding these barriers, recommendations can be made to overcome them, ultimately enhancing teamwork, collaboration, and the overall quality of end-of-life care.

International implications for practice

Many tools can be easily used to assess barriers to end-of-life care in critical care settings. We recommend monitoring and evaluating them regularly among nurses because they are significantly linked to the quality of end-of-life care. Furthermore, we advise to assess the quality of end-of-life care from patients and their families perspectives and provide them with greif and emotional support if they are unable to contribute in providing feedback that help in assissing the quality of end-of- life care. Refreshing training and education courses about end-of-life care aspects are significantly associated with the quality of care. We advise nursing management to conduct such courses for critical care nurses periodically. In general, there is an opportunity for improvement in terms of the quality of end-of-life care in critical care settings. As the critical care unit is part of a larger institution, it is worthwhile for the hospital’s management to adjust their policies regarding staffing, ICU design, visiting hours, and provide evidence-based guidelines so they can enhance the quality of end-of-life care.

Data availability

The data used to support the findings of this study are included within the article.

Abbreviations

End-Of-Life-Care

Mixed Method Appraisal Tool

World Health Organization W. palliative care key facts 2022, 2022(8 aug 2022).

Ransea K, Yatesb P, Coyer F. Modelling end-of-life care practices: factorsassociated with critical care nurseengagement in care provision. Intensive Crit Care Nurs. 2016;33:48–55.

Article   Google Scholar  

Haji Ali Beigloo R, Mohajer S, Eshraghi A, Mazlom SR. Self-administered medications in Cardiovascular Ward: a study on patients’ Self-efficacy, knowledge and satisfaction. Evid Based Care. 2019;9(1):16–25.

Google Scholar  

Ghasemi A, Karimi Moonaghi H, Mohajer S, Mazlom SR, Shoeibi N. Effect of self-management educational program on vision-related quality of life among elderly with visual impairment. Evid Based Care. 2018;8(1):35–44.

Kisorio LC, Langley GCJI, Nursing CC. Intensive care nurses’ experiences of end-of-life care. 2016, 33:30–8.

Fraser J. Palliative and End-Of-Life Care Provincial Roundtable Report. Queen’s Print Ont 2016 2016:1–31.

Ranse K, Yates P, Coyer F. End-of-life care practices of critical care nurses: a national cross-sectional survey. Australian Crit Care. 2016;29(2):83–9.

Namazinia M, Mazlum SR, Mohajer S, Lopez V. Effects of laughter yoga on health-related quality of life in cancer patients undergoing chemotherapy: a randomized clinical trial. BMC Complement Med Ther. 2023;23(1):192.

Article   PubMed   PubMed Central   Google Scholar  

Mohajer S, Mazlum SR, Rajabzadeh M, Namazinia M. The effect of laughter yoga on depression in cancer patients undergoing chemotherapy: a randomized clinical trial. Hayat. 2022;28(3):284–95.

Bagheri S, Valizadeh Zare N, Mazlom SR, Mohajer S, Soltani M. Effect of implementing family-centered empowerment model on burden of care in caregivers of the elderly with Parkinson’s disease. Evid Based Care. 2019;9(3):41–8.

Ozga D, Woźniak K, Gurowiec PJ. Difficulties perceived by ICU nurses providing end-of-life care: a qualitative study. Global Adv Health Med. 2020;9:2164956120916176.

Attia AK, Abd-Elaziz WW, Kandeel NA. Critical care nurses’ perception of barriers and supportive behaviors in end-of-life care. Am J Hospice Palliat Medicine ® . 2013;30(3):297–304.

Omar Daw Hussin E, Wong LP, Chong MC, Subramanian P. Nurses’ perceptions of barriers and facilitators and their associations with the quality of end-of‐life care. J Clin Nurs. 2018;27(3–4):e688–702.

PubMed   Google Scholar  

Gott M, Ingleton C, Bennett MI, Gardiner C. Transitions to palliative care in acute hospitals in England: qualitative study. BMJ 2011, 342.

Blackman I, Adesina MT, Zannettino L, DeBellis DA. Factors that influence graduating nursing students’ consensus about learning and end of life nursing care Background; Graduating nursing students’ attitudes and behaviours towards learning and end of life nursing care. ergo 2014, 3(3).

Dening KH, Greenish W, Jones L, Mandal U, Sampson EL. Barriers to providing end-of-life care for people with dementia: a whole-system qualitative study. BMJ Supportive Palliat care. 2012;2(2):103–7.

Dufour B. Barriers Faced by Nurses Caring For End-of-Life Patients on a Medical Surgical Unit. 2018.

Sharour LA, Subih M, Salameh O, Alrshoud M. End-of-life care (EOLC) in Jordanian critical care units: barriers and strategies for improving. Crit Care Shock. 2019;22:88–97.

Ayed A, Sayej S, Harazneh L, Fashafsheh I, Eqtait F. The nurses’ knowledge and attitudes towards the Palliative Care. J Educ Pract. 2015;6(4):91–9.

Cavaye J, Watts JH. End-of-life education in the pre-registration nursing curriculum: patient, carer, nurse and student perspectives. Volume 17. London, England: Sage Publications Sage UK; 2012. pp. 317–26.

Coffey A, McCarthy G, Weathers E, Friedman MI, Gallo K, Ehrenfeld M, Chan S, Li WH, Poletti P, Zanotti R. Nurses’ knowledge of advance directives and perceived confidence in end-of‐life care: a cross‐sectional study in five countries. Int J Nurs Pract. 2016;22(3):247–57.

Mani ZA, Ibrahim MA. Intensive care unit nurses’ perceptions of the obstacles to the end of life care in Saudi Arabia. Saudi Med J. 2017;38(7):715.

Sinuff T, Dodek P, You JJ, Barwich D, Tayler C, Downar J, Hartwick M, Frank C, Stelfox HT, Heyland DK. Improving end-of-life communication and decision making: the development of a conceptual framework and quality indicators. J Pain Symptom Manag. 2015;49(6):1070–80.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546–53.

Article   PubMed   Google Scholar  

Hong QN, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P. Gagnon M-P, Griffiths F, NICOLAU B: mixed methods appraisal tool (MMAT) version française 2018. In.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2008;52(5):546–53.

Beckstrand RL, Lamoreaux N, Luthy KE, Macintosh JL. Critical care nurses’ perceptions of end-of-life care obstacles: comparative 17-year data. %J Dimensions Crit Care Nurs. 2017;36(2):94–105.

Crump SK, Schaffer MA, Schulte E. Critical care nurses’ perceptions of obstacles, supports, and knowledge needed in providing quality end-of-life care. Dimens Crit Care Nurs. 2010;29(6):297–306.

Sharour LA, Subih M, Salameh O, Alrshoud M. End-of-life care (EOLC) in Jordanian critical care units: barriers and strategies for improving. Crit Care Shock 2019, 22(2).

Holms N, Milligan S, Kydd A. A study of the lived experiences of registered nurses who have provided end-of-life care within an intensive care unit. Int J Palliat Nurs. 2014;20(11):549–56.

Ozga D, Woźniak K, Gurowiec PJ. Difficulties perceived by ICU nurses providing end-of-Life Care: a qualitative study. J Global Adv Health Med. 2020;9:2164956120916176.

Chan CW, Chow MC, Chan S, Sanson-Fisher R, Waller A, Lai TT, Kwan CW. Nurses’ perceptions of and barriers to the optimal end‐of‐life care in hospitals: a cross‐sectional study. %J J Clin Nurs. 2020;29(7–8):1209–19.

Jordan PJ, Williams M, Clifford I. The experiences of critical care nurses with regard to end-of-life issues in the intensive care unit. Afr J Nurs Midwifery. 2014;16(2):71–84.

Xu D-d, Luo D, Chen J, Zeng J-l, Cheng X-l, Li J, Pei J-j, Hu F. Nurses’ perceptions of barriers and supportive behaviors in end-of-life care in the intensive care unit: a cross-sectional study. BMC Palliat Care. 2022;21(1):1–10.

Beckstrand RL, Lamoreaux N, Luthy KE, Macintosh JL. Critical care nurses’ perceptions of end-of-life care obstacles: comparative 17-year data. Dimens Crit Care Nurs. 2017;36(2):94–105.

Beckstrand RL, Callister LC, Kirchhoff KT. Providing a good death: critical care nurses’ suggestions for improving end-of-life care. Am J Crit Care. 2006;15(1):38–45.

Jordan P, Clifford I, Williams M. The experiences of critical care nurses with regard to end-of-life issues in the intensive care unit. Afr J Nurs Midwifery. 2014;16(2):71–84.

Chan CW, Chow MC, Chan S, Sanson-Fisher R, Waller A, Lai TT, Kwan CW. Nurses’ perceptions of and barriers to the optimal end‐of‐life care in hospitals: a cross‐sectional study. J Clin Nurs. 2020;29(7–8):1209–19.

Friedenberg AS, Levy MM, Ross S, Evans LE. Barriers to end-of-life care in the intensive care unit: perceptions vary by level of training, discipline, and institution. J Palliat Med. 2012;15(4):404–11.

Stajduhar K, Sawatzky R, Robin Cohen S, Heyland DK, Allan D, Bidgood D, Norgrove L, Gadermann AM. Bereaved family members’ perceptions of the quality of end-of-life care across four types of inpatient care settings. BMC Palliat care. 2017;16:1–12.

Calvin AO, Lindy CM, Clingon SL. The cardiovascular intensive care unit nurse’s experience with end-of-life care: a qualitative descriptive study. Intensive Crit Care Nurs. 2009;25(4):214–20.

Sheward K, Clark J, Marshall B, Allan S. Staff perceptions of end-of-life care in the acute care setting: a New Zealand perspective. J Palliat Med. 2011;14(5):623–30.

Nia MN, Mohajer S, Ghahramanzadeh M, Mazlom SR: Effect of Laughter Yoga on Mental Well-Being of Cancer Patients Undergoing Chemotherapy. Journal of Evidence-based Care 2019, 9(3).

Download references

Acknowledgements

Not applicable.

This research received no external funding.

Author information

Authors and affiliations.

Department of Nursing Sciences, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia

Yousef Saleh Rubbai, Mei Chan Chong, Li Yoong Tang & Samira Mohajer

Princess Aisha bint AL-Hussein College of Nursing and Health Science, Al-Hussein Bin Talal University, Maan, Jordan

Yousef Saleh Rubbai

Department of Nursing, School of Medical and Life Science, Sunway University, Bandar Sunway, 46200, Malaysia

Khatijah Lim Abdullah

Nursing and Midwifery Care Research Center, Mashhad University of Medical Sciences, Mashhad, Iran

Samira Mohajer

Department of Nursing, School of Nursing and Midwifery, Torbat Heydariyeh University of Medical Sciences, Torbat Heydariyeh, Iran

Mohammad Namazinia

Faculty of medicine, Yarmouk University , Irbid, Jordan

Walid Theib Mohammad

You can also search for this author in PubMed   Google Scholar

Contributions

YSR and KLA set up the search strategy, with the verification of MCC. MCC, LYT, WTM analyzed results. All authors wrote and approved the final manuscript. SM, MN provided critical review and significant revision of the manuscript for important intellectual content, proof-read, and supervised the preparation of the manuscript.

Corresponding authors

Correspondence to Yousef Saleh Rubbai or Mei Chan Chong .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Rubbai, Y.S., Chong, M.C., Tang, L.Y. et al. Barriers in providing quality end-of-life care as perceived by nurses working in critical care units: an integrative review. BMC Palliat Care 23 , 217 (2024). https://doi.org/10.1186/s12904-024-01543-y

Download citation

Received : 17 August 2023

Accepted : 08 August 2024

Published : 30 August 2024

DOI : https://doi.org/10.1186/s12904-024-01543-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Quality end-of-life care
• Critical care units

BMC Palliative Care

ISSN: 1472-684X

• General enquiries: [email protected]

Primorsky Krai

• Interim Governor of the Primorsky Krai: Tarasenko Andrei Vladimirovich
• Chairman of the Primorsky Krai Legislative Assembly: Rolik Alexander Ivanovich
• Capital of the Region, thousand persons: Vladivostok - 606.6 (on the 1st of Jan. 2017)
• The Distance to Moscow, km: 9,302
• Urban Population, %: 77.21 (2017)
• Population, thousand persons: 1,913 (on the 1st of Jan. 2018)
• Population Density, persons per sq. km: 11.62 (on the 1st of Jan. 2018)
• Area, thousand sq. km: 165 (on the 1st of Jan. 2018)
• First Person Official Web-Site
• Statistics Agency Official Web-Site
• Old and Failing Housing Stock, %: 2.00 (2016)
• Number of Employed per One Retired Person: 1.82 (2014)
• Per Capita Monthly Income, Rubles: 33,259.50 (2017)
• GRP per Capita, rubles: 382,587 (2016)
• GRP, billion rubles: 737 (2016)
• Gini Index: 0.39 (2017)
• Key Indicators

Agriculture

Demographics, electricity, environment, fixed assets, foreign trade, gross regional product, household income and consumption, industrial production, investment and construction, living conditions, population projections, services to households.

• Natural population increase
• Population density
• Urban population share
• Crude birth rate
• Crude death rate
• Fertility rate
• Life expectancy at birth
• Net migration
• Population forecast
• Young population forecast
• Employable age population forecast
• Retirement age population forecast
• Life expectancy at birth forecast
• Net migration forecast
• Labor force
• Employed population
• Unemployed population
• Labor force participation rate
• Unemployment rate
• Employment rate
• Personal income per capita
• Personal consumption expenditure per capita
• Poverty rate
• Life expectancy
• Female life expectancy at birth
• Male life expectancy
• Infant mortality rate
• Number of beds
• Crimes registered
• Offenders identified
• Particularly serious crimes
• Serious crimes
• Minor offenses
• Economic crimes
• Terrorist offenses
• Extremist crimes
• Drug trafficking
• Arms trafficking
• Murder and attempted murder
• Acceptance of bribe
• Giving a bribe
• Children in preschool education
• Students in secondary professional education
• Students in tertiary education
• Specialists completed tertiary education
• Postgraduate students
• Doctoral students
• Volume of total services
• Volume index of total services
• Value of consumer services
• Value of housing services
• Value of transport services
• Value of medical services
• GRP per capita
• GRP per capita growth
• Retail trade volume growth
• Industrial production index
• Industrial production index YoY
• Stock of fixed assets
• New fixed capital stock created
• Depreciation of fixed assets
• Deterioration rate of fixed assets
• Minimum grocery basket inflation
• Food goods inflation
• Non-food Inflation
• Services Inflation
• Fixed capital investment value
• Fixed capital investment growth rate
• New residential dwellings
• Electricity production
• Electricity consumption
• Electricity exports abroad
• Monthly electricity production
• Arable land
• Arable land share
• Forest area share
• Emissions of air pollutants
• Volume of emissions of air pollutants
• Neutralized pollutants
• Personal cars
• Number of cell phones
• Number of personal computers
• Size of dwellings
• Average product consumption per day
• Dwellings equipped with piped water
• Agricultural production
• Livestock production
• Crop production
• Livestock and poultry weight gain
• Grains and legumes
• Standing wood burned
• Terms of Use
• Privacy Policy

Our Privacy Statement & Cookie Policy

Okay to continue Our website uses cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your personal cookie settings through your internet browser settings.