Introduction to Critical Appraisal

Critical appraisal goes beyond spotting red flags (L3_3_03). It is a structured, reproducible assessment of whether a study's design, conduct, and analysis minimize bias and support valid conclusions. Three core questions of critical appraisal: 1. Is the study valid? (Internal validity — are the results true for the study participants?) 2. What are the results? (Magnitude and precision of the effect) 3. Are the results applicable to my patients? (External validity — can I generalize to my setting?) Widely used critical appraisal tools: Cochrane Risk of Bias tool (RoB 2): The standard for randomized controlled trials. Assesses 5 domains: randomization process, deviations from intended interventions, missing outcome data, measurement of outcomes, and selection of reported results. Each domain is rated Low Risk, Some Concerns, or High Risk. CASP Checklists (Critical Appraisal Skills Programme): Available for RCTs, systematic reviews, cohort studies, case-control studies, diagnostic studies, qualitative studies, and more. Each checklist has 10-12 yes/no/unclear questions. Free and widely used in teaching. Newcastle-Ottawa Scale (NOS): For observational studies (cohort and case-control). Rates studies on selection of participants, comparability of groups, and assessment of outcomes/exposures using a star system (max 9 stars). JBI Critical Appraisal Tools: From the Joanna Briggs Institute. Cover 13 study designs including prevalence studies, case reports, text/opinion papers, and economic evaluations — broader than CASP. How to use a checklist: Read the relevant tool's guidance document first. Go through the paper's Methods and Results sections with the checklist open. Answer each item based on what is reported, not on assumptions. Rate each domain. Summarize your overall quality assessment. Important: No study is perfect. Critical appraisal helps you understand how imperfect a study is and whether its flaws are likely to have changed the conclusions.