Red Flags in Published Research

Red flags to watch for: Methodological red flags: - Very small sample size without power calculation: A study claiming to find no difference between treatments with only 20 patients per group may simply be underpowered, not evidence of equivalence. - No control group: Studies that report improvement without a comparator cannot distinguish treatment effect from natural disease course, placebo effect, or regression to the mean. - Inappropriate study design for the question: Using a case series to claim treatment efficacy, or a cross-sectional study to claim causation. - Selective outcome reporting: The paper's registered protocol (on ClinicalTrials.gov) listed different primary outcomes than the published paper reports. This suggests the authors changed outcomes after seeing the data. Statistical red flags: - P-hacking: Running many statistical tests and reporting only the significant ones. Suspect when a paper reports many subgroup analyses but only highlights the ones with p<0.05. - Emphasis on statistical significance without clinical significance: A drug that lowers blood pressure by 1 mmHg may be statistically significant with a huge sample but clinically meaningless. - Missing confidence intervals: P-values alone do not tell you the magnitude or precision of an effect. If a paper reports only p-values without effect sizes and CIs, it may be hiding imprecise estimates. Presentation red flags (spin): - Conclusions that do not match results: The abstract concludes 'Treatment X is effective' but the primary outcome was not statistically significant. A common form of spin. - Relative risk without absolute risk: Reporting 'Drug A reduces mortality by 50%' sounds dramatic. But if mortality dropped from 2% to 1%, the absolute risk reduction is only 1%, and NNT is 100. - Predatory journal: Published in a journal with no recognized impact factor, suspicious editorial board, or known to accept papers without meaningful peer review. Conflict of interest red flags: - Study funded by the manufacturer of the intervention being tested, with no independent data monitoring - All authors are employees or paid consultants of the sponsor - Ghost authorship suspected (a medical writer is acknowledged but not listed as author)