How good are we? A meta-analytic study of effect sizes in medicine

Persistent Link:
http://hdl.handle.net/10150/280542
Title:
How good are we? A meta-analytic study of effect sizes in medicine
Author:
Caspi, Opher
Issue Date:
2004
Publisher:
The University of Arizona.
Rights:
Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
Abstract:
Background. Although Cohen's convention regarding small (∼0.2), medium (∼0.5), and large (∼0.8) effect sizes (ES) that originated in the socio-behavioral sciences approximately 40 years ago has been used heuristically in medicine as well, the extent to which it characterizes the range of ES for various medical therapies remains unknown. Objectives. (1) To calibrate the robustness of current interventions in medicine using different ES indices as the underlying metric, and (2) to examine whether the efficacy of therapeutic interventions in medicine have changed over the years. Methods. Three complementary studies have been conducted. Study I, which provides an indication of the range of ES in medicine currently, is a meta-meta analysis that summarizes 91 different meta-analyses of various treatments, from medicine to surgery to psychology. Study II used a pool of 250 different ES computed from 52 original clinical trials published over the last 25 years to examine time trends associated with ES. Study III used traditional and cumulative meta-analyses of 76 trials representing 30 different treatments for sepsis. Number-needed-to-treat (NNT) was calculated and adjusted for different control death rates to assess systematically objectives (1) and (2). Results. Study I: The mean ES for continuous variables was d 0.50 (SD 0.35, 95%CI 0.32-0.69) and for dichotomous variables odds ratio 1.51 (SD 1.39, 95%CI 0.9-2.13; p > 0.05). Study II: A parsimonious theory-specified model accounted for 93.2% of the variance associated with ES estimates over the years. ES was statistically significant correlated with many design features but not with year of publication. Study III: Neither traditional meta-analysis nor cumulative meta-analysis found most treatment categories for sepsis to be effective. Whereas the pooled relative risk estimate for the entire cohort of almost 22,000 patients was statistically significant with very narrow confidence interval (RR 0.92; 95%CI 0.88-0.96; p = 0.00028), the NNT adjusted analysis had a wide range, including the potential for harm. Conclusion. Cohen's convention applies well to medicine. Most therapeutic interventions in this study had a small to moderate ES, indicating they have not changed dramatically over the years. These findings suggest that the ever-increasing biomedical knowledge does not result in ever-improving therapeutic efficacy.
Type:
text; Dissertation-Reproduction (electronic)
Keywords:
Psychology, General.
Degree Name:
Ph.D.
Degree Level:
doctoral
Degree Program:
Graduate College; Psychology
Degree Grantor:
University of Arizona
Advisor:
Sechrest, Lee

Full metadata record

DC FieldValue Language
dc.language.isoen_USen_US
dc.titleHow good are we? A meta-analytic study of effect sizes in medicineen_US
dc.creatorCaspi, Opheren_US
dc.contributor.authorCaspi, Opheren_US
dc.date.issued2004en_US
dc.publisherThe University of Arizona.en_US
dc.rightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.en_US
dc.description.abstractBackground. Although Cohen's convention regarding small (∼0.2), medium (∼0.5), and large (∼0.8) effect sizes (ES) that originated in the socio-behavioral sciences approximately 40 years ago has been used heuristically in medicine as well, the extent to which it characterizes the range of ES for various medical therapies remains unknown. Objectives. (1) To calibrate the robustness of current interventions in medicine using different ES indices as the underlying metric, and (2) to examine whether the efficacy of therapeutic interventions in medicine have changed over the years. Methods. Three complementary studies have been conducted. Study I, which provides an indication of the range of ES in medicine currently, is a meta-meta analysis that summarizes 91 different meta-analyses of various treatments, from medicine to surgery to psychology. Study II used a pool of 250 different ES computed from 52 original clinical trials published over the last 25 years to examine time trends associated with ES. Study III used traditional and cumulative meta-analyses of 76 trials representing 30 different treatments for sepsis. Number-needed-to-treat (NNT) was calculated and adjusted for different control death rates to assess systematically objectives (1) and (2). Results. Study I: The mean ES for continuous variables was d 0.50 (SD 0.35, 95%CI 0.32-0.69) and for dichotomous variables odds ratio 1.51 (SD 1.39, 95%CI 0.9-2.13; p > 0.05). Study II: A parsimonious theory-specified model accounted for 93.2% of the variance associated with ES estimates over the years. ES was statistically significant correlated with many design features but not with year of publication. Study III: Neither traditional meta-analysis nor cumulative meta-analysis found most treatment categories for sepsis to be effective. Whereas the pooled relative risk estimate for the entire cohort of almost 22,000 patients was statistically significant with very narrow confidence interval (RR 0.92; 95%CI 0.88-0.96; p = 0.00028), the NNT adjusted analysis had a wide range, including the potential for harm. Conclusion. Cohen's convention applies well to medicine. Most therapeutic interventions in this study had a small to moderate ES, indicating they have not changed dramatically over the years. These findings suggest that the ever-increasing biomedical knowledge does not result in ever-improving therapeutic efficacy.en_US
dc.typetexten_US
dc.typeDissertation-Reproduction (electronic)en_US
dc.subjectPsychology, General.en_US
thesis.degree.namePh.D.en_US
thesis.degree.leveldoctoralen_US
thesis.degree.disciplineGraduate Collegeen_US
thesis.degree.disciplinePsychologyen_US
thesis.degree.grantorUniversity of Arizonaen_US
dc.contributor.advisorSechrest, Leeen_US
dc.identifier.proquest3132204en_US
dc.identifier.bibrecord.b46709186en_US
All Items in UA Campus Repository are protected by copyright, with all rights reserved, unless otherwise indicated.