- ホーム
- > 洋書
- > 英文書
- > Business / Economics
Full Description
The hidden dangers of randomized controlled trials
The need to demonstrate the effectiveness of nonprofit social programs has led to a rapid rise in the use of randomized controlled trials (RCTs), for evaluation. As a result, most nonprofit sector professionals can tell you why nonprofits should do an RCT. This book tells you why they probably shouldn't, and what to do instead.
Mismeasuring Impact explores why RCTs are being embraced as the "gold standard" for nonprofit evaluation, despite the high cost and time investment required and the serious problems with using RCTs in a nonprofit context. (Most RCTs conducted in nonprofits fail to meet required standards for rigor, undercutting their accuracy). The book describes what happens inside nonprofits when they take part in RCTs, the unintended equity issues that arise, and why nonprofits feel pressured participate in RCTs despite the problems.
University of Chicago professors Marwell and Mosley's research is based on extensive interviews with key players: nonprofit managers, professional program evaluators, and program officers in philanthropic foundations. The book argues that, ultimately, RCTs are used to poorly ground nonprofit legitimacy, not to foster nonprofit improvement. RCTs also privilege program and organizational standardization over the key strengths of nonprofit organizations: flexible innovation and responsiveness to community needs.
Nonprofits and funders need forms of evaluation that lift up these strengths. Mismeasuring Impact offers alternative approaches that build strong organizations, not just cookie-cutter programs, and which funders and nonprofits of all sizes can support.
Contents
Acknowledgments
1. FIVE PROBLEMS WITH RCTs, AND WHY THEY THREATEN THE NONPROFIT SECTOR
The Case for RCTs
Are RCTs Really the "Gold Standard" for Nonprofits?
Five Problems with Using RCTs in Nonprofit Organizations
Goals of This Book
Brief Answers to Some Common Objections to Our Argument
2. THE SIREN SONG OF RCTs: BUILDING THE GOLD STANDARD MOVEMENT
How Did We Get Here? Understanding the Growth of RCTs for Social Programs
The Evidence Battle
The Funding Battle
Spreading RCTs to U.S. Nonprofits: The Social Innovation Fund
Changing the Conversation
3. NULL EFFECTS AND DUMB EFFECTS: DO RCTs REALLY TELL US "WHAT WORKS?"
The Ubiquity of Null Effects
Dumb Effects: When Positive Effects Are Not Very Meaningful
Moving Beyond RCTs
4. "THIS WAS HARDER THAN WE THOUGHT": IMPLEMENTING RCTs IN NONPROFITS
Twice the People, Twice the Work: RCT Recruiting Challenges Staff Capacity and Organizational Mission
"It Sounded a Lot Easier Than It Actually Was"
"It Sounded a Lot Easier Than It Actually Was"
Delays and (Financial) Drains
RCTs Don't Just Evaluate, They Can Change Both Programs and Organizations
Putting Organizational Well-Being First
5. LOOKING FOR LEGITIMACY: WHY WE KEEP DOING RCTs
RCTs Can Build Organizational Legitimacy
Does an RCT Always Enhance Nonprofit Legitimacy?
Evaluators See Limits on RCTs
The View from Philanthropy
RCTs May Increase Legitimacy, but Not Without Consequences
6. WHAT KIND OF WORK SHOULD WE VALUE? HOW RCTs UNDERMINE THE EQUITY GOALS OF NONPROFITS AND THE COMMUNITIES THEY SERVE
Is Pursuing Effectiveness the Same as Pursuing Equity?
Do RCTs Privilege Evidence-Building Over People?
The Rich Get Richer
Practicing Equity: A Key Goal for Philanthropy
Evaluation, Effectiveness, and Equity
7. A BAD FIT :HOW RCTs HINDER NONPROFIT RESPONSIVENESS AND INNOVATION
Long Time Horizon
The "Hold Still" Challenge: RCTs Limit Ongoing Program Responsiveness and Innovation
"You Get One Shot"
A Black Box
The Need for Alternatives to the RCT . . . and More Evaluation Capacity Overall
Evaluation Should Better Support Responsiveness and Innovation
8. HOW MIGHT WE THINK ABOUT EVALUATION DIFFERENTLY? THREE PRINCIPLES FOR NONPROFIT IMPROVEMENT
Principle 1: Tailor Evaluation to Specific Organizational Strategies and Community Needs
Principle 2: Centering Participant Perspectives Is Key to Nonprofit Performance
Principle 3: Focus on Improvement Through Iteratively Addressing Problems Instead of Standardizing Solutions
An Improvement Orientation
9. MOVING BEYOND RCTs: LESSONS FOR NONPROFIT LEADERS, DONORS, EVALUATORS, AND POLICYMAKERS
Evidence Matters
Five Problems with RCTs
When Is the Right Time to Do an RCT?
Three Principles for Using Evaluation to Foster Nonprofit Improvement
Recommendations for Building Evidence Outside the Evidence Hierarchy
The Way Forward
Appendix: Research Methodology
Notes
Bibliography
Index
