Among many in the social sector, it’s an article of faith that randomized controlled trials (RCTs) are the gold standard for evaluating the effectiveness of social programs. There are good reasons for this belief. Only very specific kinds of evidence can demonstrate that participating in a particular social program causes whatever outcome it is designed to produce, whether it be rising income, increasing recovery from substance-use disorder, improved high school graduation rates, or something else. A well-implemented RCT can provide that evidence, and its proponents argue that it is the best way to do so.
However, is measuring specific changes—in particular, individual outcomes—the best way to tell if a nonprofit’s work is valuable? What are the consequences when this way of conceptualizing a nonprofit’s contributions is given such prominence? As we learned in researching and writing Mismeasuring Impact: How Randomized Controlled Trials Threaten the Nonprofit Sector, elevating the RCT as the singular arbiter of a nonprofit’s impact can actually have a variety of negative effects on nonprofit operations and performance. It can direct the attention of management and staff away from community-based connections, privilege larger and more traditional organizations over smaller or more innovative ones, and reduce nonprofits’ ability to respond nimbly to changing social problems. It can even mean sacrificing the most valuable evaluation-oriented mindset that a nonprofit can embrace: prioritizing continuous learning, adapting to shifting environments, and drawing on the innovative power of community collaboration.