The Challenges of Evaluating Non-Profit Organizations and their effectiveness
By Crown Family School
Faculty members Nicole Marwell and Jennifer Mosley discuss their book Mismeasuring Impact
Crown Family School of Social Work, Policy, and Practice faculty members Nicole P. Marwell, PhD and Jennifer E. Mosley, PhD discussed their newly published book Mismeasuring Impact, during a panel discussion on creating equitable metrics in the philanthropy and nonprofit sector co-hosted by the City Club of Chicago.
Moderated by Daniel Ash, president of the Field Foundation and joined by Liz Dozier, founder and CEO of Chicago Beyond as a panelist, Marwell and Mosley explored the methods used by nonprofits and foundations to determine whether the programs and investments are truly making a difference.
Marwell and Mosley articulated that randomized controlled trials (RCTs) often fail to capture the complexity of nonprofit work and can constrain innovation and equity. They examined the limitations of RCTs in evaluating nonprofit programs. Dozier shared her experiences as a former high school principal and now a nonprofit leader, highlighting the power dynamics and ethical issues associated with RCTs.
“We wrote this book to give everyone the language to be able to say why they probably shouldn’t do an RCT,” said Marwell. There are huge implications for organizations and the services it provides, Mosley said.
The research partners said for the book they spoke to foundation executives and philanthropy professionals and heard a consistent message, that randomized control is actually a poor fit to assessing programs inside nonprofit organizations.
“Nevertheless, organization managers feel like this is the legitimacy thing that ‘we need, we need something to show that this is the best way to do that,’” Mosley said. “But in the meantime, there were huge implications for their organizations around having to change the kinds of outcomes that they were looking at and not making any changes to their programs for years.”
Throughout the discussion Marwell and Mosley emphasized the need for agile, iterative evaluation methods that support responsiveness and innovation, rather than the rigidity of RCTs. Ultimately, the panelists agreed alternative evaluation methods that prioritize community voices and practical insights better serve communities.
“(Managers) felt really obligated to push their staff to the limits and change a lot of things about their organization, even if it hurt their mission to try to get that stamp of approval. And we just think that is not a sustainable model,” Mosley said.