In the United States, most social-program RCTs are carried out inside nonprofit organizations, where most service delivery happens. To support effective programs—as well as the people and communities those programs serve—we also need to commit to cultivating strong, multifaceted nonprofit organizations. To understand how the technical aspirations of RCTs intersect with the complex social demands of running nonprofit organizations, we designed a study asking professionals in different positions within the nonprofit ecosystem about the growing use of RCTs for nonprofit evaluation: nonprofit managers with direct experience in RCT implementation and those who have declined RCT participation, program officers from philanthropic foundations responsible for supporting different kinds of nonprofit evaluation (including RCTs), and evaluators who design and oversee RCTs in nonprofit settings. The following are five reasons why nonprofit leaders should exercise care when considering whether an RCT serves their organization’s and participants’ needs. But more modest effects—the kind most social programs tend to produce—require much larger sample sizes to detect with confidence.