Collaborations for Impact
Our approach relies on collaboration. Recognizing that we cannot do this alone is what drives our desire to build coalitions, share knowledge, and achieve victories across institutions, systems and communities.
We are working to develop a network of researcher-system partnership sites (i.e., school districts, early childcare facilities, hospitals and community based social service programs) to tackle the most pressing issues facing our society today. These Collaborations for Impact (CFI) sites will serve as laboratories of innovation that coordinate research focused on testing, replicating and scaling promising interventions. Over time, CFI will advance our understanding of the science of scaling.
Collaborations for Impact – Education (CFI-Ed)
Based on need and experience, the first iteration of our approach is CFI-Ed. We will begin by developing the network of laboratories of innovation with a focus on education. To identify the inaugural partner sites for 2018-19, we are announcing a call for partners (information about the call).
How to Apply/Process
Submit a brief letter of intent to apply by December 15, 2017, with full application due January 15, 2018. Complete application packages can be submitted by email to firstname.lastname@example.org (download application form here).
Contacts: Questions about this call for partners should be directed to: Min Sok Lee (email@example.com)
The Science of Scaling
The hallmark of public policy decision-making is benefit-cost analysis. To do benefit-cost analysis effectively, it takes proper measurement of both benefits and costs. Recently there has been a surge in field experiments or randomized control trials (RCTs) to measure the benefit side of interventions. While many have been found to have substantial benefits, whether and to what extent these benefits manifest themselves in large-scale settings is an open question.
With an eye towards scaling up to the greater population, CFI sites will provide the necessary insights into overcoming existing barriers to scalability. As a network of researcher-system partnership sites, individual interventions will be evaluated through replication – as opposed to the standard practice of conducting purely novel RCTs.
This work not only focuses on the “voltage” problem, but also on the broader implications of scaling interventions on the benefit-cost ratio. This approach admits economics as a powerful tool to understand the scaling problem at the macro level. In this way, we are leveraging the economic science to provide unique insights into the scaling problem.
Threats to Scalability
The work of Al-Ubaydli, List, and Suskind describes threats to scalability as being composed of two major types. On the one hand, there’s the voltage effect, which occurs when the treatment effect size is considerably smaller at scale than what researchers find in the initial investigation. The ASL work includes a model that provides insights into why this voltage effect might occur. It includes false positives, representation of population, and representativeness of the situation.
But what the theory also shows is that even if there’s no voltage effect, there’s still an important potential scaling problem in that the benefit-cost ratio might not be the same at scale as what the initial investigation showed. For example, just considering the cost side, it might be more difficult to hire 10,000 excellent tutors than it is to hire 100 excellent tutors (on both the quality and cost side). Accordingly, even if the quality of tutors is identical over the 10,000, the benefit-cost ratio might get smaller if you have to pay more for the 10,000th tutor than you paid for the first 100 tutors.