Dr. Luigi Butera speaking at SIFE 2015.
What is a field experiment?
A field experiment, like a medical trial, applies the scientific method to experimentally examine the effect of an intervention in the real world. Economic experiments randomize subjects into treatment and control groups and compare outcomes between these groups. Randomization is what guarantees that if we observe a difference between a treatment and a control group, that difference must be caused by the treatment. Randomization also allows you to control for several potential explanatory factors such as age, gender, and income level. This enables you not only to understand what works, but also why. Field experiments allow you to discover the most efficient way to allocate your resources.
What can our organization expect from using field experiments? Four Key Benefits:
Business insights from science: field experiments - like medical trials - are the "gold standard" for learning about the world. Harness the power of the scientific method to improve and revolutionize your business.
Bespoke support, for free: the Institute's economists are not consultants: they won't charge you for their time. Working with you allows them to generate original research and publish it in top academic journals.
Minimal distraction: field experiments are set up to take place organically within your business activity. Our scholars will take care of the field experiment from A to Z, with minimal/no interference with your regular business activities.
Mastery with experimentation: Every large successful company, from Google to Walmart to IEA, routinely uses randomized experiments and behavioral economics to improve their business. At the Institute you will learn how to easily incorporate these methods into your core management practice.
How much does it cost to participate in SIFE?
The Summer Institute on Field Experiments is free of charge for both scholars and firms. Scholars will receive a travel allowance.
We already use surveys and focus groups to inform our decisions. Why would we want to run an experiment?
Surveys and focus groups are an excellent way to learn about people’s intentions. However, what people say they would do is often very different from what they actually do. This is because people are affected by the context in which they make a decision, by the amount of attention they put in that decision, and by their emotions. Surveys may not capture these factors because people often do not know ex-ante what they will pay attention to or they do not consciously realize that context clues or emotions affect their decisions. Experiments allow us to observe how different interventions affect behavior in natural environments.
Can't we use econometric and statistical analysis to analyze our existing data to understand what works and what doesn't?
Analyzing existing data is an important component to research, but falls short on establishing a causal relationship. Consider the following example from a development office: an organization chooses to add a small gift to their solicitation and after one year observes that average donations have increased. This is good news, but why is this happening? The improvement may surely be due to the gift itself, but without proper randomized tests one cannot establish causation. It is possible that a third, unobserved factor is driving this effect (e.g. better general economic conditions). If this were the case, the increase would have happened also without the gift, thus spending resources on gifts was inefficient. It is also possible that donors’ responses to gifts vary, but the lack of proper randomization hides this effect. For instance new donors may be attracted by the gift, but existing donors may consider it a waste and thus reduce their giving. The average effect is positive, but the organization is losing donors, which may backfire in the long run.
Will the name of our organization be on the academic paper?
It is your choice whether the name of your organization will or will not appear in an academic paper.
Can I be a coauthor on the academic paper?
We are happy to include you as a coauthor, if you wish to.
Will you acquire personal identifiable information?
Like medical experiments, economic experiments need to be approved by an Institutional Review Board (or Ethical Review Board). The IRB process is aimed at guaranteeing the safety of participants. In the case of economic experiments, safety typically means privacy. To receive IRB approval, two general conditions need to hold: 1) the experiment does not involve deception of any kind; 2) no personally identifiable information is retained. We abide by what academics call data de-identification: while we retain individual-level data that is useful for research purposes (e.g. age, gender, income, level of education etc.), we try to design our experiments so that no personal identifiable information is collected (e.g. names, addresses, phone numbers, social security numbers). If the nature of the experiment requires acquiring some sensitive information, we will ensure that the data is de-identified before any dataset is built.
If we run an experiment, will the scholars own the data?
Absolutely not. We routinely prepare with our partners a Data Usage Agreement— a contract that specifies that the organization remains the sole owner of all data generated. The partner University, however, will own the results and analysis performed on the data generated by the experiment. This is to guarantee the academic freedom of publishing novel results in scientific journals.
What is the timeline of an experiment?
Economic experiments are set up to take place organically within your business activity. How long the experiment is run and when it is implemented will be decided in its planning, with all partners involved.
Will experiments "poison" my pool of customers?
Firms and organizations are rightly concerned that some "bad" treatments may drive some customers away. While possible, experiments are set up precisely to avoid applying a "bad" intervention to all customers. Only a subset of your customers will be exposed to new treatments and once we find the most effective way to solve your problem, you will be able to use the solution with your entire pool. As the proverb goes, it is better to be safe than sorry.
Further questions? Please email firstname.lastname@example.org.
SIFE 2016 Application Forms
Connect with Us!
New to Experiments?
- Chicago Experiments
- Training Young Scholars to Answer Big Questions with Experiments
- On The Generalizability Of Experimental Results In Economics: With A Response To Camerer
- Do Natural Field Experiments Afford Researchers More Or Less Control Than Laboratory Experiments? A Simple Model
- PHOTO GALLERY: SIFE 2015
- SLIDES: Pulling Off a Field Experiment as a Young Economist
- SLIDES: Types of Economic Data
- SLIDES: The Power of Replication
- SLIDES: Field Experiments
- SLIDES: Statistical Tests Using Data
The Summer Institute is funded by a generous grant from the Laura and John Arnold Foundation. Thank you to our partner, the University of Chicago.