One of the remarkable things about many field experiments done by development economists is the low attrition rates that have been achieved in many surveys. In an era of increasing suspicion about scams, and distrust in strangers, the fact that people are willing to repeatedly answer often lengthy questionnaires is amazing. It can take a lot of effort, but it is not uncommon to have 80-90% of respondents answer (Ghanem et al, report at average attrition rate of 15%). However, this is not always the case, especially when dealing with richer populations, large firms, and very mobile populations, and sometimes there is an unrealistic expectation of what response rates should be. Many field experiments in developed countries have the luxury of using admin data for key outcomes, but for those using surveys, what do we see? I thought I’d note response rates in a few recent developed country experiments for comparison.
· Flynn et al. 2025 carry out an RCT of a guaranteed income pilot in Oakland, California, with 660 families, with the treatment group getting $500/month for 18 months. They did 4 rounds of six-monthly surveys, with attrition rates of 26-32% in the treatment group and 50-54% in the control group (with differential attrition rates of 21-24%). It is unclear whether surveys were done in-person, by phone, or online. Respondents were compensated, but it is unclear how much.
· Bloom et al. 2025 run a 5-year, 10-wave panel online survey of firms, getting them to make quarterly sales forecasts. In waves 5-10 there were different experiments designed to overcome forecast biases by varying accuracy incentives, providing forecasting training, and getting them to evaluate different scenarios. Respondents were paid $50 for the first survey, and then $25 per short survey follow-up, in addition to being offered accuracy rewards. The initial sample was over 18,000 firms, with booster samples adding another 35,000 firms. Average response rates were 15%, that is, attrition rates averaged 85%. It is unclear how attrition rates vary with treatment status, nor what they were for the sample that were still in the study at the time of randomization.
· Gosnell et al. (2020) run an experiment with 335 Virgin Atlantic airline pilots, using admin data for the main outcomes, but an online survey that had 40% attrition for measuring pilot welfare and attitudes. It is unclear how attrition varied with treatment status, nor whether there were incentives to respond.
· But high attrition need not always be the case. Vivalt et al. 2025 have a series of papers on a guaranteed income experiment in Illinois and Texas with 3,000 low-income individuals. A treated group of 1,000 got $1,000 per month for 3 years, while the control group of 2,000 got $50/month for 3 years. They have a 18-month midline, and 30-month endline, done by a combination of in-person and phone, with remarkably high response rates of 97% for the midline and 96% for the endline. People were paid $50-100 for completing each survey, and $15-30 for shorter online surveys that also resulted in 89-95% response rates. As well as high incentives and a lot of effort, these response rates likely also reflect the way people were selected into the study in the first place – they had to respond to a mailer, complete an intake survey for which they received compensation immediately (building trust), and then all got $50/month for multiple months before being assigned to treatment or control.
· What about outside of the U.S.?
o Crepón et al. 2013 conducted an RCT in France of job placement assistance for the unemployed. Follow-up surveys were very short (10 minutes at 8 months, 5 minutes at 12, 16 and 20 months), basically just finding out whether people were employed or not. Surveys used internet, phone, and paper, and were an official survey of the Ministry of Labor, which they note tends to result in high response. The response rate was 79% at 8 months, and 71% for the sample who weren’t employed at the time of randomization. Differential attrition is less than 2%. However, even with this very short survey and official agency collecting data, attrition appears to be up to 41% by 20 months for this group of unemployed.
o Another issue in Europe is that data privacy laws can further complicate efforts. For example, van den Berg et al. 2025 conduct an RCT in Germany with unemployed workers. They attempted a survey and note that the nonresponse “was sizeable. For data protection reasons, the survey data can only be linked to the administrative data if the respondent agrees. In particular, most of the caseworkers did not allow merging of their own responses to records of their clients. For these reasons, the survey data are of limited use.” – They also report substantial item non-response, and that attrition was unbalanced by treatment status – and so they use admin data.
I thought these comparisons would be of use/interest for at least several groups of development researchers: (1) those working with populations in developing countries that are likely to be more like those in these developed country experiments- e.g. richer urban households or workers, larger firm owners, people living in places where the norm is not to answer a call from an unknown number, etc – these numbers may then be benchmarks for what you might expect; (2) those facing what they see as double standards about how attrition is viewed in field experiments in developed versus developing countries, who may want examples to refer skeptical referees or editors to; and (3) also as examples of ways to reduce the potential for high attrition through screening, survey incentives, and choice for who is implementing the survey.
Please share other recent examples of both high and low response rates in surveys from RCTs in developed countries in the comments.
Source : World Bank
