Survey Attrition and Attrition Bias in Young Lives
- Date: 01 Mar 2008
- Series: Technical Note 5
- Author: Ingo Outes-Leon and Stefan Dercon
- Download the file ( English, 1740 KB, PDF document )
Longitudinal studies, such as the Young Lives study of childhood poverty, help us to analyse welfare dynamics in ways that are not possible using time-series or cross-sectional samples. However, analysis based on panel datasets can be heavily compromised by sample attrition. On the one hand, the number of respondents who do not participate in each round of data collection (wave non-response) will inevitably cumulate over time, resulting in falling sample sizes, which will undermine the precision of any research undertaken using such samples. On the other hand, unless it is random, attrition might lead to biased inferences. Analysts often presuppose that attrition is correlated with observable characteristics such as household education, health or economic well-being, resulting in samples that include only a selected group of households. However, even if that is the case, non-random attrition does not necessarily lead to attrition bias. Attrition bias is model-specific and, as previous studies have shown, biases might be absent even if attrition rates are high.
We investigate the incidence and potential bias arising from attrition in Young Lives following the completion of the second round of data collection. Young Lives is a study concerned with analysing childhood poverty in four countries, Ethiopia, India, Vietnam and Peru. The study, which measures a range of child, household, and household-member characteristics, is following two cohorts of children in each country over 15 years – a younger cohort of 2,000 children who were born in 2001 to 2002 (i.e. aged 6 to 18 months when first surveyed) and 1,000 older children born in 1994-95 (i.e. aged 7.5 to 8.5 at the start of the survey). Sample attrition is particularly concerning in the context of a longitudinal study such as Young Lives where cohort sample sizes are modest and individuals are tracked over a relatively long period of time. This paper seeks to:
- document the rates of attrition of the Young Lives study following completion of the second round of data collection;
- investigate the extent to which sample attrition is non-random;
- analyse whether non-random attrition in the Young Lives sample might lead to attrition bias.
Alhough they range widely across countries and attrition category, we find that Young Lives attrition rates are small in absolute terms. Furthermore, attrition rates are modest when compared with other longitudinal studies in developing countries. In fact, in all study countries Young Lives has attrition rates lower than any of the comparison studies.
Second, our analysis indicates that attrition is to some extent non-random. In particular, we find that attrition is primarily an urban phenomenon and that attriting households tend to be poorer and less educated than non-attriting households.
Third, in spite of non-random attrition we find very limited evidence of attrition bias when tested on child anthropometric and school enrolment models. On the one hand, very low RSquares in the attrition probit models indicate that attrition remains overwhelmingly a random phenomenon. On the other hand, attrition probit and BGLW tests suggest that attrition on observables is unlikely to lead to significant biases. Further, limited evidence of attrition bias found in Ethiopia using attrition probit tests are not corroborated by the BGLW tests; indicating that, although uncovered patterns of non-random attrition could lead to biases, modest rates of attrition ensure that attrition bias remains very weak.
In summary, our detailed analysis of the attrition bias of the Young Lives sample strongly indicates that current attrition is highly unlikely to bias research inferences. However, some weak evidence of bias alerts us to the importance of ensuring that we continue to track the children between survey rounds to maintain our current low rates of attrition and not exacerbate the uncovered non-random patterns of attrition we have noticed to date.