Ethiopia: sampling methods

The Young Lives Sampling Procedure in Ethiopia

Download this document as a 184 KB PDF file
or read below as a web page.

Contents
Background
Sampling procedure
Stage 1: Selection of regions
Stage 2: Selection of Weredas
Stage 3: Selection of Peasant Associations (in rural areas) or Kebeles (in urban areas)
Stage 4: Definition of a sentinel site
Stage 5: Selection of households
The Raven's test for 8 year-olds

A summary of the approach to sampling in Ethiopia can be found in the country preliminary report. Any references can also be found in the bibliography at the end of the preliminary report.

Background

Given the objectives of the Young Lives Project – which is an international study of childhood poverty – the overall sampling approaches, principles and methods agreed amongst those involved in the project were as follows:

1. In each country, a cohort of 2,000 children aged 6-18 months to be selected for follow-up, and 1,000 children aged 7.5 to 8.5 years to be selected in the first round.
2. A range of children to be sampled from the sentinel sites, but the poor to be over-sampled.
3. Sentinel sites to be determined based on factors relevant for the specific country.
4. There to be 20 sentinel sites in each country, and from each sentinel site 100 children aged 6-18 months and 50 children aged 7.5 to 8 years to be selected.

Sampling procedure

The sampling framework adopted in the Young Lives Project is known as the Sentinel Site Surveillance system. A Sentinel Site Surveillance system is a system whereby, “Sites are selected intelligently and mainly based on qualitatively determined variables”. In other words, sentinel sites are selected by considering multi-dimensional policy variables and other factors relevant to the project.

The selection of sentinel sites and households within sites eligible for enumeration for the fieldwork involved a number of stages. Such selections at various levels were undertaken on the basis of predetermined factors consistent with the objectives of the project, along with consultations with relevant officials, policy-makers and experts at all levels. The stages include selection of specific regions in the country, selection of Weredas (districts) in each of the selected regions, selection of Peasant Associations (in rural areas) or Kebeles (in urban areas) within Weredas , and finally the selection of households eligible for the fieldwork. The various stages and procedures involved in the sampling process are discussed below.

Stage 1: Selection of regions

The first stage of the sampling framework involved the selection of regions in the country. One of the issues considered in the sampling process was that of coverage. It was initially felt that the sampling strategy needed to have an element of national coverage, in which the different sites were distributed over almost the whole country. This strategy had the following implications, amongst others things:

1. The cost of the fieldwork was likely to be very high.
2. The design might give more emphasis to the breadth than the depth of information that could be obtained by focusing on selected regions or areas of the country.
3. As a longitudinal study, the issue of tracking and follow-up are important elements of the sampling design and were likely to be problematic if sites were scattered and/or less accessible.

The main objective of the Young Lives Project is to collect longitudinal data in order to help us understand the development and welfare of children born into poverty. The factors affecting the development and welfare of children living in poverty (or the overall well-being of a child) are considered to be diverse and complex. Such factors may be environmental, social, cultural, economic or the interaction of any of these and others. Trying to capture such diverse and complex factors in a study of a limited scope such as this raised many questions. Nevertheless, it was important to relate these factors to the Ethiopian context.

Ethiopia is a vast country with over 70 ethnic groups comprising very diverse social, cultural, traditional and religious values, norms and practices in each society. The country is administered by a Federal system of Government and has a total of 11 Regional States and administrative regions including the capital city, Addis Ababa.

Ethiopia is predominantly agricultural. More than 85 per cent of its over 60 million people depend on subsistence agricultural production and related activities. Agricultural production is mainly rain-fed. As a result, critical food deficit and draught is a frequent phenomenon the country faces from time to time due to inadequate rainfall, in which circumstances young children and mothers suffer the most. The country also comprises a significant number of pastoralist populations in the lowlands, whose livelihood is dependent on livestock and related activities. In short, the wide range of differences in agro-ecological or agro-climatic factors plays a key role on the means of livelihood and people’s way of life.

Thus, the implication of these factors on the lives of children is likely to vary from one to another, particularly when the agro-ecological variations between regions or within a given region are large (for example, between highland crop farmers and lowland pastoralist farmers, or between those with cash crops and others).

Understandably, the problem of selecting specific regions under such complex situations and resource constraints is not an easy task. On the other hand, it is possible that a number of such factors could be captured by carefully selecting sites within a limited number of regions. In light of this, a number of qualitative and quantitative factors were used to determine the number of regions, within which specific sites would be selected in each region. A brief discussion on some of these factors follows.

(a) Focusing on limited areas of national significance

As stated earlier, focusing on a limited area of interest could be important in terms of cost, follow-up, tracking, field and data management and other issues. However, it is also important to bear in mind that those regions or areas selected for the study would have significant implications for future policy process and programme implementation in the country. Thus, informed judgment and consultation are important elements of the sampling process.

(b) Statistics on poverty settings in the country

‘Poverty’ is prevalent all over the country, though in different forms and degrees. It can be measured using a number of poverty indicators at various levels of the country. For example, a summary of poverty indicators in Ethiopia for the year 1999/2000 is shown in Table A.3. These indicators include variables such as stunting, wasting, literacy, primary school enrolment, etc, (see Table A.3 for details), as well as variables covering a range of issues related to distance, infrastructure and public services (see Table A.4). A close observation of these statistics reveals that the similarities and/or variations existing between regions are as substantial as those within regions. This implies that, while it is important to consider such data to assist in selecting certain regions along with other factors, it is difficult to simply employ strict criteria or categorize regions, say, as poor or non-poor (less poor) on the basis of such data only.

(c) Cost and accessibility of infrastructure

As noted earlier, in a longitudinal study such as this, one cannot over-emphasize the importance of the continuity and follow-up of the study, tracking, data management, cost and other resource requirements in designing the sampling strategy. Therefore, such criteria were also taken into consideration during the process of region selection.

Rural and urban poverty settings

The study should also needed to consider those factors affecting poor children, both in urban and rural areas. Such an approach would have far-reaching implications in terms of identifying the commonalties and differences in child welfare. Furthermore, policy impacts and implications are likely to be different between the two categories of poor children. However, as a predominantly rural-based society, the majority of the sentinel sites would obviously be poor rural sites.

In general, although the above were considered key factors in determining the sentinel sites for the Young Lives Project in Ethiopia, there were various discussions amongst the Young Lives Project Ethiopian team – as well as with other knowledgeable individuals. On the basis of the various facts and discussions, and given the limited scope of the project to cover such diverse factors, the following five regions were found to be most plausible and thus chosen as main centres for selecting sentinel sites:

• Tigray
• Amhara
• Oromia
• Southern Nations Nationalities and Peoples (SNNP)
• Addis Ababa.

Furthermore, the significance of these regions in terms of their total population size, as well as the percentage of children aged 18 months or under, can be seen from the summary Table A.1 below. In short, these regions constitute 96 per cent of the total population of the country, and almost 97 per cent of the total number of children aged 18 months and below live in these regions.

Table A.1 Population by regional state

Note: This may not include some zones in certain regions.

Four sites were selected from each region (with the exception of Addis Ababa and SNNP), of which three came from poor areas and the remaining one from a relatively less-poor area from the region. This allocation of sentinel sites, however, did not include Addis Ababa and SNNP for the following reason.

The issue of how many sites should be selected in Addis Ababa has been raised on various occasions. Initially, it was proposed that, as with the other regions, Addis Ababa would also have four sites. On the other hand, instead of concentrating all four sites in Addis Ababa it was thought that it would be more interesting and beneficial for the project to include intermediate towns in the survey. In view of this, it was decided that three sentinel sites would be selected from Addis Ababa and the fourth from one of the intermediate towns in the other regions such as Awassa, Nazret, Bahir Dar and Mekelle. Interestingly, officials from the SNNP region also suggested similar ideas during the consultation process. Their suggestion was based on the fact that there appears to be a fast growing urban population due to rural-urban migration and urban poverty is becoming a big issue in the regional capital, Awassa. It would, therefore, be reasonable to include such towns in the study. Taking the above into consideration, one sentinel site was selected from the city of Awassa, which is considered one of the growing intermediate cities in the country.

Stage 2: Selection of Weredas

Given the limited local information available for determining specific sites for the study, it was found necessary to undertake field trips and further consultations with local officials. In view of this, a team of researchers was dispatched to each of the regions to collect more local-level data and information, as well as undertaking consultations with relevant regional offices and other stakeholders. The main purpose of these consultations was to select Weredas (the next administrative structure after the region) which satisfy one or more of the factors already identified for the selection of sentinel sites in the region. These included:

• population density
• prevalence of critical food deficit (for example, dependence on food aid)
• calorie-intake
• child-mortality rate
• wasting, stunting, etc
• environmental degradation
• primary school enrolment
• access to health services
• infrastructure
• employment opportunities
• specific social or cultural peculiarities/practices, etc.

Furthermore, because of the differences in the rural and urban way of life in Addis Ababa, with associated poverty characteristics, the following factors were used as a basis for discussion to identify specific Weredas in Addis Ababa. These included:

• income
• urban congestion or housing problems
• poor social services such as school, health and other services
• water and sanitation problems
• prevalence of crime
• child labour, etc.

Based on these and other factors raised in the consultation process, three to five Weredas were selected in each region. One of the major problems of this process was that, while it was important in terms of involving the local officials and experts, the selection was mainly dependent on local knowledge, experience and information and thus, to a certain extent, it lacked the empirical data readily available on the above factors. Nevertheless, it was the best possible approach and provided adequate information in determining which Weredas to select.

The Weredas were chosen from each region to represent the following groups: rural-poor, urban-poor, and relatively less poor for both rural and urban households. Such a combination is important in order to give a relatively balanced picture of children’s welfare in terms of rural-urban settings on the one hand, and poor and less poor on the other. The terms ‘urban-rural’ and ‘poor and less poor’ are relative terms specifically used for the project and this may raise some questions. For example, a big city such as Addis Ababa and a small Wereda town are both referred to as urban areas at this stage. However, this does not mean that the outcomes of the survey will necessarily be collapsed together in the analysis.

The list of Weredas in each region and their corresponding characteristics are presented in Table 2.2.1.

Stage 3: Selection of Peasant Associations (in rural areas) or Kebeles (in urban areas)

Under the Young Lives Project, a sentinel site can be defined as a given area comprising at least 150 households eligible for enumeration under the project objectives. Thus, a sentinel site requires at least 100 households each with a child of 6 to 17.9 months old, and another 50 households with children aged 7.5 to 8.5 years. Since the overall average of children aged 18 months or under is estimated to be around five per cent, it may need more than 2,000 people to obtain the required sample of children under 18 months old from one sentinel site. If we further assume that another 1,000 people are needed to obtain the 7.5 to 8.5 year olds, this implies close to 600 households in total – assuming an average size of 5 persons per household. Taking this into consideration, a Wereda would be too big to be considered a sentinel site since it would be difficult and impractical to select households satisfying the required objectives from a vastly scattered and large population. This called for a narrowing down of the coverage area to the next administrative level, which in this case is the Peasant Association in rural areas and Kebele in urban areas.

A Peasant Association (PA) or Kebele is the lowest level of administrative structure in the country. The number of PAs or Kebeles from one Wereda to the next may vary significantly depending on the geographical coverage, population or other factors. It is also important to note that a Wereda classified as poor does not necessarily imply that all the PAs or Kebeles within that Wereda are also poor. In other words, unless PAs or Kebeles are carefully selected from each Wereda using criteria consistent with those applied to select the specific Wereda itself, a sentinel site may actually end up being selected from an area that does not meet its intended objectives.

Thus, in order to avoid such problems the selection of PAs and Kebeles was done through close consultations with Wereda officials and local experts. Two major approaches need to be followed when undertaking such consultations:

1. The Wereda officials should be clearly informed why and how the specific Wereda has been selected for the study. This includes explaining the objectives of the project, the criteria used and the consultations undertaken at regional levels and any available information useful for the purpose.
2. Then PAs or Kebeles will be selected on the basis of the consultations and recommendations. It is, however, important to note at this stage that such recommendations may not be always sufficient by themselves to determine the PAs or Kebeles. Other factors, such as the distance or remoteness of the PA, logistical and other issues related to the practicality of the fieldwork should be taken into consideration. For example, experience from regional level discussions shows that areas not easily accessible (for example, pastoral areas) have sometimes been recommended, but issues such as logistical and tracking problems make such recommendations very difficult. In short, one has to carefully balance the objectives of the project on the one hand and the practicability of the fieldwork and follow-up on the other.

Following the above procedures, one PA (in rural areas) or Kebele (in urban areas) was selected from each Wereda. The selected PA or Kebele could either be considered as a sentinel site in its own right or as a centre for creating a sentinel site along with adjacent PAs or Kebeles depending on the number of eligible households within the PA or Kebele. Further explanation with regard to PAs or Kebeles is provided in Stage 4 below.

Stage 4: Definition of a sentinel site

Once a PA or Kebele is selected, the definition of a sentinel site may vary from rural to urban areas or between rural areas, depending on the number of eligible households in the specific PA or Kebele. The following guidelines were followed in determining a sentinel site.

1 – Urban areas: In an urban setting, it is less likely that a specific Kebele will not have a sufficient number of households eligible for both categories (ie, one- and eight-year-olds). If, however, there exists a case where one cannot obtain the required number of households from one Kebele, then a standard procedure will have to be followed to complete the required sample size in a sentinel site.

First, the number of households in a sentinel site under the two categories in the study is the following:

• at least 100 households with a child 6 to 17.9 months old for the one-year-old questionnaire.
• at least 50 households with a child 7.5 to 8.5 years old.

Given this, if there are enough households satisfying both criteria in one Kebele, then that Kebele will be considered a sentinel site in its own right. If, however, there are not enough households fulfilling either one or both of the criteria, then that Kebele will be considered as a ‘centre’ point in establishing the sentinel site. That is, a point around which adjacent Kebeles will have to be included – using some kind of standard criterion to form a sentinel site. In this case, the following procedure will be followed as standard in establishing the specific sentinel site with the selected Kebele being at the centre of the selection process. The procedure is as defined below.

Figure A.1 Selection of Kebeles for a sentinel site

First, list exhaustively all the eligible households for both the one-year and eight-year-olds in the selected Kebele. As noted earlier, consider this Kebele the centre point of a given area of interest, or in other words a point around which the sentinel site will be established. Then, in order to complete the required sample sizes for a sentinel site move to the next adjacent Kebele, lying on the northern side of the Kebele under consideration, provided that both Kebeles are under the same Wereda administration. If this is the case, then continue listing eligible households until the required number of households is reached. If not, move towards the eastern side of the originally selected Kebele (or centre point) and choose the eastern adjacent Kebele, again provided that the Kebele is within the same Weredaadministration. Following this procedure, continue clockwise until one gets a Kebele (or Kebeles) satisfying the requirements – both in terms of their administration being within the same Wereda, as well as having the required number of eligible households for a sentinel site. These Kebeles together are then considered as a sentinel site. An example of this is depicted in Figure A.1 below. Kebele A in the diagram is the starting point, then one moves to Kebele B assuming that Kebele B is within the same Wereda. If not, Kebele C followed by E be chosen as candidates. In this case, it is possible that Kebeles A and B together could be considered as a sentinel site, given that they are in the same Weredaand the required number of households are taken from both Kebeles.

2 – Rural areas: Similarly, a PA can be considered a sentinel site if – and only if – there are the required number of eligible households for both the one-year-olds and eight-year-olds. If, however, not enough households contain one or both, then the PA should be considered the centre point around which a sentinel site will be established. The establishment of such a sentinel site follows exactly the same procedure as above. Thus, as in the urban situation, a sentinel site could be defined as a combination of more than one PA in an area, depending on the availability of households who are eligible for the fieldwork.

Stage 5: Selection of households

Normally, if households are the sampling units in a given study it is common practice to select households using, say, some kind of random sampling procedure, provided that the sampling frame is readily available or one can easily prepare the sampling frame (or the total population of households) in a given area of interest. However, this study required households with children of certain ages in order to be eligible for the study. Hence, although possible, it would have been a very time-consuming and costly exercise to prepare a full list of all eligible households in a sentinel site. On the other hand, one can assume that children of the required age would be naturally randomly distributed over a large number of households and it is, therefore, sufficient to take the first eligible households found, given the time and cost required to do otherwise. In view of this, the sampling of households for the different questionnaires was undertaken as follows:

One-year and eight-year-old children
The one-year-old questionnaire needed at least 100 households with a child aged 6 to 17.9 months. A further 50 households were required for the eight-year-old child questionnaire. All together, the study involved 50 households. Given this, the listing of eligible households needed to encompass the following steps.

Step 1: List the number of villages (Gots) in a PA and pick one village randomly. Using the randomly selected village re-write your list of villages (by moving in clockwise direction from the selected village) so that the randomly selected village is at the top of the list. In the case of urban areas, divide the Kebele into four quarters using north, south, east and west. Then, starting with the northernmost, continue clockwise to the western part of the Kebele .

Step 2: Start from village number one for the rural case (or north of the Kebele in urban case) in your list and using a door-to-door (house-to-house) check, prepare a list of those households in the sentinel site who have children of the required ages. Two forms are provided for registering the households, one for the one-year-olds and another for the eight-year-olds.

Step 3: If a household has children of both ages, priority should be given to the one-year-old, as a larger number of households is needed.

Step 4: If a household has a child (or children) of the required ages, ask the head of the household if she/he is willing to participate in the study. (Please read the consent procedure provided.)

Step 5: If agreed, include the household in the list of eligible households for enumeration in the appropriate formats provided, and also explain when the enumerator will be back to interview the household.

Step 6: Continue listing the eligible households until you obtain a minimum of 100 households for the one-year-olds and 50 for the eight-year-olds.

Step 7: If, for any reason, an already selected household(s) is(are) not able to participate during the interview, or they have unexpected problems rendering them unable to be interviewed (for example, change of heart for unknown reasons, moving to another area for some time, death, etc), then the household should be replaced by another eligible one using the same procedure. It is common to have a few extra households on the list for such reasons.

The Raven's test for eight-year-olds

The Raven's test has raised several issues and there have been a number of discussions along the following lines in relation to the test:

• how many children should be interviewed
• the level of skills required in undertaking the interview
• should the test be done in all sites or should it just focus on a few sites
• the lack of previous experience in undertaking the Raven's test
• the additional time and cost required in undertaking the test in all sites.

After lengthy discussions on the above key issues, the research team reached the following conclusions:

1. The Raven's test is a significant exercise and an important part of the study. However, it could be a time-consuming and costly exercise to interview all the eight-year-olds in all sites. Thus, it was agreed that 25 per cent of the children (or a total of 250 children) would be covered in the study.

2. Furthermore, scattering the 250 children throughout the 20 sites could have two consequences:

• It might be that all enumerators needed training in similar skills (or to similar levels) to carry out the Raven's test, as well as certain ideal situations being needed (see the manual for Raven's test for further information).
• The quality of information might not be as good as information collected by focusing on specific sites involving only a few well-trained enumerators for the purpose.

In short, given this, it was agreed that the Raven's test would be covered in five urban areas including all sites in Addis Ababa (see Table A.2 for details)

Table A.2 Number of children by sentinel site where Raven’s test was covered

Table A.3 Summary of poverty indicators in Ethiopia for the year 1999/2000

S = Stunting; SS = Severe stunting; LT= Literacy %; GPER = Gross primary enrolment ratio; NPER = Net primary enrolment ratio; GSER = Gross secondary enrolment ratio; NSER = Net primary enrolment ratio; P0 = head count index; W = wasting; SW = Severely wasted; P1 = poverty gap index; P2 = Severity of poverty index; FP0 = food poverty head count index.

Source: Calculated from the HICES and WMS data of 1999/2000 collected by Central Statistical Authority (References available at end of preliminary country report)

Table A.4. Mean distance to reach public services (by reporting level)