Audience Dialogue

Know Your Audience: chapter 4, part B
Interview preparation

Choosing the respondents

If not all people in the household are to be surveyed, the next step for an interviewer is to choose one or more respondents. There are several methods for this (covered in more detail in the chapter on sampling, section 14). All of these methods are designed to give everybody an equal chance of selection:

1. The birthday method: choose the person (usually above a minimum age) who last hard a birthday, or will next have a birthday

2. The grid method: find out how many people in the household are over the minimum age to be surveyed, and choose one by looking up a table

3. A quota method: e.g. fulfilling a quota that 20% of interviews must be with men aged over 45.

4. The youngest/oldest man/woman method - choosing alternately the youngest man, youngest woman, oldest man, and oldest woman.

The birthday method (no. 1) can't be used unless everybody in a household knows the birth dates of everybody else. The grid method can't be used when household members don't know or won't tell the interviewer how many people live in the household. (For example, in some countries, with taxes on each person, households may pretend there are fewer people than there really are.) The quota method isn't as accurate as the others, and tends to under-represent people who are often away from home. Respondents sometimes find the youngest/oldest man woman method hard to understand ("How can I be the youngest man, when I'm 70 years old?" - Answer: He's the only man, so he's both the youngest and the oldest).

One way of overcoming the problem of choosing respondents within a household is to choose everybody. This works well when questionnaires are self-completed (it stops people from filling in others' questionnaires), but results in less efficient surveys in the case of personal interviews. Because people in a household tend to be similar to each other, the effective sample size is less than when one person is interviewed in each household.

Another alternative is to base the number of interviews in a household on the number of people. This overcomes some of the theoretical problems in choosing one person per household, a method which over-represent people living in small households. A common version of this is to interview one person in households with one or two adults, and two people in households with three or more adults.

Whichever method is chosen, the interviewers need to know

Making appointments

When a respondent who lives in the household has been chosen, that person may not be there when the interviewer first calls. In these cases, it will be necessary to make an appointment to return, when the chosen person is home.

But with door-to-door surveys, the samples are usually in clusters of households. Because of the time and expense involved in making many visits to a cluster, the interviewer needs to arrange appointments to avoid too many visits. Therefore appointment times need to be approximate. Instead of saying "I'll come back at 6.47pm tomorrow" the interviewer needs to say something like "I'll be back about 7pm tomorrow."

Of course, appointment times need to be recorded by the interviewer. It's also a good idea to give each household where an appointment is made a card showing the interviewer's name and the time and day of the appointment. This helps to ensure that the respondent will be at home at the appointed time.

Repeat visits

In societies where a lot of people are away from home working (e.g. most developed countries) it's usual for an interviewer to make 3 or 4 visits to a cluster of households. In societies where most people work at home, or nearby, two visits is often enough.

These visits can even be on the same day, but at different times. For example, in a survey that I organized in northern Ethiopia in 2000, interviewers had large clusters that often took several hours to walk around. These were roughly square-shaped, about 2 kilometres on each side. When an interviewer had finished the first traverse of a cluster on one day, they could then go around it again. This was usually several hours later, and people who were not at home on the first visit were often there when the interviewer returned.

When a household has been interviewed, that household is not visited again: it is only the households where interviewing was not completed that are revisited. Several studies have found that the best number of visits is a maximum of 3 or 4: on later visits, few interviews are made. It's usually a better use of survey funds to restrict the number of visits to 3 or 4 - depending on the distance and cost of interviewer travel to each cluster.

If, on the last visit, some respondents have never been found at home, substitutes can be made.

Substitution of respondents

In cluster surveys, each cluster is designed to include a fixed number of households - often around 10 of them. And usually one interview is done at each household. So the total sample in a survey is the number of interviews in each cluster, multiplied by the number of clusters. For example, a survey with 40 clusters of 10 households, and one interview at each household, will produce a total sample of 400.

What happens if some of those planned interviews can't be made? This can happen for several reasons:

I call these lost interviews. There are two ways of dealing with these: either the total sample size can be smaller than planned, or another interview is made in that cluster, to bring the total number of interviews back to the planned number. The latter choice is known as substitution.

The advantages of substitution are that the final sample size will be known in advance, and that the balance of respondents across clusters will follow the sample design. The disadvantages of substitution are that any problems with response rate are hidden, and simply adding extra households at the end of a cluster will not compensate for refusals if a particular type of person is refusing.

In general, I have found that it is better to use substitution. The main reason for this is that certain types of area have much higher numbers than others of lost interviews. These are usually inner-city areas and places with transient populations. Without substitution, these areas (and the types of people who live in them) are usually under-represented in a survey.

Substitution is normally done by adding extra households to the end of the route in a cluster.

Take the example of a cluster that should have 10 interviews, with a skip interval of 4: every fourth household is approached for an interview. So 40 households need to be walked past. Suppose that at one of the 10 households contacted, somebody refused to be surveyed, and that the selected respondent at another household didn't keep an appointment made on the previous visit. If this was the interviewer's last visit to the cluster, 2 more households need to be added. These are added to the end of the planned route. Beyond the 40th household, another three are skipped, and the interviewer tries to get an interview at the 44th household, and then another at the 48th. If one of these refuses, or the selected person is not home, the interviewer will have to walk past another three dwellings, and seek an interview at the 52nd household. In the end, 10 interviews are completed, even though the cluster may have grown much larger than the original span of 40 dwellings.

Screening respondents

When a survey does not cover the whole population, there must be a screening process to eliminate people who are not eligible to participate. This is done by asking a few questions, and eliminating people or households who don't give suitable answers. Sometimes these screening questions can be asked of anybody in the household, but sometimes the selected individuals must be asked. It's easier if anybody in the household can be asked the questions, but this is feasible only if everybody in the household knows the relevant details about everybody else.

For example, a survey might cover only people who listen to a particular radio station. Let's call it FM99. (This survey would not be able to find out what proportion of the population listen to the station, because non-listeners would not be included.) So a suitable screening question would be "Does anybody in this household ever listen to radio FM99?"

If the answer is No, the interviewer notes this in the log, and moves on to the next household in the route. If the answer is Yes, the interviewer finds out how many people listen to the station, and selects one or more respondents. If nobody who is there when the interviewer first calls know the answer to the question, the interviewer will have to call back later, when other people are at home.

When only a small proportion of the population are involved in the activity of interest (e.g. listening to FM99) asking screening questions in door-to-door surveys is very expensive. If only 1 person in 100 listens to FM99, about 100 households must be visited to find one listener. For this reason, door-to-door surveys usually cover the whole population. At least it is then possible to find out that 1 person in 100 listens to FM99.

But in telephone surveys, where people can be contacted much more quickly, screening is less expensive, so more common. Even for telephone surveys, when fewer than about 1 in 10 people qualify to be interviewed, screening is relatively expensive. It's common to spend more time on screening out the 9 in 10 unwanted people than in interviewing the 1 in 10 who you really want to contact. In this situation a good solution is to include a screening question on a survey of the whole population which is being conducted for another purpose. Then, as long as they give their permission, these respondents can be recontacted later for a more detailed interview.

Screening scores

Sometimes a screening question is obvious - for example "Do you ever listen to FM99?" But when you want to survey potential users of a service, not existing users, it's much harder to develop a suitable screening question. One way of doing this is to find out which types of people use a service, then ask a series of questions to identify these people. For example, I once needed to find out about people who were potential listeners to a radio station. An earlier survey had found that existing listeners to this radio station tended to read a particular newspaper, to prefer current affairs to sports programs, to have tertiary education, and two other criteria that I don't remember. We asked 5 screening questions. In simplified form, these included:

"Which newspapers do you read?"

"Would you rather listen to a sports program or current affairs?"

"What is your highest level of education?"

...and two other questions.

For each of these questions, a respondent who gave the same answer as most listeners to the station was given one point. Respondents who received 4 or more points for the 5 questions, and did not already listen to the station, were "screened in" - i.e. included in the main survey.

p> Interview logs

Another necessary piece of paper is the interview log: the interviewer's record of the attempts made to contact each selected household. With cluster surveys, when the interviewer visits a number of different households in a neighbourhood, a single log is used for the cluster, with one line per household. Usually there are between 4 and 20 households in a cluster.

With surveys that don't use clustering - such as telephone surveys - there is one page per household, or per telephone number. In this case, the logs can be smaller: often A5 size. The advantage of having a separate log for each interview is that the logs can be sorted into heaps: for finished interviews, numbers that will not produce interviews (e.g. refusals), pending appointments, and numbers not yet contacted. It's possible to do a telephone survey without using logs: the same information is recorded at the beginning of each questionnaire. However, a lot of paper will be wasted, because many telephone numbers do not result in completed interviews. Logs for telephone surveys are described in more detail in chapter 7.

In cluster surveys, logs are almost essential, because they give interviewers a record of their progress in the cluster, on a single sheet of paper.

Information on logs

The interviewer log contains a lot of information:

1. about the cluster, the interviewer, and the visits:

2. about the households and the interviews

On the next page is an example of a general purpose log, which you can change as necessary to suit your situation. The top section is usually filled in by the office staff, before the log is given to the interviewer. The interviewer only has to fill in the dates and times of visits, and the lines for the households.

INTERVIEW LOG FOR AMHARA SURVEY, 2000
Locality name: Fanshoma village (S.E. corner) Cluster no. 14
Address of starting point 27 Main Rd Interviewer B. Bloggeh
Visit 1 Monday 8 / 5 Arrival time 0927 Departure time 1235
Visit 2 Tuesday 9 / 5 Arrival time 1525 Departure time 1840
Visit 3 Thursday 10 / 5 Arrival time 1930 Departure time 2110
C = completed interview L = come back later A = appointment made
N = nobody home U = unable to interview R =refused
Address Visit 1 Visit 2 Visit 3 Interview Verified
29 Main Rd N L, C   5  
33 Main Rd C     1  
37 Main Rd L R      
41 Main Rd U        
45 Main Rd C     2  
49 Main Rd N A Thu pm C 6  
6 Conduit Rd C     3  
20 Conduit Rd N * see over      
?? Conduit Rd: Mrs Papamoa A (Tue) C   4  

Notes on the above Interview Log

You can see from the above notes what a wealth of information is available from an interview log - but of course each log must be accurately filled in, to be useful.

The back of the log is usually blank. This space is used for explanatory notes about particular households (which are rare). I also encourage interviewers to write general comments about a cluster on the back of the log. These comments can be useful when planning later surveys, or for resolving problems.

Persuading the unwilling respondent

When a person selected as a respondent is found, the next step is for the interviewer to persuade that person to be interviewed. In most developing countries, this is no problem. However, if refusal rates rise above about 10%, this can affect the accuracy of the survey. (The people who refuse to take part may be consistently different in some way - for example, if they are told this is a survey about radio, the people who don't listen to radio may often refuse.)

It's therefore important for the interviewer to be persuasive. When the interviewers are being trained for the survey, they can be given a list of likely excuses, and given reasons to overcome those.

Reason for refusing

Response from interviewer

I hardly ever watch TV, so I'm no use to you.

Everybody's opinion is important, whether they watch TV or not. If you don't watch, you will only be asked a few questions.

I'm too old for this. Interview my children instead.

It's very easy, and we want to get opinions from everybody, no matter how old or young.

I'm too busy.

I can make an appointment to come back later, at a time when you're not busy.

I think TV programs are terrible, so I won't co-operate.

This is your chance to let the TV stations know the public's opinion of the programs.

Whenever somebody refuses to take part in the survey, the interviewer needs to find out the reasons for refusal.

Temporary and permanent refusals

Very often, when somebody refuses to be interviewed, this is because of the situation at the time. A woman whose baby is demanding to be fed, a man who is about to go off to work, a family having a meal, watching a favourite TV program, or being visited by friends - in all these situations, an interview will probably be unwelcome. So on receiving a refusal, the interviewer should find out the reason for this, and then ask "Can I come back later?"

In most cases, the respondent will agree to be interviewed on another day.

When a respondent cannot be interviewed

Sometimes you may find a person who cannot be interviewed. They may be deaf, mentally deficient, drunk, senile, ill, not have a language in common with the interviewer, or in some other way not able to communicate well. Usually, about one or two people in 100 are in this category. But are they always like this (e.g. deaf), or only temporarily (e.g. drunk)? If the difficulty in interviewing a selected respondent is only temporary, the best solution is to make an appointment to return another day. If the difficulty is permanent, the interviewer should write a note explaining the problem, and find a substitute person to interview. There are several ways to find a substitute:

There are arguments for and against all three of these approaches. Usually the second method is best, because the other two methods will distort the sample slightly.

It's important to distinguish between somebody who cannot be interviewed, and somebody who is not in the target population. If the survey is trying to represent the whole population of an area, people should not be excluded because they do not take part in the activity the survey is studying. For example, if you are doing a survey of radio listening, and trying to find out what percentage of the population listen to radio, people who never listen to radio must be included in the survey - otherwise you'd find that 100% of people listen to radio.

Even if some respondents don't think it's useful to interview them, don't need to answer many questions, they should at least be counted.

Some interviewers think that people who don't do the activity cannot be interviewed. An important part of interviewer training is to make interviewers understand the difference between those who don't do the activity, and those who can't be interviewed. Mistakes in this area will seriously affect the results of the survey. At the end of a survey, the percentage of potential respondents declared uninterviewable should be calculated for each interviewer. If any interviewer has more than a few percent in this category, all these cases should be carefully investigated.

Incentives

In many societies, people will speak freely to an interviewer with no need for incentives, specially if the organization sponsoring the survey is well known and the respondent can see some connection between giving opinions and receiving improved services. This is usually the case with media surveys, but often not the case for commercial products.

In other countries (Vietnam, for example), it's normal to offer a gift to people who give their time in this way. The incentive is not essential to gain cooperation, but part of a normal social exchange. In other societies, people would be insulted to receive what they might see as bribes, simply for giving their opinions. This particularly applies to gifts of money.

In still other places (e.g. parts of the USA) people are so tired of surveys that only a substantial gift will induce them to cooperate in the survey.

When choosing an incentive, you need to bear in mind:

The reason for a gift is that it should neutralize any perceived disadvantages of taking part in the survey. A problem is that different respondents really need different levels of incentive, but it's impossible to know in advance the ideal amount to give each respondent. It seems unfair to give uncooperative respondents larger incentives than cooperative respondents receive - but sometimes this may be necessary - e.g. offering larger incentives in rich areas than in poor areas.

Examples of incentives that I've used in surveys are lottery tickets, plastic kitchen articles, pens, and money (often equivalent to about one hour's average wages). In a survey in Vietnam in 1998, the local officials suggested a gift of a half-kilo packet of the spice MSG. I was doubtful, and the interviewers had to carry a lot of extra weight, but it was obvious from the expressions on some respondents' faces that they were quite delighted with this gift.

It may seem obvious, but incentives to take part are only effective if respondents know about them in advance. Except in societies where a gift will be taken for granted, it's no use to go through the interview, and only then tell the respondent there's a gift. I've found that incentives work much better - in terms of increasing respondents' cooperation - when they're given out at the beginning of the interview, not at the end.

However, in most cases, it's not necessary to offer incentives in developing countries. In developed countries, the most effective incentive is often a ticket in a lottery with a valuable and desirable prize. For example, if 1000 people are to be surveyed, to offer one prize worth $1000 will probably be a much better incentive than to give everybody a gift worth $1. (This is why lotteries are successful.)

One example of a prize was in a survey of musical tastes in Australia in the late 1980s. The prize offered was a CD player. At this time, the people who were most interested in music already had CD players, and the people who were not very interested probably would not have wanted one - specially as they would then have had to buy CDs to play on it. So the response rate was fairly low. This prize was probably not an effective incentive. Because the survey was intended to cover the whole population, it would have been better to offer a prize not related to music, but one which most people would value - perhaps a TV set, for example.