Audience Dialogue

Know Your Audience: chapter 4, part A
Fieldwork

If you are working in the media, you probably think of interviewing as a way of getting a story that can be broadcast or printed. The goal of journalistic interviewing, apart from accuracy, is to produce a story that will be of interest to the audience.

Research interviewing is different. The main purpose is not to keep an audience interested, but to get accurate and comprehensive information about the respondent. (In research, we have "respondents" rather than "interviewees.") Of course, journalistic interviewing also needs to be accurate, but reader interest is the first consideration. No two journalistic interviews ask exactly the same questions, but in survey interviewing, everybody is asked the same questions. The goal is comparison of respondents, and the ability to produce information about whole populations, rather than individuals.

There is one form of research interviewing which is very close to journalistic interviewing: that is depth interviewing. It is covered in chapter 10 below. Depth interviewing is not used for drawing firm conclusions about populations, but for initial explorations of a topic or a sample.

Fieldwork

When interviewers talk about the field, it's not a farm, but the place where they go to do their interviews: whether at people's homes, public places, or even a call centre where they do telephone interviewing. The term fieldwork includes all interviewing, as well as the activities that go with it: preparation, interviewer supervision, and verification.

There are two main forms of fieldwork: face-to-face interviews, and telephone interviews. Telephone interviews are much less laborious (no walking - just ring a number) but also more restrictive because nothing visible can pass between the interviewer and the respondent. The special techniques of telephone interviewing are covered in chapter 7, but most of this chapter applies to all types of interviewing.

Principles of survey interviewing

In theory, survey interviewing is very simple. You have a questionnaire, you read out the questions exactly as they are written, and you record the respondent's answers. What could be simpler?

In practice, as you will soon discover when you begin interviewing, many things can go wrong. Some people refuse to be interviewed. Some will say "I don't know the answer to that question - what's your opinion?" Some will even try to interview you. They may not answer the exact question you ask, either because they have misunderstood it, or because they don't want to.

One of the first principles of interviewing is that the interviewer must not affect the response in any way: the respondent should give the same answers to any interviewer. By your tone of voice, or even by your facial expression, you can show a respondent that you like or dislike some answers. Therefore it's important to ask each question in a completely neutral way, giving no hint of your own opinion.

Even the way you look and dress can affect respondents. Many studies have shown that respondents provide the most accurate answers to interviewers who are similar to them, in social status, sex, and skin colour. Interviewers who dress formally in a poor area can scare respondents - who may then give answers that they think the interviewer would like to hear, but which may not be true.

Much of the skill in interviewing lies in establishing a feeling of trust. The respondents need to be able to trust you - even though they don't know your opinions.

Interviewing people in their homes

It's usually best to interview people in their homes. People usually feel more comfortable at home, and are not in a hurry to finish the interview. If anything needs checking (such as the bands on a radio) this can be done more easily at home. It's harder to lie to an interviewer at home, with other people present. Finally, homes can be sampled more accurately than people can- because homes (unlike people) can't move around, so it's easier to count and list them.

Though it's usually best to interview people at home, in some societies this is not possible. In the cities of Papua New Guinea, crime rates are so high that most homes are guarded by fierce dogs, locked gates, or security guards, and it's not possible to reach many homes.

In traditional communities in Western Samoa, there's another kind of problem with surveying people at home. Samoans live in extended family groups, and it's traditional for the male head of household to make all dealings with strangers. This makes it difficult to interview women and young people.

Another problem with interviewing people at home occurs in western societies with high crime rates. In the USA, some people live in walled communities: housing estates with guards at the gates. Nobody is allowed in, except with the permission of the inhabitants. This also can apply to people who live in large blocks of flats.

When people protect themselves against contact with strangers, it can be very difficult to interview them. However, in most developing countries, this is not a problem, except when trying to interview very rich people.

Interviewing in public places

Where it's difficult to interview people at home, one alternative is to interview them in public places.

The main problem with interviewing in public places is that some people spend a lot more time in public places than others do. This can be an advantage when the subject of the survey is the public place itself - see chapter 9, on event surveys. But when you want to get a true cross-section of the population, you usually find that interviewing in public places will produce an uneven balance of sex and age groups. Though quotas can be used to ensure that (for example) equal numbers of men and women can be interviewed, other problems remain.

For radio and TV surveys, interviewing in public places will usually produce audience figures that are too low. Most radio listening and TV viewing is done at home, and the people who spend the most time listening to radio or watching TV therefore spend less time in public places.

Using age and sex quotas won't compensate for this. The only solution (which, from my experience, doesn't work very well) is to include a question asking how much time each respondent spends at home and in public places, and to use this information to give a different weight to each type of respondent. This method requires information from a census or highly accurate survey - and it still doesn't cover people who spend no time at all in public places - such as old and sick people, who often spend a lot of time listening to radio.

Telephone interviewing

In countries where at least about 80% of households have a telephone, telephone interviewing is one of the best methods of collecting survey information. But in countries such as the USA, people are bombarded by telephone calls from organizations trying to sell things. The public's defence is to use answering machines, silent numbers, call-screening services, caller identification, and so on, to avoid receiving calls from strangers. Some of these organizations pretend to be doing surveys, when their real purpose is to sell something.

This has made the public in these countries suspicious of surveys, and the cooperation rate with true telephone surveys has become very low - well below 50%. In this situation, telephone surveys no longer produce a representative sample of the population. Usually the wealthier households are under-represented, because these are the ones most often contacted by advertisers, and the ones most able to afford answering machines, etc. See chapter 7 for specific information on how to do a telephone survey.

Computer-aided telephone interviewing (CATI)

Most of the large market research companies in developed countries no longer use printed questionnaires for telephone surveys. Instead they use a computerized system, known as CATI (computer-aided telephone interviewing). In recent years, this is being called CAI, because it's no longer restricted to telephone interviewing - it's also used for door-to-door interviewing. The interviewers carry laptop computers.

With a CATI system, one question at a time is presented on the computer screen. The interviewer reads out the question, and types in the respondent's answer, or clicks it (if it is a multiple-choice question) with a mouse. The next question then appears on the screen.

Preparing for fieldwork

Workspace

It's difficult to run a survey without a good-sized office. In this space you can brief interviewers, keep piles of questionnaires, and analyse the data when the survey is finished. It's useful to have a large table. During training sessions, people can sit around it. When questionnaires are coming back, they can be sorted into heaps on this table.

Because the survey information is most vulnerable after the questionnaires have been filled in, but before the information has been entered onto a computer, the office should be made as safe as possible from fire, burglary, etc. If the completed questionnaires are lost, most of the research work is wasted.

Hiring interviewers

The qualities needed by a good interviewer include

  1. Ability to establish rapport with a wide variety of people. Successful interviewers are usually friendly, talkative types, with a lot of curiosity about others,
  2. Ability to fill in forms and questionnaires correctly. It's no use being friendly and talkative if you don't record the answers correctly - unfortunately, many of the friendly, talkative types are not good with the paperwork.
  3. Being physically fit: able to walk long distances and carry reasonably heavy loads, and not being upset by bad weather. Interviewers need to spend a lot of time outdoors, and survey organizations often underestimate the amount of walking that interviewers need to do.
  4. Not becoming upset when people refuse to be interviewed. (This is more a factor in western countries than in developing countries, and more for telephone than face-to-face interviewers.)
  5. Being resourceful and able to exercise initiative. (More important for face-to-face surveys than for telephone interviews.)
  6. In a door-to-door survey, interviewers should know their assigned area reasonably well. However, if the survey covers confidential subjects, respondents usually give more honest answers to interviewers who are strangers. This is a problem with surveys on subjects such as health, but not usually a problem for media surveys.
  7. Being able to read a map, and create a rough sketch map of the streets where they been interviewing people.

It's usually an advantage to hire interviewers with some experience of survey work. Even if the organization they worked for previously has different procedures from yours, much of the work will be similar, and they will already know what conditions to expect.

Training interviewers

In my experience, interviewers need at least two days' training before they can do any useful work. If they have done similar work recently, training can be shorter than this, but usually at least 4 hours' training will be needed for each new questionnaire.

A typical 2-day training program is :

1st morning: in classroom: principles of interviewing and respondent selection.

1st afternoon: introduction of questionnaire for this survey, with detailed explanations of each question. Trainer interviews a trainee, demonstrating how interview should go. Trainees interview one another, and any problems are then discussed. If supervisors are being trained at the same time, they can check the completed questionnaires. Interviewers are then given a fresh questionnaire, and sent home with instructions to interview a relative or friend overnight.

2nd morning: Questionnaires done overnight are checked, and any problems discussed. Training in completing logs. Interviewers are then sent out to nearby area to practice selecting a route and filling in logs.

2nd afternoon: Logs and routes are checked. Interviewers are then sent out to interview strangers with the questionnaire from the current survey. These are brought back, checked (by trainee supervisors), and problems discussed. Finally, a test is given (e.g. 20 multiple-choice questions). Trainees who pass the test are given their supplies. Those who fail are held back for further training in their areas of weakness.

The above training program intentionally includes a lot of repetition, several opportunities to discuss problems, and an approach that emphasizes understanding as well as procedures. Classroom instruction is interspersed with practical work.

In some cultures, this will seem an odd way to do training, but I've tried a number of methods, and have found that this works best: particularly the alternation of explanation, demonstration, practical exercises, problem discussion, and further explanation.

Prepare interviewer instructions

When the interviewers are being trained, they should be given a printed set of notes, which they can use for reference when they encounter a problem.

At the very least, the interviewer instructions should be a photocopy of the survey questionnaire which has been filled in as an example. Handwritten notes can be added, pointing out likely problems and giving information on how to fill in some items.

When fieldwork logs are being used, each interviewer should also be given a photocopy of a completed log, to use as an example.

Interviewer instructions can also include:

This railway diagram shows that everybody was asked the first two questions, but after question 2, some respondents skipped to question 14. After Q3, some people were asked Q4 and others were asked Q5. Some respondents were not asked Q7.

Which questions were asked of all respondents? The diagram shows that the only questions asked of all respondents were 1, 2, 14, and 15. Without using a railway diagram, it's difficult to answer this question accurately.

Training supervisors

Supervisors need to learn the same topics as interviewers, but to understand these more thoroughly than the interviewers do. They also need to understand the principles of editing, coding, and verification. Supervisor training is usually the same as interviewer training, plus an additional day:

3rd morning: verification, log checking, and revision.

3rd afternoon: editing and coding,

For a large survey, which will use many interviewers, it's a good idea to hold two training sessions:

When you hold several training sessions, the pilot testing can be done by the supervisors or the interviewers trained first, before all the questionnaires are printed. With the experience gained in the first, smaller training session, the large training session should run more smoothly.

Training office staff

When completed questionnaires are returned, they need to be counted, checked, edited, coded, and (if computers are being used) entered into a computer. This is covered in more detail in the next chapter. Supervisors also need to understand these processes - except perhaps computer data entry.

Area checking

When a survey will cover a fairly small area - one city or region - it's often possible to visit some clusters before the interviewers go there. This will give the survey organizers some idea of the problems likely to be encountered by interviewers.

If population information is poor, and a highly accurate survey is needed, you will need to use block listing (explained in section 11 of the chapter on sampling). This means that interviewers or supervisors will need to visit each cluster in advance, to work out how many households are there, and which ones should be sampled.

Even if there's no need to check all the clusters in advance, or the budget doesn't allow this, it's always a good idea for the supervisors to visit a few of the clusters that seem to be difficult in some way. So when the interviewers come to their supervisors with their problems, the supervisors will know what the interviewers are talking about.

Verification for door-to-door surveys

The international standard is that 10% of all interviews are verified (or validated). An interview can be verified in several ways:

The purposes of verification are to check that the interview was actually done (that the interviewer did not sit at home and make up all the answers) - and also to gain an estimate of the variability of answers, or the extent to which respondents change their minds

For verification to be effective, interviewers must know their work will be verified. They should also know that some of that verification will be unpredictable. For example, if verification is done only by having supervisors accompany the interviewer, it won't be so effective.

In most circumstances, cheating by interviewers is rare. But when an interviewer is inexperienced, and conditions are difficult, and there's a financial incentive to cheat, it occasionally happens. In all my years of managing audience research, I've known this to happen only a few times. Without verification, such cheating would be very difficult to detect; but the main deterrent is for interviewers to know that a tenth of their interviews (and they don't know which) will be verified.

For verification, you should prepare a shortened version of the questionnaire, omitting all questions which would not produce the same answer on a different day. Demographic details (age group, occupation, sex, etc.) hardly ever change, so these questions should be repeated to make sure the same respondent is being verified. Knowledge, habits, and awareness are more likely to change within a few days. Questions on attitudes produce much less stable answers, so there's little point in verifying these. There's usually no need for a verification interview to take more than five minutes, or include more than two pages.

Checking the early interviews

Soon after completing each interview, the interviewer should bring that questionnaire to his or her supervisor for checking. Interviewers should of course check questionnaires themselves, as soon as they have filled them in, but it's surprising how much a second person can see that the first person doesn't notice. At the beginning of a survey, supervisors should check each interviewer's questionnaires frequently - even daily - but later checking can be less frequent.

When supervisors go out with interviewers in their fieldwork, this should be done as early as possible. That way, any consistent mistakes than an interviewer is making can be corrected before many interviews are affected.

Interviewers' preparation

Before an interview can begin, an interviewer usually has a lot of work to do. In a normal door-to-door survey using clusters, or she must

Only then can the interview begin.

Plan cluster visits

It's usually necessary to make at least 2 visits to each cluster, because some respondents won't be home on the first visit. This principle is very important for audience research: ignoring it will produce overestimates of radio and TV audiences (because most media use is at home). I recommend making at least 3 visits to each cluster before substituting other respondents.

If a cluster is a long way from the interviewer's home, and the budget doesn't allow for overnight stays, an interviewer can make 2 or 3 visits on the same day, coming back at different times if necessary to find a respondent at home.

Travel expenses are a large part of the total cost of any door to door survey. The bigger the area that the survey covers, and the further interviewers must travel, the higher the travel expenses will be. Planning cluster visits so that interviewers travel a short a distance as possible is a very effective way of reducing the cost of a survey. For example, sometimes transport costs can be shared when several interviewers must go to a group of neighbouring clusters.

Finding the household

Much of the material in the next few sections has also been covered above in the chapter on sampling, but here it is presented from an interviewer's point of view, with attention to the practical problems.

Interviewing in clusters

Because many of the costs of a door-to-door survey are related to travel, reducing the amount of interviewers' travel can save a lot of money.

Therefore, most door-to-door surveys are done using clusters. Instead of selecting households scattered at random all over the survey area, households are grouped into clusters: usually 30 or more of these, with between 4 and 20 households in each cluster. The larger the cluster size, the less efficient the sample - but the smaller the sample size, the higher the costs. A cluster is usually small enough for an interviewer to walk from one end to the other in less than half an hour.

Every cluster has a defined starting point. This could be a dwelling (taken from a population list), or it could be a place such as a street intersection. If a detailed map of the area is available, each interviewer should be given a copy of that map, and the starting point should be shown on the map. If there are no maps, or if the maps do not show enough detail, the interviewer will take longer to find the starting point, and the verifier may not be able to find the same point.

Follow the route

When the cluster's starting point is found, the interviewer follows a set of rules for finding households. Some examples of these rules are:

I recommend using the first of those rules, if possible: i.e. follow the left hand side of the road, turn left if the road ends in an intersection, and come back along the other side if the road finishes with a dead end. This rule usually works well, in both urban and rural areas, and is easy for interviewers to follow.

Even if the starting point is a dwelling, it is normal not to interview anybody there. Why is this? Mostly because population lists always seem to be incomplete or out or date. Even the best population lists commonly omit at least 10% of dwellings. So by not interviewing at the starting point, all other dwellings are interviewed on the same basis. Further information is given in section 12 of the chapter on sampling.

Skip intervals

People who live next door to each other usually have very similar characteristics and opinions. So by making a cluster larger, spreading it out, a wider range of people will be interviewed, and a better cross-section of the whole area obtained.

One way of effectively increasing the spread of the sample without increasing the number of households per cluster is to use a "skip interval" - not interviewing at every neighbouring dwelling, but to leave gaps between the surveyed dwellings. This slightly increases the distance that interviewers must walk, but usually ensures that the end of a cluster is a different kind of neighbourhood from the beginning of the cluster.

I once made a study of this, and found that the best skip interval was 4 - i.e. interviewing at every 4th dwelling on the route: interviewing and one, and missing out the next 3. With this rule, using a cluster size of 10 (with one respondent per household), the last dwelling surveyed would be at least 40 dwellings away from the starting point. (More than 40, if some people refused to participate, or were not found at home.)

The more similar people are to their next-door neighbours, the larger the skip interval should be. I recommend in all cases a skip interval of at least 2 (i.e. interviewing at every second household), but no more than about 6. Above that, the interviewers tend to make mistakes counting households, and also have to walk much further, with no great increase in the diversity of the sample.