Whether to do a survey
Do a survey if most of these are true
- the concepts you want to ask about are very clear
- you need exactly the same information from a large number of people
- your research team has a high level of organizing skills: coordinating interviewers, clerical work, etc
- you have a computer with statistical or spreadsheet program (or, better still, a skilled researcher) to do the analysis.
Don't do a survey if most of these are true
- nobody in the research team is competent at using a spreadsheet or statistical software
- you need the results within a week
- you're not sure of the exact wording of the questions to ask
- you want the result of the research to be understanding rather than numbers.
If a survey is appropriate, read on to choose the best delivery method. If a survey is not appropriate, consider using a qualitative method, or response cultivation.
Choosing a survey method
Use personal interviews if most of these are true
- The survey covers only a small area, so that it takes interviewers no more than an hour or so to reach any part of the area.
- You have no previous experience at doing surveys.
- In areas where few people have a telephone (most developing countries) - or response rates on telephone surveys are very low (the USA).
- Respondents need to be shown something that cannot be described in words only (e.g. screen shots from a TV program).
- For long questionnaires - when interviews last for half an hour or more.
- There's a high chance of finding respondents at the planned place of interview.
Use telephone interviews if most of these are true
- The survey will cover a large geographical area.
- Telephone call costs are cheaper than interviewers' travel costs.
- An interview takes no more than about 15 minutes.
- It is possible to get an accurate sample list (e.g. from an up-to-date and complete telephone directory).
- You have a number of telephones you can conveniently use.
If several of those conditions don't apply, telephone interviewing is probably not a good idea.
Use observation when these are true
- The behaviour or object you are studying occurs in a public or accessible location.
- Observation will produce more accurate findings than simply asking people.
Observation is often done in conjunction with a normal survey. For example, in a face to face survey, the interviewer will observe the respondent's sex, rather than asking. And when respondents are being interviewed at home, a trained interviewer, though observation, is the best placed person to answer questions such as which short-wave bands are available on radios in the home.
Do a mail survey if most of these are true
- Your audience is highly educated (e.g. nearly all of them have completed secondary school).
- You have a complete and up-to-date list of names and addresses.
- You are in a country with a widespread, fast, and cheap postal system, that provides for business reply mail and/or freepost.
- You don't need the survey results for a few months (people are slow to respond to mail surveys).
- Your audience is very interested in the topic of the survey and conscientious at responding.
- You are highly skilled and experienced at writing questionnaires - or you have the patience to test a number of versions of the mail questionnaire by doing it as a personal interview. (If you're a beginner, expect to go through at least 5 versions before the questions are completely clear to respondents.)
A forbidding list, isn't it? You'd be wasting your time trying to do a mail survey, unless nearly all of those conditions apply.
Do an in-publication survey if most of these are true
- You are in charge of a print medium, and want to survey your present readers.
- You have strong reason to believe that the response rate will be very high (e.g. by offering a powerful incentive to return the questionnaire, and removing all barriers that might dissuade response).
- The questionnaire inserted in the edition is intrinsically interesting to fill in.
Do an on-the-spot survey (visitor survey or workshop) if one of these is true
- You have a fixed venue (such as an art gallery) and want to survey your visitors.
- You need people to assemble in a group so that they can react to material that you will present to them in a theatre situation.
Do an internet survey if most of these are true
- Almost all of your audience have internet access (hardly anywhere in the world, yet - maybe only Sweden and South Korea, in 2005).
- You have the computer skills to set up an internet survey - or you can find somebody to do this for you, exactly as you want it.
- You thoroughly test the internet survey before making it available to respondents.
Most of the time, it's not a good idea to do an internet survey among the general public. The main exception is when you're researching the audience of a specific website.
Use mixed methods when appropriate - for example...
- With personal interview surveys of business managers, first contact them by telephone to make appointments.
When you want people to keep a diary of their activities, seek agreement first by telephone, then send questionnaires by mail. (Because some personal contact will increase the response rate - and a low response rate is the main problem with mail surveys.)
Choosing a qualitative method
Do a set of consensus groups if most of these are true
- You need results within a week or two.
- Most potential participants live fairly close together.
- There is a suitable venue where they can meet.
- The budget can afford to pay about 30 people to come to meetings.
- You want quantifiable data as well as reasons for preferences.
Do in-depth interviewing if most of these are true
- The budget is very low, but the researchers have plenty of time.
- There are two people who understand the issues and will work well together on this project, as co-interviewers.
- Interviewers are able to travel to a venue of the participant's choosing.
- Interviewers have a high level of skill in conceptual thinking (e.g. university-level training in social sciences).
- There is no need to numerically extrapolate results to a population.
Use response cultivation if most of these are true
- A significant proportion of your audience spontaneously contacts you regularly. (How much is "significant"? - Perhaps more than 10% of the audience each year. If you don't get that already, encourage more feedback.)
- You have no reason to expect large fluctuations in your audience from one month to the next.
- You can't afford to conduct regular surveys. (Bear in mind that response cultivation is a second-best option.)
- You have at least one staff member who's interested in this type of work, and well-organized enough to do it accurately and regularly.
