Audience Dialogue

Glossary of questionnaire design and psychology

A glossary of terms used in questionnaire design, as well as types of questions asked in surveys, issues covered in research, and the psychology of questioning.

The tendency of respondents to try to please an interviewer by answering "yes" whenever possible. Rotation can be used to overcome this problem.

Similar to Demand character: it's what you can do with a control. The affordance of a knob is to be turned ("Turn me"); a button is something you expect to press; and so on. Ever tried to click on something on a Web page that looks like a button, but isn't? That's an example of a poor affordance: a common finding in usability studies.

Cognitive dissonance
Changing your perceptions to match your circumstances. E.g. a religious group expected the world to end on a certain day. When it didn't happen, they decided that their faith had saved the world.

Cognitive interviewing
Asking questions in a sequence designed to help respondents remember their actions and experiences. Used by police in questioning witnesses of crime, but equally useful in improving people's recall of past events. This requires highly trained interviewers. For more information see Gordon Willis's Cognitive Interviewing: a How-To Guide.

Customer satisfaction measurement
A rapidly growing branch of market research: assessing the satisfaction level of an organization's customers. See also mystery shopping.

Demand character
Xrefer defines this as "a Gestalt term for those characteristics of a stimulus situation that invite particular modes of reaction to it." Got that? (It came from the Penguin Dictionary of Psychology.) We prefer Kurt Koffka's explanation, from his 1935 book Principles of Gestalt Psychology: "To primitive man each thing says what it is and what he ought to do with it...a fruit says Eat me; water says Drink me; thunder says Fear me." See also affordance.

A type of survey question that asks for information about the background of the responet, such as age group, gender, occupation, religion, education, and other types of questions asked in a census. The resulting measures are called demographics.

Type of self-completion questionnaire, which a respondent fills in over a period, recording his or her activity.

Donkey vote
A term used in preferential elections to refer to people who number their voting preferences 1, 2, 3, etc. down the ballot form. Also applied to the tendency for survey respondents to prefer answers listed earlier (or later). Can be solved by rotation.

A filter question determines the route taken through a questionnaire by a respondent. For example, the question "What is your occupation?" could be preceded by the filter question "Do you have an occupation?" Those who answered No would be filtered out, and not asked the question about their occupation. Similar concept to a screener questionnaire.

Fundamental Attribution Error (FAE)
The assumption that, if somebody behaves in an undesirable way, it happened solely because of the person, not the situation they are in. "He would say that, wouldn't he?" expresses it well. See also Ultimate Attribution Error

Plotting survey results on a map, so that the locations of respondents giving particular answers can be clearly seen. This is often done for marketing purposes - such as plotting the streets where rich people live, so that they can be bombarded with junk mail.

Hindsight bias
When presented with some research results, some say "I knew that all along." But they usually don't do so well when asked to guess the results beforehand!

A "physics envy" term for a questionnaire, test, or other documented procedure for measuring human reactions or behaviour.

"Just-noticeable difference" - the smallest different that a person can perceive. For example, in detecting a difference in volume between two successive sounds, the JND is usually around 2 decibels.

KAP surveys
KAP means "knowledge, attitudes, and practice". Sometimes it's extended to KAPB, adding "beliefs" at the end. Sometimes it's called KAB, with the final B standing for behaviour. This usually refers to surveys in the area of social marketing. A series of questions covers people's awareness about a issue (usually related to health), their attitudes to it, and their behaviour.

Leading question
A question that leads a respondent to give a particular answer. Often used in push polling. A classic example in a 1980s pseudo-survey was "Do you want Soviet-style conditions imposed on Australia?" The inevitable answer of No was interpreted as support for a particular policy.

Local government area: a town or rural area area administered by one authority, for which population figures and maps are usually available.

Likert scale
An answer scale, usually with a range of 5 ordered categories, e.g.
Strongly agree - Mildly agree - Neutral - Mildly disagree - Strongly disagree.

Mirror sample
When you suspect that respondents may not tell the truth about their behaviour, you can use a mirror sample. Instead of asking about "you", ask each respondent to think of a close friend of the same sex and age group, and ask them to answer for that friend - without naming the friend. The theory is that, because they don't know all the details about their friends, they'll really describe their own behaviour. Mirror samples have been used in KAP surveys, to ask about sexual behaviour, in studies of AIDS. Similar to third person technique.

When a survey interviewer confirms a respondent's answers by repeating them - e.g. "So you are in the 30-34 age group, you are married, with two children aged under 16 - is that correct?"

Multiple-answer & multiple-response questions
These terms often confuse people. A multiple-answer question is one which can have more than one answer, e.g. "Please tell me all the reasons why you did not re-subscribe." The opposite is a single-answer question, e.g. "Please tell me the main reason why you did not re-subscribe." - And a ...

A multiple-response question is one where a set of possible answers is offered to the respondent, who is asked to choose one of these. E.g. "Which age group are you in: 18 to 34, 35 to 54, or 55-plus?" In other words, multiple responses are offered to the respondent, who chooses only one option. The opposite of a multiple-response question is an open-ended question, e.g. "What is your age?"

"Non-English-Speaking Background". A term used mostly in Australia, describing people whose first language was not English, and their children. Sometimes they are called (jokingly) Nesbians.

A type of question (also known as an open question) where it is left up to the respondent to volunteer an answer.

Projection in this context means psychological projection: one of the defence mechanisms identified by Freud. When using projection, people see their own problems in others. For example, an interviewer who does not understand a question in a survey may write down that the respondent did not understand the question. The other sort of projection occurs when extrapolating from a sample to a population.

Similar to demographics, but divides the population into groups based on psychological characteristics rather than age group etc. For example, comparing the answers of men and women is a demographic approach, but comparing risk-takers with cautious people is psychographic.

The set of questions in a survey. Don't confuse a questionnaire with a survey. To say "250 surveys were returned" displays ignorance; actually, "250 questionnaires were returned." See instrument.

The way in which people's attitudes and behaviour can be influenced by having been surveyed. See also acquiescence.

When asking people to remember their behaviour in survey questions, two types of recall can be used: aided recall and unaided recall. Aided recall: "Did you watch channel 2 yesterday?" Unaided recall: "Which TV channels did you watch yesterday?" Aided recall usually produces higher percentages for that activity.

Revealed preference
A type of data used in conjoint analysis, based on people's actual behaviour - e.g. from databases of purchases. Contrasts with stated preference.

One way of overcoming the donkey vote, acquiescence, etc, in a questionnaire is to rotate questions and/or answers, so that they are not presented in the same order to every respondent.

(1) A set of possible answers to a question. A simple version is "On a scale of 1 to 10, how would you rate the Prime Minister?" As well as scales of numbers, there are also scales of words, such as the Likert scale.
(2) A set of questions which are trying to measure different aspects of the same thing. E.g. an IQ test might have a scale of 20 questions, whose answers are combined into one figure.
To develop a good scale is not a simple task. Much development work is needed to demonstrate that a scale actually measures what it purports to measure.

A screening questionnaire, as used by recruiters to determine who is eligible to attend a group discussion. Similar to filter questions, asked early in a questionnaire to weed out those not eligible to answer the remaining questions.

Selective perception
Noticing only what it suits you to notice - or not noticing things that don't interest you. The responses that people choose to give to open-ended questions are often influenced by selective perception.

Stated preference
A type of data used in conjoint analysis, based on people's information about their preferences - taken from questionnaires. The opposite of revealed preference.

Third person technique
Sometimes used in the same sense as mirror technique, and sometimes as an interviewer asking a respondent about himself or herself in the third person, making it easier to answer some uncomfortable questions. For example:
Interviewer: "Did he ever steal items from shops as a child?" ["he" here means "you"]
Respondent: "Yes, once he stole an apple from a fruiterer's stall."
This approach helps respondents distance "me, now" from "me, in the past".

Ultimate Attribution Error (UAE)
An extension of the Fundamental Attribution Error: that assumptions about one person's perceived habitual behaviour become applied to a whole group of people, often related to some visible or well-known characteristics of that group. A black-skinned man stole your wallet - so now you believe all non-white people are robbers: that's an example of the UAE. Much the same as essentialism.

Volunteer bias
Error arising from a low response rate, due to the fact that some types of people (usually young, well-educated) are more willing than others to take part in surveys.