Audience Dialogue

Know Your Audience: chapter 11, part B
How to run a group discussion

5. Hold the discussions

Each group needs a discussion leader, or moderator. This person (preferably not a radio or TV presenter, in whose presence people may withhold criticism) feeds the group with stimuli, or material to react to, as well as encouraging the reticent to speak up, and discouraging the talkative from dominating the proceedings. And when the topic is radio, one essential function of the moderator is to stop the participants from talking about television!

Each group should have a second person from the organizing team, to act as secretary. Though it is possible for an experienced moderator to fulfil both functions, it is valuable to have a second organizer present, so that the two can compare their conclusions after the participants have left. But if too many people from the organizing team are present, participants are likely to feel nervous and inhibited. If there are 12 participants, there should not be more than about 5 organizers present. Apart from the moderator and secretary, the other organizers should hardly speak at all. Other people who may be present include:

Initial questionnaires

It hardly ever happens that all participants in a group arrive at the same time. Even when we ask people to be sure to arrive at the advertised starting time, some arrive late, and others arrive much too early. As soon as they arrive, participants are keen to know what will be happening. It can be tiresome to repeat this over and over again, as each new participant arrives. To give them something to do, we usually have a preliminary questionnaire, which people can fill in as soon as they arrive. Those who arrive late can fill in their questionnaire while the discussion takes place.

As well as giving participants something to do while the others arrive, these questionnaires can collect useful information. They can also be used to raise questions that participants can think about and discuss later: such as "If you could make one change to FM99, what would it be?"

These questionnaires are short and simple. I try to restrict them to a single sheet of paper, with questions only on one side. As some people prefer to communicate in writing, we let participants keep their questionnaires throughout the discussion, and invite them to write their thoughts and comments on the blank back of the questionnaire.

Seating arrangements

A good type of room arrangement is a large table (or several smaller tables pushed together) around which the participants sit in chairs. The tabletop is useful for filling in questionnaires, and for food, drink, and microphones.

Another good arrangement is a D-shaped one. The participants sit in a semi-circle, with the moderator near the centre. The straight part of the D is the wall on which the results are displayed. The secretary sits at one end of the D. If there is a photographer or video operator, this person usually stands somewhere near the other end of the D.

In some cultures, people prefer to sit on the floor. This is no impediment, but if the participants are to fill in questionnaires, you will need to supply a writing surface (such as a clipboard) for each participant.

Displaying data

An essential part of the consensus group technique is the poster: large sheets of paper, taped to the nearest wall, or on an easel. On these posters, the secretary writes the findings of the group, in large letters so that all participants can read them. (If some or most are illiterate, symbols can be used as well as words.) It’s also possible to use a blackboard or whiteboard, but large sheets of paper are best because they can be taken away and kept as a record of the discussion.

Lately we have found a better method. Instead of using a few large sheets of paper, we have used many small sheets of paper, about A4 size: one for each statement.

Electronic recording

If possible, the whole discussion should be recorded, either on video tape or on an audio cassette. Video has many advantages, but it has the disadvantage of requiring a camera operator. Though it is possible to set up a video camera and simply point it at a group of people sitting in a semicircle, most detail will be lost, without an operator to zoom in on each person as they speak, and to record the reactions of other participants.

You might expect that to have a video operator present would greatly distract the participants, but we have found this is not so. After the first few minutes, participants usually stop noticing the video camera. (This is obvious when replaying a video recording, because participants seldom look directly at the camera.) The video operator should intrude as little as possible, and stand back at a reasonable distance. Note that, to focus on a whole group of 12 or more people, you will often need a larger room than you might expect — or a video camera whose lens can zoom out to an unusually wide angle.

If a video camera is not available, the next best thing is to record the discussion on audiocassette or minidisc. This does not require a separate operator — the secretary or moderator can work it. Some requirements for successful audio taping are:

• With a video recording, you can see who is speaking, but with audio, some voices may sound the same. To help identify voices, the moderator should address participants by name as often as possible. The moderator may also need to describe other things that are happening which will not be recorded on the tape. For example, if a participant shows the others a printed advertisement, the moderator should describe what is happening.

• You need a good microphone. After much experimentation, we discovered the boundary layer or PZM microphones, which are made by several manufacturers. A boundary mike looks like a flat piece of metal, about 20 cm square. It sits on a tabletop or other flat surface, and records all sounds reaching that surface. The microphones built into cassette recorders are usually of too low a quality to pick up voices clearly, specially when several people are speaking at once, and the microphone is more than a few metres away. Microphones designed for broadcasting don’t work well in this situation, because they are designed to pick up sounds from only one main direction

• Double-check that everything is working! It is surprisingly easy for something to go wrong when you are making a recording. Batteries may go flat (in either the recorder or the microphone), plugs may not be inserted fully, the tape can be set to "pause" and forgotten, the volume control can be turned right down, the machine can be switched to "play" instead of "record", and so on. I prefer a tape recorder with an indication of the volume (flashing LED lights or a VU meter), and a tape compartment with a clear window so that you can check that the tape is turning.

Identifying participants

We use name tents to help identify the participants to each other and to the organizers. A name tent is a folded piece of cardboard, with a participant’s name written on it, like this:

name tent

If the name tents are put on the table in advance, this can be used as a way of planning where people sit. If a few of the participants are from the same family, they are likely to distract the other participants by whispering if seated next to each other, so it is a good idea to separate them.

If the participants are not sitting at a table, an alternative to using name tents is to draw a map of the seating arrangements, marked with all the participants’ names. This can be displayed on a wall for all to see.

6. First stage: introduction

When almost all participants have arrived, the discussion can begin. In the first stage, the moderator introduces himself or herself, and asks each other participant to do the same. Here we are looking for information that will help to understand that person’s later comments. For example, if the topic is a radio station, I find it helpful to ask participants to describe their households, their daily routine, and their radio listening habits. Another purpose of this first stage is to help each participant gain confidence in speaking to others in the group.

Not a lot of detail is needed here: each person should talk for one or two minutes. The moderator’s own introduction will set an example for others to follow.

This stage usually about 20 minutes, for a group of 12 people.

7. Second stage: discussion

After all participants have made their introductory talk, the moderator begins the second stage, by outlining the issues that the discussion will cover. This should be done very broadly, so that issues will be placed in their context. For example, if the study is about a radio station’s programs, it is useful to gather opinions on other media which may compete with radio for people’s time: television, newspapers, and so on.

If the participants do not already know which station is conducting the research, it may be best not to tell them just yet, so that their opinions will be more unbiased. In some cultures, if participants know which organization is conducting the research, they are reluctant to criticize it in the presence of people from that organization. Almost certainly, they will identify the organization later in the discussion, so the early stages are the only opportunity to hear completely unbiased opinions.

When each participant speaks in turn, this ensures that everybody has a say, but doing this for hours makes conversation very awkward. My preference is to begin and end the discussion phase by asking each participant in turn to say something about the topic. For the rest of the discussion phase, anybody can speak, as they wish. Sometimes the moderator needs to intervene to prevent one or two people from dominating the discussion, or to encourage the shyer participants to have their say.

The organizers should not be drawn into the discussion. If a participant asks the moderator "what do you think of this matter?" the moderator should answer "it’s your opinions that are important here, not mine. I don’t want to influence you. " The purpose of the meeting is for the listeners to provide information to the organization that is conducting the research. But participants may try to reverse the flow of information, and begin questioning the organizers in detail. If the organizers respond, much of the meeting time can be wasted. The moderator should handle such questions by stating that there will be a "question time" at the end of the meeting.

The discussion itself can be either structured or unstructured (the difference is explained below), and will typically run for 1 to 2 hours. The moderator can usually control the duration. While discussion takes place, the secretary notes down statements which most of the participants seem to agree with. These statements are usually not written on a poster at this stage, but on a sheet of paper for the secretary’s later use. We’ve found that displaying statements soon seems to stop participants from thinking further about a topic. If anything is written on a poster at this stage, it should be a question or a topic, not a conclusion.

Unstructured discussions

An unstructured discussion is one in which the moderator merely introduces the broad topic, and sits back to let everybody else talk. The advantage of this approach is that participants feel unfettered by restrictions, and may provide useful and unexpected insights. The disadvantage is that much of the discussion may be irrelevant, and will not provide useful information. Therefore it is normal for the moderator to interrupt when the discussion drifts away from the stated topic. For example, if the stated topic is radio, the discussion will often drift onto television.

If the organizers have a list of questions they want answered, an unstructured discussion will usually cover most topics, without the moderator having to introduce the topics one at a time. The moderator can have a list of topics, and check them off as they are covered. Towards the end of the discussion period, the moderator can ask specifically about topics that have not been discussed.

Structured discussions

With a structured discussion, the moderator introduces issues one at a time, and asks the participants to discuss them. The moderator should avoid asking the participants any questions which can be answered Yes or No. (If that is the type of information you want, you should be doing a formal survey, not qualitative research.) Instead, the moderator should say things like:

• "Tell me some of the things you like about the breakfast program on FM99."

• "Why do you no longer listen to that program?"

Both of the above questions are the type that seek detailed responses, and are loose enough to allow for unexpected answers. If the questions asked are too specific, you risk not finding out the essential facts.

Another type of structured discussion involves playing excerpts of programs from a prepared tape. Reactions to specific programs are uslaly more useful than general comment. For this purpose, we normally prepare tapes of 10 to 20 short program extracts, averaging around one minute — just long enough to illustrate a particular point, or jog the memories of people who are not sure whether they have heard or seen a program before.

Play one item at a time, then let people talk about it for a few minutes.

Consensus group participants often express themselves vaguely, making comments such as "I’m not very impressed with the news on FM99." Statements like this are not useful, and need to be followed up with questions, such as "What is it that you don’t like?" or asking them for some specific examples. If the moderator does this a few times early in each discussion, the other participants will see what sort of comment is most useful.

Structured vs unstructured

The main advantage of an unstructured discussion is that it will often produce ideas which the moderator and secretary have not thought of. The main disadvantage of unstructured discussions is that they take a lot longer, because participants tend to drift off the subject.

However, a discussion need not be wholly structured or wholly unstructured. It can begin in one of these modes, and move to the other. It’s best to begin unstructured, and introduce structure later. If you do it in the reverse order, beginning with a structured discussion then allowing some unstructured time at the end, participants seldom come up with fresh ideas.

Generating statements

The output of a consensus group is a set of statements. These can come from either the organizers or the participants:

From the moderator or secretary:

From participants:

Initially, some participants make timid statements that provide little information, even though most others would agree. For example, "the sky is blue" is clear and concise - but how useful is it? Even worse: "the sky is often bluish, but sometimes it’s more grey." When statement-making begins, the moderator should encourage participants to make more daring statements, which may only just reach a consensus. For example: "In this area, in the dry season, the sky is mostly blue for five days each week."

In some societies, such as Cambodia, oratory skill is prized, and participants often make short speeches instead of statements. In other societies, such as Australian farmers, statements are so short that they will not be clear to others outside the group. In both cases, it is up to the moderator to convert such utterances into clear, concise statements - typically 10 to 20 words, much like a survey question. The same criteria apply as in questionnaires (see the Questionnaires chapter): statements must be unambiguous, deal with a single issue, and be clear enough for people to agree or disagree with.

At this stage, the wording of a statement is mainly up to the person making it. Too much correction by the moderator can dissuade other participants from making statements. If there are minor problems with wording, these can be fixed later.

When participants can think of no more statements, it’s often good to have a short break. Participants can have a drink, and some time to marshal their thoughts. In a recent consensus group in Vietnam, the trainee moderator thought participants were a bit gloomy, so he got them all to sing a song!

8. Third stage: consensus

In the final stage, we seek consensus on the issues discussed in the second phase of the session. By this time, the secretary will have a list of statements that he or she thinks most participants may agree with. The secretary now displays each statement in turn, and the moderator invites participants to agree or disagree with it.

The object is to find a statement phrased in such a way that the great majority of participants are able to agree with it. The original statement will usually need to be altered somewhat so that as many people as possible can agree with it.

The simplest way for participants to show agreement is by raising their hands. This works well in some cultures, where people are more assertive or individualistic (e.g. Western and Chinese-influenced societies), but in more group-minded cultures (e.g. Buddhist societies, Japan) participants are often reluctant to show that they disagree with others.

A more discreet way to indicate agreement is for a participant to hold up a palm-sized piece of cardboard. This can be held close to the chest, and is much less obvious to others than an upraised hand.

We normally give each participant a bright green card - the colour needs to contrast with the participants’ clothes, so that the cards will be visible in the videotape or photographs.

Voting is in two stages. For each statement in turn, the moderator reads it out, and shows it written on the wall.

The first stage is to check that everybody knows precisely what it means. Participants are asked to raise their hands (or their cards) if they are certain of its exact meaning; often they will not be. If even a single participant admits to being uncertain, the statement should be reworded. Sometimes the person who first made a statement needs to be asked exactly what it means. The secretary or the moderator will then suggest a revised wording, and participants are again asked if they know exactly what it means. When all participants are certain of the meaning, the voting can go ahead.

Participants are now asked to display their cards or raise their hands if they agree with the statement. Some people want to half-agree with a statement. We tell them: "Unless you’re sure that you agree, don’t hold your card up."

As the moderator reads out each statement, and points it out on the wall, and asks how many agree with it, the secretary (who is facing the participants) counts the number of cards or hands being held up.

We declare a consensus if at least three quarters of participants agree: at least 8 out of 10 voters, 9 out of 11, or 9 out of 12. Whether the threshold of consensus is set at 70% or 80% or 90%, it makes little difference to the results.

If only a few participants do not agree with a statement, they are asked which parts of it they don’t agree with. The moderator asks these people "Could you agree with it if a few words were changed?" Often, this is possible. If a statement expressed in an extreme way is softened a little, more people will agree with it.

For example, this may strike some people as extreme...

• People who do not listen to FM99 should have a loudspeaker set up in the street outside their home and be forced to listen to FM99 all day.

Not many people would agree. A few more might agree that...

• People who do not listen to FM99 should be given a free radio which only receives FM99.

But hardly anybody would disagree with...

• People who do not listen to FM99 should be told that it exists.

When consensus has been reached, the secretary writes the modified statement on the poster, together with the numbers who agree. For example, if 11 out of 12 people agreed:

People who do not listen to FM99 should be told that it exists. 11/12

When there’s no consensus

Sometimes it is not possible to reach consensus, and the group will divide into two sections, with no common ground. In such a case, try to get two statements, each with as many as possible agreeing, and record both statements, with appropriate annotation.

It’s best to put non-agreed statements aside until all statements have been worked through, then come back and reconsider the non-agreed statements. There are two reasons for this. Firstly, it’s possible to spend so long arguing over non-agreed statements that the group has no time to finish properly. Secondly, after all the original statements have been worked through, participants will have a better knowledge of each other’s positions, and more will easily agree on statements which they could not agree on at first.

If few people agree with a statement (no more than about 4 out of 12) it’s often useful to reword the statement as the exact opposite of the original. You might expect that if 4 of 12 agree with a statement, 8 should agree with its opposite. But this is often not true - sometimes almost everybody will agree with the opposite. In other cases, the group will be evenly split.

When between one third and two thirds of the participants agree with a statement, this can signal several things:

(a) The statement may be confused, or contain several different ideas: so try splitting the statement into two separate statements.

(b) There is a fundamental division of opinion within the group: in this case, reword the statement to maximize the split, so that there is a consensus within each of two factions.

After the secretary has finished going through the statements that were noted during the discussion stage, participants are asked to add statements that they feel most others will agree with. Each participant in turn can be asked to make a statement, which is then modified and voted on in the same way.

The moderator may then offer some final statements for evaluation. Unless this is the first of a set of consensus groups, now is the time when statements agreed in earlier group sessions can be shown to participants. Because the wording has already been clarified in the earlier groups, there’s usually no need to modify it now. It’s simply a matter of voting, which can be very quick in this case.

When the purpose of the project is to help an organization understand its audience, I’ve found it helpful to add some groups with the staff as participants, keeping staff of different status levels in separate groups. The statements produced by staff are often very different from those produced by the audience, and different groups of staff (unlike different audience groups) often produce statements that are very different from each other’s. This can be very educational for management.

In a typical 2-hour session, most participants agree on about 20 to 30 statements. As a final step, the moderator can ask participants to classify the statements into groups of similar statements, then for each of these groups of statements to produce a new statement summarizing the whole group.

Finally, the statements can be laid out on a large sheet of paper. Imagine all possible statements being spread out in two dimensions, as if on a map. An irregular shape on this map might define the statements with which most people agree. At the centre of the map are the statements that are so obvious that they are hardly worth stating. For example, all regular listeners to a radio station might agree with "I usually like what I hear when I listen to FM99." (Otherwise, they probably wouldn’t listen.) Towards the edge of the irregular outline on the map are the borderline statements, at the boundaries of agreement of the station’s listeners. An example of a borderline statement might be "Though I usually listen to FM99, I listen to other stations when I don’t like the FM99 program." These borderline statements tend to be more interesting, and less obvious. Beyond that borderline are the statements on which agreement could not be reached.

The consensus-seeking stage of the discussion will typically last between 30 minutes and one hour, depending on how many statements are offered for discussion. Sometimes a group will split into two sub-groups, which hardly agree on anything. In these cases (which are rare) the consensus stage will take much longer.

Why separate the discussion and consensus stages?

You may wonder why discussing issues and reaching consensus are presented as two separate stages of the discussion. Wouldn’t it be more efficient to take each topic one at a time, reach consensus on that, then move on to the next topic? I have tried this, but found it impedes the flow of discussion. Also, returning to a topic at the consensus stage gives people more time to gather their thoughts, and consensus seems more easily reached after a time gap. The only exception is when the discussion is structured, by being divided into a number of clear parts — for example when a lot of short program excerpts are being played to the participants, and they reach agreement on each one separately.

9. Summarize the results

Sometimes the most difficult part of running a consensus group is to persuade the participants to leave at the end of it. With some groups, nobody wants to go home, and most participants may sit around and talk for an hour or two longer; my record is 5 hours. I find these late sessions very useful. By that time, the participants know each other (and the organizers) much better, and may volunteer information which they did not want to do in the "official" part of the discussion. For this reason, I usually leave a tape recorder running for as long as possible.

The outcome of each group is one or more posters listing the agreed statements, showing how many people agreed with each statement.

After three group sessions, you will have three sets of statements. The reason for holding three groups is that one group may develop its thoughts in quite a strange way, perhaps due to one or two powerful personalities. With two groups, one of which may be atypical, you won’t know which is which, but with three, if the results from one group are very different from those of the other two, you will know that one is atypical.

Though you will never get three groups coming up with exactly the same set of statements, we have always found strong similarities. If at least two of the three groups came up with similar statements, these statements can be safely assumed to be representative of the whole audience sampled. No matter how differently the three groups are selected, we usually find a lot of agreement between the lists of statements when the discussions have been conducted in the same way. Observing the similarities will give you confidence that the results are true of the entire population studied, not only of the three disparate groups.

Have the three lists of statements typed out, or lay them out in map-like form as described above. This will be the basis of a report resulting from the discussions. Add a description of any taped stimulus material, and the criteria used for selecting listeners, complete a survey summary form, and that may be sufficient. If you have taped the discussions (whether on audio or videotape) you can copy relevant comments to illustrate each of the agreed statements, and include these in a more detailed report.

In summary, the consensus group technique is one that can be used by inexperienced researchers with reasonable safety. It is more difficult to draw wrong conclusions with this technique than with other types of qualitative research, and the findings are often more directly useful than those from formal surveys. However, the technique is not a simplistic one. Even highly experienced researchers can use it, to supplement the focus group technique. Inexperienced researchers will find that as they conduct more and more consensus groups, they will be better able to fashion the agreed statements. As well as being an accurate reflection of the participants’ opinions, these statements are more usable by the organization which has commissioned the research.