This page is most of chapter 13 of Know Your Audience, by Dennis List, the first edition (2001). Since then, the consensus group technique has been improved in various small ways, but the basic method is the same. Over the next few months (i.e. by late 2005) we plan to update this page to cover those small improvements.
In the past, qualitative research has been regarded as too difficult for untrained people to carry out successfully. You may have heard of depth interviews or focus groups: two of the commonest forms of qualitative research. As well as requiring a high level of training (ideally a postgraduate qualification in psychology), both of these techniques are very time-consuming. The numerical results from a survey are processed by a computer in a few minutes, but the results of qualitative research are usually many hours of speech. Typically, the interviews are tape-recorded or videotaped, and often transcribed into printed form. The researcher then spends days listening to the tapes, watching the videos, and reading the transcripts, summarizing the findings. In my experience, one hour of interviewing usually requires about four hours of work afterwards - and beginners take longer, because they're not sure how to go about it. So there is a need for a simpler method.
Since the late 1980s, I have been developing a simple method for qualitative studies, by which local broadcasters can come into closer contact with their audience. This is a method requiring verbal skills (as commonly possessed by broadcasting staff) rather than the numerical skills needed for traditional survey research. This new method, which I call "the consensus group technique" (or simply consensus groups) is midway between a focus group and a public meeting. It also draws on other techniques known as nominal groups, search conferences, the Delphi method, citizen juries, and consensus conference, including some elements of all of those.
How does a consensus group differ from a survey? Think of it like this:
Though this chapter uses radio audience studies as an example, consensus groups can be used for any kind of audience. It has also worked well in assessing the adoption of innovations.
The technique has two main stages: recruiting participants, and holding discussions. But before participants are recruited, the same strategic concerns apply as with any other social research method. In short, the organizers must decide exactly what is to be covered, and among what population. Once the scope of the study has been decided, this is a brief outline of the procedures that follow, in 9 main stages...
1. Within the area to be studied (e.g. a radio station's coverage area), three sampling points are chosen, contrasting as much as possible.
2. At each of the sampling points, a venue is arranged. All you need is a suitable room.
3. A short screening questionnaire is prepared. The purpose of this is to find people eligible to take part in the consensus groups.
4. At each sampling point, people are interviewed using the screening questionnaire. These interviews find people who are both eligible and willing to attend a consensus group.
5. The group meets, either immediately after the interviewing, or up to several weeks later. This meeting will run for about two hours. At each meeting, there would be approximately 12 participants, and 2 organizers: a moderator and a secretary.
6. The first stage of the meeting is introductory. Each participant briefly introduces himself or herself to the others, giving some basic information about their habits - e.g. radio listening.
7. In the second stage of the meeting, the topics are discussed by all participants. The moderator steers the discussion around the arranged scope of the study, and the secretary takes notes.
8. In the third and final stage of the meeting, consensus is determined. On a whiteboard (etc.) the secretary writes down statements which most participants are expected to agree with. Statements are modified, depending on what participants say. When each statement is ready, participants vote on it. Typically, around 30 statements are agreed on by most of the participants. This list of statements is the main outcome of the meeting.
9. When three meetings have been held (one for each sampling point found in step 1), the three lists of statements are compared. Any statements shared by at least two groups are the outcome of the study.
Now let's look at each of the above stages in more detail...
A sampling point is simply a geographical area, where a small survey can be carried out. It can be either a group of homes, or a public place where passers-by are recruited for the study. For consensus group sampling to work effectively, at least three sampling points are needed. They are chosen to be as different as possible. For example, if your study is being done on behalf of a radio station, the population will probably be all people who live in that radio station's coverage area. Within this area, you should identify three contrasting localities. For example, if the station covers a city and outlying rural areas, you might choose:
Whatever the basis on which the three areas are selected, the main goal is to make them as different as possible. This is a type of maximum-variation sampling (as explained in chapter 2).
Three sampling points is a minimum, but there can be more. We have found that each additional sampling point adds less and less variation in the statements. However, in some situations, more than three sampling points are needed to adequately cover the variations in a population. For example, a technique we have often used compares three groups of a station's potential listeners with three groups of its current listeners. This requires choosing two groups at each sampling point. If both the sampling points and the type of person invited to a group are different, you cannot make clear conclusions about the causes of any differences in the results - hence the need for a lot more groups in this case.
A venue for a consensus group is a space that will hold about 15 people and is free from interruptions. It need not even be a room; in Papua New Guinea we held some group discussions outdoors, with no problems.
In Australia, many hotels have rooms available for hire for meetings, and we often use these. We have also used clubrooms of voluntary organizations, borrowed office areas at night, government offices, restaurants, shops, and so on. Another possibility is to use a private house, paying the owner a small fee for the use of their room. It may be better not to use the premises of the organization sponsoring the research, as people may be reluctant to criticize an organization when they are on its premises: for example, if your research is for a radio station, avoid using the studios as a venue. But this depends on the type of people attending, and on the organization.
Here are some factors to take into account when choosing a venue:
The main purpose of the preliminary screening survey is to find participants for the consensus groups. They must be both eligible and willing to participate. If, for example, you are assessing the programs on a radio station, you may decide there is little point in inviting people who don't listen to the station. In this case, the key question on the screening questionnaire would be
"Do you listen to FM99 at least once a week?"
(This is better wording than asking simply "Do you listen to FM99?" If no time limit is included, people who listen only very rarely to the station would answer Yes, and would not be able to discuss the programs in detail.)
If you are interested in increasing the station's audience, you may want to speak to potential listeners to the station. There are several ways to define potential listeners on a questionnaire. Our own studies have found that when a station increases its audience, this is usually because people who did formerly listen to the station, but infrequently, began to listen more often. So most of your potential listeners are probably listeners already - but not very often. In a screening survey, you could ask
"How often do you listen to FM99: at least once a week, very occasionally, or never?"
All those giving the middle answer would be considered potential listeners.
If you are trying to assess the audience to something that does not yet exist (such as a planned new radio station), you will need to define the potential listeners in another way. This can be done from two starting points:
"With the answers you've given, you're eligible to take part in a group discussion, to talk about news and current affairs in more detail. We'd like to invite you to come to this discussion next Tuesday night, the 15th of October. We're holding it at the Flinders Lodge, in Dequetteville Terrace, Kent Town, starting at 7pm, going for up to two hours. People usually find these meetings very interesting, and if you attend we'll pay you thirty dollars to cover your expenses. Would you like to come along?"
1 Not interested
Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The essential points included in the above question are:
The third type of question which can be included in a screening questionnaire is the demographic question: their sex, their age group, and perhaps their occupation and other similar data. There are two reasons for obtaining this demographic information:
As a screening questionnaire will have few questions, you can save paper by not having a separate sheet of paper for each interview. Using one form for everybody can be more convenient.
Interviewing for a screening survey is carried out using the principles described in chapter 3. However when doing a screening survey for consensus groups, it is not essential to interview people in their homes, following a prescribed route.
Unless the group discussions are to be held as soon as enough eligible respondents are found, it is best to use only one or two interviewers at each sampling point. Because you will be aiming for 12 participants in each group, a large number of interviews will not be required - unless those eligible to take part are only a small percentage of the population. In order for 12 people to turn up, you will probably need more than 12 to agree. In Australia, even when we send a letter to confirm the details of the discussion, and recontact each participant the day before the discussion, usually 10% to 20% of those who accepted fail to turn up. Therefore, we usually get acceptances from one more person than we really want: if we want 12, we obtain 13 acceptances.
In other countries, the proportion who fail to turn up may be different from the 10% to 20% figure we have experienced. We have found that unless we confirm each person's attendance (by a follow-up letter and/or telephone call) many more than 20% do not turn up. We have also found that people who say they "might" come nearly always don't.
One of the worst things you can do in a consensus group is to extend a weak invitation to a lot of people to come. As this is very easy, it may seem tempting. If you are running a radio station, you may think "Why not advertise on air that we are doing a research study, and invite listeners to come along?"
The problem is that you have no control over the number of people who turn up. It could be nobody, and it could be hundreds. We have found that 12 people is about the ideal number for a consensus group. With fewer than about 8 participants, there is too much danger of the responses being atypical - one very persuasive person might influence the others. With more than about 15, the group either runs on for far too long, or else it is abbreviated so much that many participants don't have a full chance to offer their thoughts.
In Australia, usually less than 50% of people eligible to attend a group will agree to do so. Bearing in mind those who are not eligible, the eligible people who do not want to come to the group, and those who say they will come but do not, sometimes it takes a lot of interviews to fill one group of 12. For example, if one person in 10 is eligible, and a third of those attend a group, that's 30 interviews for each person who attends, or 360 interviews to fill a group. It is therefore a good idea to make the eligibility criterion fairly broad. Another step you can take to reduce the number of interviews is to offer an incentive to attend. If you can persuade two thirds of the eligible people to attend instead of one third, only half as many interviews will be needed. Therefore if you pay the people for attending, this can greatly reduce the total cost.
We normally do the screening surveys between one week and two weeks before the discussion. If given more than two weeks' notice, people tend to forget. With less than a week's notice, many people can't attend, because they have already made plans to do other things at the time.
But it is not essential to wait that long. Another possibility is to hold consensus groups on the spot. We did this in Hanoi, Vietnam, where a number of interviewers were sent out to interview people in the street outside the venue. All people who met the criteria and had an hour to spare were invited to a group discussion then and there. It took only ten to fifteen minutes to find enough participants. Of course, this only works when a large number of eligible people are nearby, but if your organization is one that has a lot of visitors (such as a museum) this could be an easy way to find participants.
The above description of screening questionnaires involves a separate questionnaire for each person. People are interviewed individually. In the resulting groups, each participant will usually not know any of the other participants - except in small towns, where many people already know each other. In some ways this can be an advantage, because participants will not offend their friends by giving opinions they feel the friends might disagree with. But in other ways, it can be disadvantageous to hold a discussion among strangers: participants may feel unwilling to reveal their opinions to people they do not know and cannot trust. Which of these two disadvantages is the stronger will vary from country to country. In Australia, my experience is that when the participants in a group already know each other, they tend to express their feelings more freely.
Sometimes it is best to restrict each group to similar people - not asking all eligible persons to attend, but only some of them. For example, when obtaining opinions of listeners to a radio station, in a country where men and women do not mix socially with strangers of the opposite sex, you might need to organize four groups instead of three: two of men only, and two of women only.
Sometimes it is better to have separate groups of younger and older people. In some countries it may be best not to mix supporters of different political parties in the same group. This separation can be done partly by careful selection of sampling points, and partly through screening questionnaires. Bear in mind that the purpose of restricting a group to a particular type of person is to enable the members of a group to speak more freely.
It's usually best to dissuade husbands and wives from coming together. They tend to inhibit each other. Often only one of them will join the discussion. As each group is quite small, it would be better to invite two people who would give separate opinions.
A completely different approach, which can also work well, is to organize a discussion among a group of people who already know each other, such as members of a sporting club, a group of people who work together, or a group of students. And of course, the groups need not have a common purpose: they can simply be groups of friends or neighbours. However, I advise against using a group made up of members of a family. There is too strong a chance that they will not be typical of the population, because the entire study would then be limited to three families. These groups (of people who already know each other) are called affinity groups.
When affinity groups are used for a study, each group needs to be as different as possible from each other (replacing the three sampling points). For example, don't choose three sporting clubs, or three groups of neighbours. This type of sampling is most effective when there is the largest possible contrast between the types of person in each consensus group.
One problem which restricts the use of affinity groups is that not everybody in an affinity group may be eligible. If a radio station is studying its listeners, it does not matter if a few people in an affinity group are not listeners, but if most people are non-listeners, the group will not provide useful information. Also, if people are not interested in the topic being studied, they are likely to disrupt the discussion by talking among themselves. Therefore affinity groups are best when all or most people in the population are eligible to be interviewed.
Each group needs a discussion leader, or moderator. This person (preferably not a broadcast presenter, in whose presence people may withhold criticism) feeds the group with stimuli, or material to react to, as well as encouraging the reticent to speak up, and discouraging the talkative from dominating the proceedings. And when the topic is radio, one essential function of the moderator is to occasionally remind the participants to stop talking about television!
Each group should have a second person from the organizing team, to act as secretary. Though it is possible for an experienced moderator to fulfil both functions, it is valuable to have a second organizer present, so that opinions can be compared after the participants have left. But if too many people from the organizing team are present, participants are likely to feel inhibited. If there are 12 participants, there should not be more than 4 organizers present at once. Apart from the moderator and secretary, the other organizers should hardly speak at all. Other people who may be present include:
It hardly ever happens that all participants in a group arrive at the same time. Even when we ask people to be sure to arrive at the advertised starting time, some usually arrive ten minutes late, and others arrive much too early. As soon as they arrive, participants are keen to know what will be happening - but it can be tiresome to repeat this over and over again, as each new participant arrives, specially when you're getting ready for the discussion. So to give participants something to do, we usually have a preliminary questionnaire, which they can fill in as soon as they arrive. Those who arrive late can fill in their questionnaire while the discussion takes place.
As well as giving participants something to do while the others arrive, these questionnaires can collect useful information. They can also be used to ask questions which the people can think about and discuss later: such as "If you could make one change to FM99, what would it be?"
These questionnaires are short and simple. I try to restrict them to a single sheet of paper, with questions only on one side. As some people prefer to communicate in writing, we let participants keep their questionnaires throughout the discussion, and invite them to write their thoughts and comments on the blank back of the questionnaire, and hand it up at the end.
A table is not essential, but very useful. Usually the best type of room arrangement is a large table (or several smaller tables pushed together), around which the participants sit in chairs. The tabletop is useful for filling in questionnaires, and for food, drink, and microphones.
In some cultures, specially in Asia and Africa, people prefer to sit on the floor. This is no problem, but if the participants are going to fill in questionnaires, you will need to supply writing surfaces, such as clipboards.
An essential part of the consensus group technique is the poster. This is one or more large sheets of paper, taped to the nearest wall (or on an easel). On these posters, the secretary writes the findings of the group, in large letters so that all participants can read them. (If some or most are illiterate, symbols can be used as well as words.) It's also possible to use a blackboard or whiteboard, but large sheets of paper are best because they can be taken away and kept as the record of the discussion.
If possible, the whole discussion should be recorded, either on video tape or on an audio cassette. Video has many advantages, but it has the disadvantage of requiring a camera operator. Though it is possible to set up a video camera and simply point it at a group of people sitting in a semicircle, most detail is lost unless there's an operator to zoom in on each person as they speak, and to film the reactions of others.
You might expect that a video operator would greatly distract the participants, but we have found this doesn't happen. After the first few minutes, participants usually stop noticing the video camera. (You know this when replaying a video recording, because participants hardly ever look directly at the camera.) Of course, the video operator should intrude as little as possible, and stand back at a reasonable distance. Bear in mind that, to focus on a whole group of 12 or more people, you will often need a larger room than you might expect - or a camera whose lens can zoom out to an unusually wide angle.
If a video camera is not available, the next best thing is to record the discussion on audio cassette. This does not require a separate operator - the secretary or moderator can work it. Some requirements for successful audio taping are:
We use name tents to help identify the participants to each other and to the organizers. A name tent is a folded piece of cardboard, on which is written the name of the participant sitting at that place on the table.
If the name tents are put on the table in advance, this can be used as a way of planning where people sit. If a few of the participants are from the same family, they are likely to distract the other participants by whispering if seated next to each other, so it's a good idea to separate them.
If the participants are not sitting at a table, an alternative to using name tents is to draw a map of the seating arrangements, marked with all the participants' names. This can be displayed on a wall for all to see.
When almost all participants have arrived, the discussion can begin. In this stage, the moderator asks each participant in turn to introduce himself or herself. Here we are looking for factual information which will help others understand the opinions which will be stated later by that person. For example, if the topic is a radio station, I find it helpful to ask participants to describe their households, their daily routine, and their radio listening habits. Another purpose of this stage is to help each participant gain confidence in speaking to others in the group.
Not a lot of detail is needed here. I have found it best for the moderator to introduce himself or herself in the style needed, as an example, before asking participants to do the same.
At this point, each participant is given a voting card - a bright coloured piece of cardboard, about the size of a playing card. These are used when voting in the final stage (easier to count than hands that are half-up), but are also used to indicate agreement with what the current speaker is saying.
This stage usually takes between one and two minutes per person, or 15 to 20 minutes for a group of 12 people.
After all participants have done their introductions, the moderator begins the main discussion, by outlining the issues that the discussion will cover. This should be done very broadly, so that issues will be placed in their context. For example, if the focus of study is to be a radio station's programs, it is also worthwhile to gather opinions on other media which compete for people's time, such as television and newspapers.
If the participants don't already know which organization is conducting the research, it may be best not to tell them just yet, so that their opinions will be more unbiased. In some cultures (much of Asia, for example), when participants know who the research is for, they may be reluctant to criticize it in the presence of people from that organization. But even if they're not told the organization's name at the beginning, they'll usually work it out later, so the early parts are the only opportunity to hear unbiased opinions.
When participant take turns to speak, this ensures that everybody has a say, but doing this for an entire session can make conversation very awkward. My preference is to begin and end the discussion phase by asking each participant in turn to say something about the topic. For the rest of the discussion phase, anybody can speak, as they wish. Sometimes the moderator should intervene to prevent one or two people from dominating the discussion, or to encourage the shyer participants to have their say.
The organizers should not be drawn into the discussion. If a participant asks the moderator "what do you think of this matter?" the moderator should answer "it's your opinions that are important here, not mine. I don't want to influence you."
The purpose of the meeting is for the listeners to provide information to the organization which is conducting the research. But sometimes participants try to reverse the flow of information, and begin questioning the organizers in detail. If this gets out of hand, much of the meeting time can be wasted. The moderator should handle such questions by stating that there will be a "question time" at the end of the meeting.
The discussion itself can be either structured or unstructured (the difference is explained below), and will typically run for between 30 minutes and 2 hours. The duration can usually be controlled by the moderator. While discussion takes place, the secretary notes down statements that most of the participants seem to agree with ( e.g.if several are waving their voting cards). These statements are usually not written on a poster at this stage, but on a sheet of paper for the secretary's later use. We've found that writing statements on a poster too soon seems to stop participants from thinking further about a topic. If anything is written on a poster at this stage, it should be a question or a topic, not a conclusion.
An unstructured discussion is one in which the moderator merely introduces the broad topic, and sits back to let everybody else talk. The advantage of this approach is that participants feel unfettered by restrictions, and may provide useful and unexpected insights. The disadvantage is that much of the discussion may be irrelevant, and will not provide useful information. Therefore it is normal for the moderator to interrupt when the discussion drifts away from the stated topic (for example, if the stated topic is radio, the discussion will often drift onto television).
If the organizers have a list of questions they want answered, an unstructured discussion will often cover most topics, without the moderator having to introduce the topics one at a time. The moderator can have a list of topics, and check them off as they are covered. Towards the end of the discussion period, the moderator can ask specifically about topics that have not been discussed.
With a structured discussion, the moderator introduces issues one at a time, and asks the participants to discuss them. The moderator should avoid asking the participants any questions which can be answered Yes or No. (If this is the type of information you need, you should be doing a survey.) Instead, the moderator should say things like:
Both the above questions are the type that seek detailed responses, and are loose enough to allow for unexpected answers. If the questions asked are too specific, you risk not finding out the key facts.
Another type of structured discussion is one that involves playing excerpts of programs from a prepared tape. Reactions to specific programs are often more useful than generalized comment. For this purpose, we usually prepare tapes of 10 to 20 short program extracts, each around one minute - just long enough to illustrate a particular point, or jog the memories of people who are not sure whether they have heard or seen a program before.
Play one item at a time, then let people talk about it for a few minutes.
Many participants in these groups have a tendency to vagueness, making comments such as "I'm not very impressed with the news on FM99." Such statements are not useful, and need to be followed up with questions from the moderator ("What are the things that you don't like about the news?") or asking them to recall some specific examples. Doing this a few times near the beginning of each discussion will show the other participants what sort of comment is wanted.
The final section of a consensus group is to reach consensus on the issues discussed in the second phase of the session. By now, the secretary will have a list of statements which he or she thinks most participants will agree with. The secretary copies each statement onto a poster, and the moderator asks participants to vote on it, e.g. by raising their cards. We regard it as consensus if at least 70% of participants agree with it (i.e. at least 9 out of 12 people). If only a few people do not agree with a statement, they are asked why. "Could you agree with it if a few words were changed?" the moderator will ask. Quite often, this is possible. For example, if a statement expressed in an extreme way is softened a little, more people will agree with it. For example, this may strike some people as extreme:
Not many participants might agree. A few more might agree that
And even more might agree that:
The goal of this type of research is to describe any consensus that exists: in other words, a statement phrased in such a way that the great majority of participants are able to agree with it. The first statement offered will usually need to be altered somewhat so that as many people as possible can agree with it.
When consensus has been found, the secretary writes the modified statement on the poster, together with the numbers who agree.
For about a quarter of the statements (on average) it is not possible to reach consensus, and the group will divide into two sections, with no common ground. In such a case, try to get two statements, each with as many as possible agreeing, and record both statements, with appropriate annotation.
After the secretary has finished going through the statements she or he noted down during the discussion stage, participants are asked to add statements that they feel most others will agree with. Each participant in turn can be asked to make a statement, which is then modified and voted on in the same way. Finally, the secretary asks the moderator to offer further statements for evaluation.
The success of consensus groups depends very much on the quality of these statements. There are some statements which most people will agree with, yet which offer no insight into a topic. For example, everybody may agree that the sky is blue - but how does that help? With experience, the moderator can steer the group towards statements that are on the border of being controversial. That is, statements with which most people agree, but they only just agree.
Imagine all possible statements being spread out, as if on a map. An irregular shape on this map might define the statements with which most people agree. At the centre of the map are the statements which are so obvious that they are hardly worth stating. For example, all regular listeners to a radio station might agree with "I usually like what I hear when I listen to FM99." (Otherwise, they probably wouldn't listen.) Towards the edge of the irregular outline on the map are the borderline statements, at the boundaries of agreement of the station's listeners. An example of a borderline statement might be "Though I usually listen to FM99, I listen to other stations when I don't like the FM99 program." These borderline statements tend to be more interesting, and less obvious.
The consensus-seeking stage of the discussion will typically last between 30 minutes and one hour, depending on how many statements are offered for discussion. Sometimes a group will split into two factions, which hardly agree on anything. When that happens (which is rare) the consensus stage will take much longer.
You may wonder why discussing issues and reaching consensus are presented as two separate stages of the discussion. Would it not be more efficient to take each topic one at a time, reach consensus on that, then move on to the next topic? I have tried this, but found it impedes the flow of discussion. Also, returning to a topic at the consensus stage gives people more time to gather their thoughts, and consensus seems more easily reached after a time gap. The exception is when the discussion is structured, by being divided into a number of clear parts - for example when a lot of short program excerpts are being played to the participants, and they reach agreement on each one separately.
Sometimes the most difficult part of running a consensus group is persuading the participants to leave at the end of it. With some groups, nobody wants to go home, and most participants may sit around and talk for an hour or two longer. I find these late sessions very useful. By that time, the participants know each other (and the organizers) much better and may offer insights that they did not mention in the formal session. It is thus best not to turn the tape recorder off too early.
The outcome of each group is one or more posters of statements, together with numbers showing how many people agreed with each statement.
After three group sessions have been held, you will have three sets of statements. The reason for holding three groups is that one group may develop its thoughts in quite a strange way, perhaps due to one or two powerful personalities. With two groups, if they produce quite different findings, you won't know if one group is atypical. But with three groups, if the results from one are very different from the other two, this will be obvious.
Though three groups never come up with exactly the same set of statements, we have always found strong similarities. If at least two of the three groups came up with similar statements, these statements can be safely assumed to be representative of the whole audience sampled. No matter how differently the three groups are selected, we usually find a lot of agreement between the lists of statements when the discussions have been conducted in the same way. Observing the similarities will give you confidence that the results are true of the entire population studied, not only of the three disparate sampling points.
Have the three lists of statements typed out, and this will be the basis of a report resulting from the discussions. Add a description of any taped stimulus material, and the criteria used for selecting listeners, complete a survey summary form (as in Appendix 7) and that may be sufficient. If you have taped the discussions (whether on audio or videotape) you can extract relevant comments to illustrate each of the agreed statements, and include these in a more detailed report.
In summary, the consensus group technique is one which can be used by inexperienced researchers with reasonable safety. It is more difficult to draw wrong conclusions with this technique than with other types of qualitative research, and the findings are often more directly useful than those from formal surveys.
However the technique is not a simplistic one; it can be used even by highly experienced researchers. As you become more experienced at conducting consensus groups...
An extension of the consensus group to a larger situation is covered in Chapter 14: the co-discovery conference, which is based on similar principles, but involves more people in total (though fewer per group), and takes more time.