Not just another focus group: making the case for the nominal group technique in criminology

Abstract

Background

The Nominal Group Technique (NGT) is seldom applied in criminological research. This article explores the potential of NGT as a tool for criminological research.

Methods

NGT is a highly structured technique combining characteristics of an individual survey and a focus group.

Results

It offers various benefits: 1) it limits researcher influence and influence from group dynamics; 2) increases the likelihood of equal participation for all group members; 3) affords equal influence to (conflicting) values and ideas; 4) can be used in an exploratory (phase of a) study as well as to generate hypotheses about topics that are unfamiliar to the researcher; and 5) is useful for determining the ideas of a research population that is socially or culturally different from that of the researcher.

Conclusions

NGT is particularly relevant in applied research as a decision-making tool and as a consensus method. It also holds promise as an online tool for criminological research focused on sensitive topics where participants take part anonymously.

Introduction: group techniques in social sciences and criminology

In qualitative research, there are various reasons to use group techniques – e.g. the practice of allowing individual subjects to participate together as a group – to collect data. Interactions that take place using group techniques add depth to the obtained information and allow shared meanings to emerge, which is not possible in individual interviews (Macphail, 2001). Group techniques also tend to stimulate the expression of new, bias-free ideas. They have shown to be particularly effective in research on vulnerable groups and young people. By allowing people to participate as a group, their influence is increased. Group techniques also counterbalance the direct impact of the researcher on the discussions and partially neutralises the power differences between participants and decision-makers or professionals (Wilkinson, 1999; Madriz, 2003). Group techniques allow access to research participants who may find one-to-one interaction frightening or intimidating. Such participants tend to be more willing to talk to peers than researchers about sensitive topics (Large & Beheshti, 2001; Madriz, 2003).

The most well-known and frequently-applied group technique is the focus group. ^aSince the end of the 1980’s, focus groups have become common in the social sciences. Focus groups involve a small number of participants discussing topics raised by a moderator, who guides the interview process. The method is based on group dynamics; interaction between group members is the defining characteristic. This interaction distinguishes a focus group from a group interview (where the focus is on the interaction between the researcher and the participants, MacDougall & Fudge, 2001). Group discussions tend to produce lively and timely images of the motives, interests, questions and ideas of the participants involved (Merton et al. 1990). Focus group research is regularly applied in qualitative criminological research. For example, it is used to study the ideas and perceptions of professionals working in the criminological field (see, among others, Douglas & Cuskelly, 2012; Kaye et al. 2014), ideas and perceptions of the general public and young people on crime and the justice system (Boda & Szabo, 2011; Dirikx et al. 2012), and perceptions of professionals on sensitive topics such as police peer retaliation (Cancino & Enriquez, 2004) and the use of force by the police (Klukkert et al. 2009). It has been regularly applied to study sensitive topics such as abuse, violence, drug use and sex work and the populations – victims, offenders and prisoners – involved (see among others Wachholz & Miedema, 2000; Surratt et al. 2004; Garcia & Lane 2012).

However, group techniques – and focus groups in particular – have disadvantages as well. The most frequently mentioned disadvantages are associated with group dynamics. Group discussions may be dominated by one or more individuals (Macphail, 2001). Participants may feel pressured to conform with peers and/or dominant individuals or ideas in the group. The group dynamic may influence the attitudes of participating individuals (Bristol & Fern, 2003). Groups can impede individual reactions resulting in ‘groupthink’. In most group techniques, it is difficult to guarantee equal participation among all group members (Macphail, 2001). Moreover, the central role of the researcher can fundamentally influence a group discussion (Campbell, 2002).

In sum, the epistemological value of data generated through methods involving group dynamics notwithstanding, these dynamics pose a challenge for researchers. In this respect, it is worthwhile to discuss the Nominal Group Technique (NGT) in detail. ^bNGT is a highly structured group technique that uses face-to-face meetings. NGT combines both individual and group phases. The purpose of the structure and individual phases is to limit group dynamics and dynamics of social power. The technique prevents dominant people from controlling the group and limits the interplay of the researcher in the generation of ideas (Aspinal et al., 2006). Overall, it is aimed at giving each person an equal opportunity to generate ideas, speak and vote (Macphail, 2001). Group discussion and group interaction carry equal weight since individual judgements are combined. NGT proceeds along a fixed format consisting of four phases, with individual and group phases. ^cThe individual stages enhance the opportunity for all participants to contribute ideas and influence decision-making equally because each person has the same power in voting. It limits the influence of dominant, more confident or outspoken group members, or those with perceived high status (Gallagher et al., 1993; Gaskin, 2003; Jones, 2004). The individual stages also enhance the opportunity for minority voices and votes to become part of the generated data. Moreover, the ideas that are proposed, discussed and voted on all come from the participants and not from the researcher, thus limiting the influence of the researcher. Apart from NGT’s individual phases, the whole process takes place in a group setting, which allows access to research participants who may find one-to-one interaction frightening or intimidating.

NGT has been regularly applied in research on education and health (services). ^dFor example, it is used in needs assessment research (Keatinge et al., 2000) to study unmet needs (Martinez & Carter-Pokras, 2006; Drennan et al., 2007), among other things. In criminological research, NGT is rarely ever applied. ^eWe encountered very few publications that applied NGT, even within the broader field of criminology. In 1973, Zastrow used NGT to study young peoples’ and professionals’ goals and priorities for curbing youth delinquency. In (Jordan 1992) studied alternative policing strategies in the U.S. for meeting the needs of urban communities by the year 2001 with the help of a panel of law enforcement executives and community leaders. In a 2007 U.S. study (Government Accountability Office GAO 2007), an expert panel discussed challenges and alternative strategies for monitoring and evaluating the results of international projects fighting human trafficking. NGT has been used as a group problem-solving technique by community police officers to establish a relationship between themselves and community members and to assist the community in developing action plans to confront its problems (Wiatrowski & Campoverde, 1996). In drug research, NGT has been applied in a piece of participatory research concerning harm reduction treatment with clients and to develop culturally relevant outcomes to measure progress (Ruefli & Rogers, 2004). Among these, only two studies can be called recent. One applied NGT in a group of hotel security experts to reach consensus on a baseline anti-terrorist strategy for a hotel (Paraskevas, 2013). Another used NGT in an exploratory study of young people on the role of software piracy in decisions to adopt video game consoles (Kartas & Goode, 2012). For criminological research, NGT could potentially be used for research aiming to focus research on the unique perspective of a particular research population. NGT presupposes that the members of such a population are considered subjects and not research objects. At the same time, it is recognized that no ‘right’ or ‘true’ or general meaning exists (Weber, 2003). For research based on other theoretical approaches, NGT can be particularly relevant in applied research as a decision-making tool and as a consensus method.

While focus groups are used widely in criminological research, NGT has been mostly neglected. In what follows, we will discuss and reflect on the strengths, weaknesses and potential applications of NGT. We also give an example of NGT in criminological research by reporting on the use of NGT in a study on the needs assessment of drug (prevention) policy (Vander Laenen, 2011). Since NGT is scarcely used in criminological research, it is worth describing the sampling, format and process of decision-making and the data analysis.

The nominal group technique: a brief manual

Preparation

NGT is a single-purpose technique (Delbecq et al. 1975) and only a limited number of topics and issues can be covered. The question posed at the start of the NG meeting itself is critical. It will determine the quality of the generated ideas. NGT’s success depends on the unambiguous formulation of the question (Tuffrey-Wijne et al., 2007). A research question has to be concrete, in a language that is appropriate for the participants, and sufficiently refined. As a rule of thumb, avoid asking too general a question. For instance, do not ask ‘what administrative problems do you expect in the development of an integrated drug policy?’ because respondents will resort to generalities such as ‘poor communication’ or ‘insufficient motivation’. It is more useful to ask the respondents to list critical incidents, examples or descriptions of behaviour that illustrate the problems they expect (Delbecq et al. 1975). This will promote detailed and realistic responses. To this end, guidelines may be added to the central research question. Still, the question should be sufficiently open-ended for participants with novel solutions. Consequently, sufficient time and effort must be invested in developing the research question. The framing of the starting question is determined by the goal of the NG meetings. It can be developed by the research team or it can be developed in earlier stages of research. In view of its importance, composing and pilot testing more than one question is advised.

The researchers need to bring a sufficient number of flash cards (for the participants to write down their individual ideas during phase 1), a flip-chart (for the researcher to write down the individual ideas during phase 2), two different colored markers (one to write down their ideas, one to tabulate the scores), 3) sufficient pens for the participants; 4) and cardboard for the participants to write down their names. Next, the table should ideally be arranged in a u-shape so that the participants can all see the flip-chart. Before executing a NG meeting, it is also important to develop a road map to increase the structure of the meeting (to be used by the panel chairman) as well as an observation schedule to write down additional information (for the observer). Reflections of the participants generated throughout the NG meeting are written down in the observation schedule. The wording and the expressions used by the participants are also written down. Discussions or disagreements on some of the ideas and the reflections of the participants on the priority list should receive specific attention.

Sampling

Sampling for qualitative methods differs from sampling for quantitative methods. In the former case, samples are purposive rather than random; they aim to select cases that will provide rich data (MacDougall & Fudge, 2001). The same holds for sampling in a NG. Depending on the type of research, NGT participants will vary. They can be professional experts or any group of people who have expertise in a given subject. This is based on the notion that anyone whose interests are affected by a problem are experts regardless of whether they are professionals (Aspinal et al., 2006). The number of participants for a NGT varies from study to study, ranging between six to twelve participants (Gaskin, 2003). Delbecq et al. (1975) strongly suggest limiting the number of participants to between five and nine. For young people, it is advisable to limit the number of participants to six or seven (Heary & Hennessy, 2002). The number of NG’s to be organised will depend on the type of research. There are no clear guidelines on the number of NG’s to reach saturation, e.g. the point at which no new information or themes are observed in the data. It is clear though that for a comprehensive assessment of a problem area to emerge “several NGT sessions should be conducted, using target groups whose experience directly relates to the problem area, but from different perspectives” (Delbecq et al. 1975, p. 113). In studies, the number of NG’s varies between a single or a few NG meetings up to ten NG meetings (Ramirez et al., 2000; Aspinal et al., 2006) and more (Vander Laenen, 2009).

The traditional NGT procedure

NGT proceeds along a format consisting of four phases: the silent generation of ideas in writing; round-robin feedback from group members to record each idea; discussion of each recorded idea for clarification and evaluation, and individual voting on priority ideas. At the start of the NG, the purpose and the procedure of the meeting are explained. The researcher can introduce the participants or they can introduce themselves. The ground rules for the interaction and the role of the panel chairman and of the observer are explained. During phase one, the researcher states an open-ended question without allowing discussion and he hands out flash cards to each participant. Each participant spends several minutes in silence individually brainstorming all the possible ideas and writing them down using a brief sentence or some words. During phase two, the ideas are collected by the researcher by sharing them round-robin fashion (one response per person each time), while all ideas/items are recorded on a flipchart using a key term or a short sentence. No comments from participants are allowed, but clarification in response to questions is encouraged. The round-robin technique is used to limit the ownership of the ideas since the group members discuss the individual ideas and the individual who wrote down the idea does not have to clarify nor ‘defend’ it. The items written on the flip chart use the wording of the participants. During this phase, participants are allowed to write down new ideas, and present them when his/her turn occurs in the round-robin (Gallagher et al., 1993). The third phase is aimed at discussing each recorded idea one-by-one for clarification in the group. Similar ideas are joined and identical ideas are discarded. The role of the panel chairman is to ensure that every participant gets a fair chance of clarifying an idea (this can be his own idea or the idea of another participant). In this phase, the role of the chairman is similar to the role of the moderator during a focus group. During phase four, each participant evaluates the ideas and individually and anonymously votes for the best ideas. Variations to the voting system are applied in research (Frankel, 1987), e.g. the best idea gets five points, the next-best four, and so on. Each participant can score each idea from one to five; each participant can freely attribute six points to the ideas, etc. The votes are then shared in the group and tabulated so it becomes visually clear which ideas received the most points and are prioritized. It is possible to add a fifth phase when consensus is required. To this end, after the results from the vote in phase four have become clear, a discussion in the group of the preliminary vote is held and a re-vote takes place. A NGT lasts on average between 60 and 75 minutes. Some authors who use NGT to reach consensus say that two hours may be necessary (Gaskin, 2003).

The modified NGT

In the literature, several authors have reported modifications to NGT since its introduction by Delbecq et al. NGT is regularly used in combination with other qualitative (group) techniques, such as interviews and focus groups. NGT has been applied as part of a focus group (Gaber & Gaber, 2002; Rosemann & Vessey, 2008) and NGT’s and focus groups have been used as separate techniques within a single study (Ramirez et al., 2000; Ginsburg et al., 2002). Other variations are possible. In some NG’s evidence is used alongside expert opinion before voting (Kapiriri et al. 2003; Ayuso-Mateos et al., 2013). Aspinal et al. (2006) prepared a meeting by identifying issues and ranking them before attending the NG. It is also possible to allow more discussion during phase three of the NG, consistent with a focus group format (Fein et al., 1997).

In recent years, online applications of group techniques are on the rise. These can take the form of real-time synchronous or asynchronous applications. In the synchronous form, participants are online simultaneously at a prearranged time and group discussion is interactive and in real time. The asynchronous form refers to a website where participants log in during a set research period (for instance one week) and participate at a time that is convenient for them without live interaction taking place between the participants (Tates et al., 2009). As is the case for traditional real life group discussions, online group techniques have been applied as a standalone technique. They are applied following qualitative interviews to identify expert opinions. In these examples, NGT is used for the voting on these opinions (Lossius et al., 2013). In a study we are currently executing, ^f39 harm reduction strategies were identified during interviews with professionals and during focus groups with problem drug users. These strategies are then voted on by 120 professionals via a modified asynchronous online NGT and by 30 problems drug users via paper.

Data-analysis

NGT combines quantitative and qualitative data collection methods and also yields data that can be analysed both quantitatively and qualitatively. Qualitatively, a content analysis of the individual items can be carried out. This enables themes to be constructed out of the individual items. Moreover, the ‘discussion’ sections of the procedure can be analysed by means of coding and using standard Computer Aided Qualitative Data Analysis software. Note that this necessitates the use of recording equipment, which is not always allowed in some settings (e.g. in prison) or may be refused by some participants.

NGT enables data to be generated in such a way as to limit the amount of analysis required after the group session (Aspinal et al., 2006). The data processing and analyses of NGT results is relatively simple. After a NGT, the researcher has already compiled an overview of the ideas for each group, the ideas with the highest score and the number of participants that scored a specific idea. In the event that different NG’s respond to the same research question, further analysis is needed, as each group generates different lists of items and these are difficult to compare. Some studies collate all group scores to give an overall priority of items from the different groups (Dening et al. 2012). Other studies have developed a standard set of priority items derived from all the group meetings that can be voted on by all the participants from the NG’s (Vander Laenen, 2009) or by larger groups of participants (Tuffrey-Wijne et al., 2007). Quantitatively, it is possible to compare results across individuals belonging to several categories, e.g. based on background characteristics such as age, gender and ethnicity, provided that participants complete a questionnaire (Ramirez et al., 2000; Ginsburg et al., 2002).

A case study: NGT in drug research

Study design: rationale for using NGT

The study comprised a needs assessment of drug (prevention) policy. The aim of the study was to ask young people with emotional and behavioural disorders what they would wish to see in a drug prevention policy. (Vander Laenen, 2009; Vander Laenen, 2011). We opted for a qualitative mixed-method research design combining focus groups, NGT, a survey and feedback sessions. The design allowed the research subjects to participate actively. The research consisted of four parts that involved the young people both in and after each parts (see Table 1). ^gPart one was aimed at developing the research question to be used during part two of the study. To this end, open focus groups were used. During part two, the participants generated and prioritised ideas using the NGT. Because participants in each group generated and evaluated different ideas, it was necessary to evaluate the ideas in a standardized fashion. Thus, a survey was developed and administered. In part three of the study, participants clarified the most important ideas from part two and advised on how they wanted these ideas to be implemented. For part three, focus groups were used. During part four of the study, the researcher returned to the field with the results during feedback sessions. The research subjects were 160 young people aged 12 to 21 with emotional and behavioural disorders (EBD) who lived in a residential setting within the disability system in the province of Eastern Flanders, Belgium (D’Oosterlinck et al., 2006).

Table 1 Study design and data collection

The participants were asked for oral informed consent. Their parents were asked for passive informed consent. Participants were paid for their participation, and their anonymity and confidentiality were guaranteed. No staff members were present during the groups (Balen et al. 2006).

Preparation

The starting question was developed in the first part of the study during open focus groups. During these focus groups, the researcher asked the young people one broad question – ‘What should good drug prevention be like?’ – in order to develop the central youth-centred research question. The analysis of the focus groups showed that the concept of ‘drug prevention’ could not be used for the central question and that harm minimization and care should be included alongside prevention. These findings led to a three-fold central research question for the NGT, which was worded as follows: “How can we make sure that young people do not use drugs? How can we make sure that young people who do use drugs do not get into trouble? What should be done to better help young people who get into trouble with drugs anyhow?”. Two definitions were added to this question (one on ‘drugs’ and one on ‘getting into trouble’). Next, two general guidelines were added. The first guideline stressed that the researchers were looking for the ideas the participant himself found most important; the second guideline stressed the importance of generating achievable, concrete ideas. The starting question (and the two definitions and guidelines) were pilot tested in one NG meeting. Since the participants did not know whether they should respond to each of the three elements of the question, an additional guideline was added to indicate that this was indeed expected. For the NGT a road map, describing in detail the process of the NGT and an observation schedule for the observer was developed^h.

The NGT procedure

In our study, the NGT procedure was mildly modified to accommodate the research population. In phase one (the idea-generation phase), some participants where assisted by the researcher in writing down their ideas, i.e. the participants formulated the idea and the researcher wrote it down for them. In phase two (the round-robin feedback), the researcher read each idea aloud to the group in order to further limit the ownership of the ideas and to avoid embarrassment for participants with limited reading skills. In phase four, the researcher collected all individual votes and shuffled them to guarantee maximal anonymity. During this phase, the researcher asked participants to assist her in tabulating the votes in order to increase the contest element of the technique and to allow participants to expend excess energy. In the study, reaching consensus was not a goal and so the procedure ended when the votes were tabulated. Moreover, the results of the NGT were not the end of the study since the participants added meaning to the results and looked for practically workable solutions to the identified problems during a separate research phase, using focus groups. Two researchers, a panel chairman and an observer executed the NGT. The NGT meetings lasted on average 54 minutes.

Data collection and analysis

The study was executed in seven different settings; in each setting, two NG meetings were organised. The results of the NG meetings were analysed per age group by comparing the different ideas to look for recurring ideas. After four-to-five NG meetings, no new ideas were added to the list of ideas discussed during previous NG meetings. Still, to honour the engagement of the residential settings in the study and out of respect for the participants, each of the parts of the study (see Table 1) was executed in each of the settings. Consequently, seven NG meetings per age group were conducted.

To make full use of the rich information provided by the individual items and to gain insight into the dominant discourse as well as the minority voices and votes, ⁱeach item was analysed and the analysis was not limited to the items with the highest score. To this end, the “drug, set, setting model” of Zinberg (1984) was used to identify substance-orientated, person-orientated and environment-orientated prevention strategies. This model was supplemented with strategies for early intervention and for counselling/guidance. Each of the individual items was attributed to one of the strategies in the Zinberg model by two researchers. This resulted in an excel file consisting of the number of ideas within the Zinberg-model (in general, for the two age group, for male and female participants). This analysis showed a difference between the results from the open focus groups in the first part of the study and the results from the NGT. For example, girls more frequently proposed and prioritized person-orientated prevention strategies while boys proposed and prioritized more substance-orientated strategies, a difference that was not found during the open focus groups. Next, during the focus groups with the participants aged 16 to 21, the dominant discourse was that drug use and in particular cannabis use could not be avoided, while during the individual stages of each NG meeting, one or more participants proposed prevention strategies that were aimed at preventing any type of illicit drug use. The individual generation of these ideas led to a discussion in the focus groups during part three of our study on prevention strategies that were proposed by only one (or a few) participants during the NGT. Next to the analysis of the individual ideas, the items with the highest top-five and top-three scores were analysed using the Zinberg model to look for differences and similarities between the different NG meetings, the two age groups and the boys and girls. Table 2 provides an overview of the items in the top five.

Table 2 Number of top-5 items and scores divided by age group

Each group gave priority scores only for those solutions generated within their own group. In order to have the population generate a standard prioritization, a survey was developed that included the three ideas with the highest score from each nominal group (NG) meeting. Initially, this resulted in a priority list of 21 items. To optimize the diversity of the survey in line with the Zinberg model, in some cases an idea from a NG session with a low score was added to the priority list. This also allowed more adolescents to be exposed to the creative response. For instance, no idea on early intervention was included in the priority list despite its proven effectiveness, and earlier research stressed the importance of trust for young people in communication. Therefore, the item ‘talk about it with people that can be trusted’ was added. Since the NGT was executed in two age groups, two separate priority lists were developed. The ideas were written as the participants had worded them and placed in random order. Two versions of the surveys were used, listing the ideas in different orders to decrease the likelihood that the order of presentation would affect the responses (Ramirez et al., 2000). The participants in the NGT meetings were asked to rate how each item would prevent (problem) drug use or help problem drug users on a five-point Likert scale. The surveys were self-administered after standard instructions were read.

Hence, after the NG part of the study we effectively knew what the priority items were for the research population. We did not know how the population clarified the items, nor how participants wanted the priorities to be put into practice (Rich & Ginsburg, 1999). It was possible to mediate this shortcoming by modifying the NGT and by including a focus group discussion at the end of a NGT meeting. However, this is only feasible when the research population is able to stay focused for at least two hours or more, which was not the case for our participants. Moreover, this would imply that the participants could only discuss the priorities of their own group and not the results of other groups or survey results. Therefore, we organised focus groups as a separate research phase to add sufficient context and meaning to the prioritised items. The surplus value of using a NGT preceding focus groups was that a diversity of the prevention strategies was discussed during the focus groups rather than simply the strategies that were part of the dominant discourse.

Table 3 Strengths and weaknesses of NGT

NGT methodology: strengths and weaknesses (Table 3)

Strengths of NGT

The primary advantage of NGT over other group techniques is the enhanced opportunity for all participants to contribute ideas and influence decision-making. It limits the influence of dominant, more confident or outspoken group members as well as group members with perceived high status (Gallagher et al., 1993; Gaskin, 2003; Jones, 2004). Focus groups can reflect the perceptions of the most outspoken or opinionated members and can focus on what the group considers most interesting or controversial but not necessarily most important (Fein et al., 1997). A greater number of ideas are generated in NGT’s than in other group processes. Van de Ven and Delbecq (1974) compared the conventional interacting group with NGT in terms of the quantity of ideas generated. They found that NGT was more effective than conventional interaction groups (de Ruyter (1996) and Eisele (2007) found the same results). Because of the individual phases in the process, each idea has equal opportunity to be put on the agenda and each person has the same power in voting. Shedlin & Schreiber (1995) and Tuffrey-Wijne et al. (2007) state that the experience of giving and ranking ideas may be particularly empowering for people who have been traditionally excluded from participating in research. Furthermore, NGT’s structure limits the influence of the researcher (Macphail, 2001). The participants generate and score the priorities. Tellingly, the term participant-led priorities is used in the literature. The role of the researcher is limited; he/she follows a series of pre-stated steps, whereby the procedure will not significantly differ between the different NG meetings. NGT is highly structured, with clearly defined phases and goals. For research with children and young people, the use of a more structured technique rather than more open-ended methods such as focus groups helps to create a non-threatening environment for the participants (Porter, 2012). However, it can also be a disadvantage because participants need to feel comfortable with – and remain within – this structured group process (Jones, 2004). NGT is a technique that is positively evaluated by participants. Some authors state that it gives a greater sense of accomplishment and satisfaction for the participants than focus groups (Jones, 2004). Next, the NG meeting is often one of the first experiences where verbal glibness is not the overriding base of power (Delbecq et al. 1975; Ruefli & Rogers, 2004), which is particularly satisfying for (young) people who experience difficulties expressing themselves. In the case study described above, the visibility of the results of the NG indeed proved rewarding for the participants; their priority list was literally hanging on the wall. Zastrow (1973), p. 111 called this the ‘game mystique’. At the end of a focus group, results can be overly abstract and are not visible. It is relatively easy to interpret the results of NGT meetings, as ideas are generated and voted on/ranked during the session itself. In focus groups, the recorded discussions have to be transcribed ad verbatim and analysed, which is a time-consuming and skilled exercise (Gallagher et al., 1993). However, in order to avoid missing too much rich information, further in-depth analysis of the individual ideas and results of a NG is advisable, so that the time required to analyse NGs is comparable to the time invested in focus group data analysis.

An advantage of online applications of NGT and of group discussions in general is that they offer anonymity, which has been found to stimulate disclosure in sensitive topics (Tates et al., 2009, Stover & Goodman, 2012). Importantly, the online form can enable researchers to gain access to groups that are difficult to engage via others research methods, in particular young people and vulnerable groups (Stewart & Williams, 2005; Yu et al. 2011; Thomas et al. 2013). For researchers, the online application saves (travel) expenses and time for transcribing data since the data are already in writing (Tates et al., 2009). The asynchronicity of online groups increases the flexibility and convenience of logging in and is particularly advantageous for reaching people with a busy schedule (Stewart & Williams, 2005). Next, the asynchronous format gives participants ample time for reflection and allows them to respond at length and change or nuance their opinion (Tates et al., 2009). That being said, in our study on harm reduction strategies we experienced that the response to the open-ended questions was rather limited.

Weaknesses of NGT

As a sampling technique, NGT shares a disadvantage characteristic of all qualitative group techniques: as a rule, purposive sampling is used. As in focus groups, the results of the NGT cannot be generalized to a wider population due to the limited number of participants. Notwithstanding the possibility to test the results of a NGT by using a survey with a representative sample, the initial selection of the NG participants is a key factor and is open to bias due to the specific characteristics of the participants, both in terms of who is selected to attend (a possible selection bias), and who agrees to participate (a potential response bias) (van Teijlingen et al., 2006). In research with professionals aimed at consensus, the level of expertise is crucial to its success and the validity of the data it generates (Harvey & Holmes, 2012). Contrary to focus groups, NGT is a single-purpose technique (Delbecq et al. 1975) and only a limited number of topics and issues can be covered. The starting question in itself is a critical and potentially weak element. NGT’s discussion stages do not guarantee anonymity, which may limit participants’ willingness to express their views, especially on more controversial issues (Fein et al. 1997; Van Teijlingen et al. 2006). Even structured group processes may be influenced by powerful social dynamics, especially in adolescent populations (Ramirez et al., 2000). NGT can be used with participants with limited writing skills on the condition that the researcher assists the participants during individual phases of the procedure. However, to participate, a certain level of verbal ability and a capacity to express ideas is essential. NGT therefore is not suitable for people with severe and profound intellectual disabilities; for this group, for instance asking simple open questions and using communication aids such as symbol cards can facilitate research participation (Beail & Williams, 2014). A major disadvantage of NGT in a qualitative research design is that is does not add sufficient depth to the research results. The requirements for a moderator in a NGT meeting are less demanding than those for a moderator in a focus group meeting since the role of the researcher in a NGT is more limited. However, in order to create a safe and structured environment within the group, ensure that all participants are able to express their views and keep particular personal or professional views from dominating the discussion, a skilled and experienced group facilitator is still essential (Gallagher et al., 1993; Tuffrey-Wijne et al., 2007). Furthermore, the potential for researcher bias remains, particularly when the results are further analysed and a survey with additional questions is developed from the NGT results. During a NGT, ideally two researchers should be present to support participants during the individual phases in reading and writing. One researcher can pay particular attention to individual needs of participants while the other researcher can continue the process with the group. In fact, two researchers are always advisable in group techniques, because much (non)verbal communication escapes the attention of a single moderator (Vander Laenen, 2009).

An important limitation of online applications of NGT – in particular when studying marginalised or older populations – is that limited computer access or computer illiteracy may restrict participation. This is why in our study on harm reduction strategies we had to print out the priority list for the problem drug users. Both synchronous and asynchronous group discussions have the disadvantage that they lack nonverbal cues. A consequence of this is a heightened potential for misinterpretation of the written communication, which might negatively impact the group dynamic. Moreover, the lack of a real-time group dynamic has been criticized, especially in the case of focus groups since interaction is key (Tates et al., 2009). A disadvantage specifically linked to synchronous group discussion is that the discussion can be dominated by the fastest-typing participant (Stover & Goodman, 2012).

Developments and potential applications

Online applications

Online group techniques are seldom applied in criminological research. An exception to this is the well-known study by Williams (2007) who conducted a virtual focus group with online community members to map the maturation of regulation within an online social setting. It is clear that the rise of (organised) cybercrime will only increase the necessity of the online applications of research techniques if we want to get into contact with respondents from this online community for research purposes. In view of the adaptability of NGT, its use in an online form could be applied in different research designs. Next, online applications are interesting to apply in criminological research on sensitive topics where participants are anonymous. It would allow the study of mixed groups, as we did in our study on harm reduction (with professionals from the police, justice department, treatment sector, local government and problem drug users all prioritizing the same ideas). Why not use an online NGT to study procedural justice, for instance, with both the police and offenders or victims and offenders participating in one NG meeting?

Exploratory research

Taking the characteristics of NGT into account, NGT can be used in an exploratory (phase of a) study as well to generate hypotheses about topics that are relatively unfamiliar to the researcher or to uncover ideas that are of importance to a research population socially and culturally different from the researcher. It can also be used to generate content information about a topic unknown to the researcher (Gallagher et al., 1993).

Modifying and mixing methods

A NGT can be used simply as a technique to increase external validity. Although NGT is a qualitative research method, it is also possible to let a representative sample validate the results. This allows for subgroup analyses and the possibility of studying correlations. We want to stress the potential of the NG as a technique that merits further application in particular as part of a mixed-method research design, either as part of a design combining qualitative methods or as part of a design combining qualitative and quantitative methods. We concur with Boda and Szabo (2011) that qualitative and quantitative methods should be viewed as complementary rather than as rival camps given the strengths and weaknesses found in single-method designs (e.g. the lack of key contextual factors regarding why priorities exist in quantitative survey and the selection bias limiting the generalizability of results for purposive sampling, Weathers et al., 2011). Boda and Szabo (2011), p. 339 make a compelling argument for this in their study on the role of the media in shaping attitudes towards crime and the justice system. They state: “We should look for more sophisticated theories to explain the formation of public opinion and the role of the media in it, and, in addition to the quantitative analysis of statistical data, we should move towards audience research and the use of more qualitative methods”.

Applied research

NGT can also be used as a standalone method. It is particularly relevant in applied research as a decision-making tool and as a consensus method. Its purpose is to gain insight into the problems or issues of importance and identify solutions for these problems and issues. For instance, it can be used to identify areas for hot-spot policing or to study alternative strategies for family violence or radicalisation. If the goal of the technique is to reach consensus, and consensus cannot be reached, the method’s group phases allow researchers to gain insight into differing perspectives, the elements of the topic where (no) consensus can be reached and why this is the case. As a collaborative process, which gives all participants a(n equal) voice, NGT holds great potential for actively involving vulnerable populations in gathering and prioritizing their ideas. It can be used to identify critical problem dimensions in hard-to-reach or hard-to-study populations. It was successfully used in health research where it has been applied to study different cultures (Martinez & Carter-Pokras, 2006; Malpede et al., 2007) and to study young people (Ginsburg et al., 2002). It can be used to involve multidisciplinary and multi-agency experts in particular to increase the likelihood of equal representation for all group members (Harvey & Holmes, 2012). This is the case when there is a (perceived) power disparity among participating experts, which is the case, for instance, for participants from health and social services and participants from the criminal justice system (Vander Laenen, 2014). As an established technique for citizen participation, it can be used to involve citizens in (action) research,research with local communities on prevention strategies for crime and disorder (Junger et al., 2012) or on legitimacy. The study by Wiatrowski and Campoverde (1996), who used NGT as a group problem-solving technique applied by community police officers to establish a relationship between themselves and community members and to assist the community in developing a plan of action to confront its problems, is a good example of this. Moreover, results from this type of research can offer guidance for criminal justice organizations as to how to best adapt strategies and training in a fast-changing and increasingly cosmopolitan world characterized by changing social values (Bottoms & Tankebe, 2012).

Conclusion

NGT is a highly structured technique combining characteristics of an individual survey and a focus group. Its structure limits researcher influence and influence from group dynamics. It increases the likelihood of equal participation for all group members and equal influence of (conflicting) values and ideas. NGT can be used in an exploratory (phase of a) study, can be used to generate hypotheses about topics which are relatively unfamiliar to the researcher, or to become familiar with the ideas found to be relevant to a research population that is socially and culturally different from the researcher. NGT is particularly relevant in applied research as a decision-making tool and as a consensus method. Its purpose is to gain insight into the problems or issues of importance and to identify solutions for these problems and issues. Since NGT limits the influence of group members with perceived high status, the technique is also valuable in research where participants have different levels of power.

Suggestions for further reading

Delbecq, A.L., Van de Ven, A.H., & Gustafson, D.H. (1975). Group techniques for program planning: a guide to nominal group technique and Delphi processes. Glenview, Illinois: Scott Foresman.

As the title suggests, it is a practical guide that explains in detail the NGT. With regard to the NGT, it includes a description of preparatory tasks, of the NGT process, its strengths and weaknesses and responses to frequently asked questions regarding the technique.

Merton, R.K., Fiske, M., & Kendall, P.L. (1990). The focussed interview, A manual of problems and procedures, Second Edition. New York: The Free Press.

The manual does not focus on NGT but it does provide vary useful information for researchers doing group interviews. In particular, it provides practical information on strategies for the moderator of a group to minimize undesirable forms of social interaction.

Endnotes

^aA basic WoS search of the term ‘focus group’ resulted in more than 12,000 hits.

^bNGT was developed in 1968 by Delbecq et al. and is derived from research in social psychology. NGT was originally developed as a technique to facilitate the involvement of disadvantaged citizens (Eisele, 2007).

^cRead on for a detailed discussion.

^dA basic WoS search of the term ‘Nominal Group Technique’ generated nearly 500 hits.

^eA basic WoS search of the term ‘nominal group’ under the category “criminology and penology” resulted in only one hit: a study by Zastrow (1973).

^fIn 2014 we used NGT as part of a three-stage mixed method design study to conduct a needs assessment study on harm reduction strategies in a local community (stage one is a needs assessment via interviews with multi-agency experts and focus groups with problem drug users; stage two is a two-part validation of the identified needs via a modified online NGT; stage three is a feasibility study via focus groups with professionals and drug users).

^gThe design was based on a design used by Ginsburg et al. (Ginsburg, Menapace & Slap, 1997; Ramirez et al., 2000; Ginsburg et al., 2002).

^hThe road map consisted of a detailed description of the wording of the introduction, the goal and the procedure of the meeting, the central research question, the NG-format consisting of four phases, and the closing comments. For each element, the roles of the moderator and observer were outlined as well. In the observation schedule, the number and the characteristics of participants and the general atmosphere during the NG was registered. Next, for each phase of the NG the duration, questions asked by the participants, difficulties experienced by the participants, nonverbal participant interactions and interactions between the researchers and the participants were written down.

ⁱIn total, the 14 NGT meetings resulted in 311 dif-ferent ideas. After discussing and erasing identical ideas, 290 ideas were retained. On average, a group generated 22 ideas. Participants wrote down an average of just under four ideas. To limit the researcher bias, the moderator and the observer independently coded the items. The initial inter-rater-agreement was 85% which was good (>80.0%) compared to other qualitative research. For the classification of the remaining items an agreement was reached between both coders.