14 Online Focus Group Moderating Tips

Doing online moderation is somewhat different from moderating traditional groups. Here are 14 tips:

1. Recruit well. Just as with traditional groups, you’ll need to over-recruit to account for no-shows, and you’ll need to compensate participants at the same level as with traditional groups.

2. Prepare your moderator’s guide thoroughly and know it well. You’ll be typing and thinking at the same time, so you won’t have a lot of time to glance at your guide.

3. Prepare long descriptions and/or links ahead of time in a text document. Test them to make sure they work and be ready to cut and paste them into the discussion. Practice this ahead of time to avoid fumbling.

4. Feel comfortable with your facility before you begin. Make sure it’s simple to use. Bells and whistles may be fun to look at, but as a moderator, you want to make sure that your facility works quickly and simply and that there’s someone on staff to offer assistance.

5. Arrive early at the facility to greet early arrivals. Acknowledge their arrival and let them know that you’ll be beginning soon. As each participant arrives, be sure to acknowledge him or her. If you don’t, the participant can be insecure about being in the right place at the right time.

6. Set expectations. At the beginning of the focus group, make the participants feel comfortable with the online format. Tell them that spelling and grammar are not important. You’re looking for honest opinions.

7. Set participant ground rules. Tell participants that it’s okay to agree or disagree with one another, but ask them to be sure to answer all the questions from the moderator.

8. Learn participants’ names and keep track of what each person is saying. Respond to individuals by name. This is extremely important! If you don’t do this, you will lose people from the group discussion. If you want everyone to respond, be sure to say this. Remember that you won’t have the physical presence of the participants and visual cues to keep people involved, so you have to keep track of them and use names to assure participation.

9. Ask everyone to answer a question at once. Moderators often begin traditional focus groups by going around the table, asking each person to answer one at a time. In an online group, you can achieve the same effect by asking everyone to respond at once. Tell participants they need not wait for others to type in their answers. Both moderator and participants will see each person’s response as they finish typing, and dialogue can follow.

10. Be prepared for less continuity in the conversation flow than with traditional groups. Differences in typing speeds combined with a lack of physical presence means that some participants may spend a longer time than others answering a question. Their responses may come once you’re already on to another topic. In essence, a good online moderator has to be skilled at handling two or three conversations at once.

11. Develop excellent keyboard skills and a great memory. Some moderators find it tough to type and remember names and conversation at the same time. This takes practice, so you may want to do some mock focus groups before you do the “real thing.” Observing an expert moderator is also very helpful.

12. Make the focus group conversational, “chatty,” and elicit the personalities of the participants. Use colloquial expressions. Use “smiley faces” and other Internet symbols and phrases, but be sure to explain shortcut phrases the first time to use them, i.e. LOL (laughing out loud). Failure to do this makes some participants uneasy that they are not as Internet savvy as other participants, and this can reduce participation.

13. Keep track of participants. If you haven’t heard from someone in awhile ask, “Mary, are you still with us?”

14. Practice makes perfect. It’s often a good idea to hire an experienced online moderator for your first few groups. By observing, you’ll quickly learn the “tricks of the trade.” Good Luck!

  • Share/Bookmark

Demystifying Sampling Error

Consider the following hypothetical newspaper excerpt:

An October poll of 800 registered voters found that, if the election were held that day, Candidate-X would beat Candidate-Y, 55% to 40%. While Candidate-X held a 15 point lead, his numbers have “slipped” from a September poll that showed Candidate-X at 58% and Candidate-Y at 39%.

Although data are reported this way all the time, something in the above paragraph is incorrect. What is it? Simply put, there is no statistical basis for claiming that Candidate-X’s lead has “slipped” at all.

Why? Because the margin of error for a poll of eight hundred people is plus or minus 4 percentage points, and Candidate-X’s September-to-October difference fell within that 4-point margin. So, statistically speaking, there was no change between September and October.

From another angle… if you asked a question from this poll one hundred times, ninety-five of those times the percentage of people giving a particular answer would fall within 4 points of the percentage that gave that same answer in this poll. Statisticians refer to this as a confidence interval (the ‚Äúninety five out of a hundred‚Äù referring to the 95% confidence limit).

And here’s another very important way to think about it: For every 20 times you repeat a poll, one of those times you will get an answer that is completely wrong (because that poll was the one in twenty where the results fell outside the margin of error).

The bottom line: Never place all of your faith in a single survey. Only by looking at numerous surveys will you have the most accurate picture.

  • Share/Bookmark

Standard Deviation

Statisticians often refer to data that are “normally distributed” (that is, where most values are close to the mean and fewer are at the extremes). For example, a community’s weekly calorie consumption would typically be normally distributed, with a few “outliers” consuming far more or far fewer.

When graphed, normally distributed data form the classic bell curve. Per the above example, this horizontal axis would show calories consumed while the vertical axis would show how many people eat x calories).

Standard Deviation Graph 1

Of course, not every data set’s curve looks this perfect. Some are flatter, some are steeper, and some have means that lean to either side. But all normally distributed data resemble this basic shape.

What the standard deviation tells us is how tightly data are clustered around the mean. When values are tightly clustered in a steep bell curve, the standard deviation is small. When values are spread apart in a flatter curve, the standard deviation is larger.

Graphically, one standard deviation (the red area) away from the mean (the center vertical line) represents about 68 percent of the people. Two standard deviations away (the red area plus the green area) accounts for about 95 percent. And three standard deviations away (the red, green, blue areas) accounts for about 99 percent.

Standard Deviation Graph 1

If the above curve were flatter and more spread out, the standard deviation would have to be larger to account for 68 percent of the people. That’s how the standard deviation tells us how spread out the values are from the mean.

If you were comparing test scores across school districts, for example, the standard deviation would tell you how diverse each district’s scores are. Let’s say District-A has a higher mean test score than District-B. Does this mean that kids in District-A are really smarter? Perhaps not.

Because a bigger standard deviation means that more kids scored toward one extreme or the other, a few follow-up questions might determine that District-A’s mean scores skewed higher because the State sends “gifted and talented” kids there. Or that District-B’s mean scores skewed lower because “mainstreamed” special education students were sent there. As you can see, the standard deviation can reveal a less obvious but highly relevant layer of information.

The standard deviation also can help you to evaluate the merit of highly publicized research studies. For example, in a study that claims to show a relationship between eating spinach and building muscle mass, a large standard deviation might suggest that such claims are not valid as they first appear.

  • Share/Bookmark