Discussion Guide Creep
Why managing the client is the most important part of writing your qual guide
I remember at RIVA moderator training, we were taught the basic mechanics of writing a qualitative guide—keep the questions open-ended, the language neutral, and the content focused on the project objectives. Moderators are also taught about different activities and devices (mind mapping, fill-in-the-blank, message ratings, write-a-letter-to-a-friend, etc) that help mix things up and reveal more interesting, layered responses than outright asking.
But there’s one lesson about writing a guide that most moderators learn from experience, not necessarily their training—and that’s how important it is to manage the client’s input.
In fact, this lesson is so important, I really think training institutes should lead with it. You could write a beautifully put together, creative, thoughtful guide, but if you have a client who wants to add poorly designed questions that add time you don’t have, then all that work putting it together was for naught.
Sometimes I come into projects as “paid lips,” which means a moderator who is just a moderator—I don’t write the guide, report, do any of the client management, all I do is come in and moderate. All too commonly, the guide that’s given to me is way too long. I see this happen most often when the “client” is not just one or two people but a whole slew of cooks in the kitchen. Without a point-of-contact client who is in charge of managing different stakeholders’ ideas and input, the feedback can get unwieldy, leading to guides that are way too long and that were very obviously written with specific stakeholder interests’ in mind, rather than with a focus on the overall project goals.
It's a lot easier to say “ok sure we’ll make that add” when a client suggests an addition to the guide, but those “ok sures” start adding up, derailing the overall intent of the guide as well as the timing. Receiving an ill-conceived, much too long guide like this is demoralizing to moderators, because it sets us up for failure from jump. There is no way we can get through the guide with the time allotted, and because so many stakeholders are usually involved, someone is going to end up disappointed because their question is going to get skipped.
A too-packed guide really limits the meaningful conversation and probing that are so valuable in qualitative research. The moderator finds themselves focused on just getting through the guide rather than digging for the rich insights clients want and need.
Here are my strategies for managing clients when it comes to their input on guides:
· Before I even write the guide, I have a comprehensive and specific conversation with the client about project goals and what they want the research to cover. This gives me a good idea of what length to recommend for the focus groups, IDIs, or ethnos so that we can get to their most important questions. This piece is “preventive” in that it helps avoid trying to pack 2 hours’ worth of material into 90 minute groups
· The first draft of the guide I submit to them is intentionally a little shorter than the allotted time—this leaves room for their adds
· I tell clients that if they want to add anything substantial, they need to also suggest what to cut to make room—this is a rule I really have to stick with and hold them accountable to follow
· When I know there are going to be multiple stakeholders weighing in, I send them the guide via Google doc and ask them to comment before the meeting. This way I can review the feedback beforehand and think about how and what to add while still having time for the rest of the questions
· If all of the above fails and they’re still hammering for more questions, I tell them they have to increase the time of the groups—maybe 90 minutes becomes 2 hours—and that the increase will be subject to increased recruiting and incentive costs. This usually gets them to back off or at least prioritize their additions
Discussion guide creep is real and has serious consequences for the success of your qualitative research. The strategies I recommend above take a little courage and confidence, because it’s never easy to pushback against a client. But at the end of the day, YOU’RE the expert they are hiring because you know how to do research—so tell them!