The Long and Short of It – Online Surveys

A new white paper from Survey Sampling International (SSI) has found that long online surveys lead to less engaged survey takers.  But according to Market Tools, which has published its own analysis of the factors affecting respondent engagement, the reality is less clear-cut.

SSI compared a 20-minute survey to a shorter version to test for fatigue effects and the impact on response quality. Both sliding scale and open-ended questions were tested, and the study was conducted first in 2004 and replicated in 2009.

In all cases, response completion and engagement decreased with the long survey, according to the SSI white paper. Author Pete Cape concluded, “If researchers work to keep surveys shorter, it will not only help ensure response quality, but it will also make for more motivated and responsive respondents.”

Market Tools, however, concluded that there are a number of design variables that lead to respondents’ rating of the survey, rate of abandonment, the first incidence of “speeding” through the survey, and the percentage of pages sped through. They concluded that while survey length is a good predictor of most respondent engagement measures, there is wide variation in the design variables that are influential in respondent engagement.

Engagement is driven by a complex interaction among design variables. There is no fixed maxim about survey length that applies in all cases.

Questionnaires need to be as short as possible while still accomplishing its objectives.  But, survey design factors also play a key role and can mitigate respondent fatigue and abandonment.

Both white papers are available online, SSI’s here and Market Tools’ here.