FEBRUARY 17, 2011

Web panels: the new modus operandi or a temporary mistake?

An article by A. O'Connell, originally published in the Harvard Business Review, was recently made available on WebSM.org, a Web Survey Methodology on-line resource. The article analyses the growing challenges associated with telephone surveying, and explores some of the new survey research methods that have been put forth in recent years, such as on-line panels. This article also clarifies some of the issues related to on-line panels, such as the emergence of non-representative "professional panellists": these individuals tend to have a higher degree of education and have a greater chance of being white, young or old than the general population. O'Connell states:

"The American Association for Public Opinion Research, a standards-setting body made up of nearly 2,000 survey professionals from private and government organizations, takes a dim view of web panels that are drawn from volunteers. Stuck between traditional polling, which is getting more difficult and expensive, and cheaper surveying of unrepresentative panels, the field is due for a radical rethink. At least one thing is clear: In their quest to probe the public mind, survey firms will make greater use of the vast amounts of digital data that people wittingly or unwittingly provide about themselves."

In parallel, a report by EKOS Research Inc. documents the lay of the Web-panel land in Canada. This report was written following the MRIA Net Gain 4.0 panel on the subject (update: some results from this study are presented in "Canadian Online Panels: Similar or Different?", Vue, February 2011, pages 16-20); it draws similar conclusions:

"All of the opt-in online panels feature a dramatically higher incidence of professional respondents who conduct a surprisingly large number of online surveys in search of the modest financial incentives offered. Certainly the general public, either as citizens or consumers, bear little resemblance to these fairly downstream hyper active users of the Internet, who in addition to doing surveys at least weekly (and for a large number daily) and participating in a high number of panels reveal an unusual propensity to video games, coupon cutting, and instant messaging, and exhibit unrepresentatively low levels of income."

The EKOS report emphasizes the benefits of panels produced based on random selection rather than the opt-in approach.

These difficulties implicit to on-line polling, in particular on political matters, were also recently reported in The Star, which mentions that

"There's broad consensus among pollsters that proliferating political polls suffer from a combination of methodological problems, commercial pressures and an unhealthy relationship with the media."

Based on MRIA industry statistics, on-line research has gained significant momentum in Canada in recent years. Web surveys are now the method of data collection of choice, and Web panels could be the tool that will take the industry to a sustainable modus operandi for the future. Issues of quality and reliability of Web panels still abound, however, and considerable care must be taken in their use until more empirical research is conducted on them.

 Return to the home page
 List all news