Home Page Company Clients Reports CallWeb SampleCalc EPlanCalc French

Don't confuse movement with progress.
(Denzel Washington)

AUGUST 18, 2012

How do cell phones affect telephone surveying?

A 2010 study recently published by Statistics Canada revealed that an increasing proportion of households are preferring cell phones to landlines: 13% of respondents reported using a cell phone exclusively, compared to 8% in 2008. The shift is particularly significant for younger households: half of the households aged 18-34 relied only on cell phones, a considerable increase from 34% in 2008.

The study also reported that about two thirds (68%) of exclusive cell phone users were renters rather than homeowners. On the flipside, in 2006, the same proportion of Canadians overall reported owning their home.

However, the implications of these findings in terms of representativeness remain unclear — we could not identify any Canadian studies that analyzed the characteristics of cell phone-only households in comparison to landline and mixed households. Even if this information existed, it is possible that such studies would have already been dated, given the high rate of change.

Measurement and Representativity

Some information is available from the US, however. The American Association for Public Opinion Research (AAPOR) Cell Phone Task Force published a paper in 2008 that points to two main measurement issues related to cell phone surveying. "First, there currently is no reliable evidence that the data gathered in good cell phone surveys are of lower quality than in comparable landline surveys. However, the Task Force believes it is advisable that researchers remain attentive to this concern. (...) Second, many new survey items may be needed for use in adjusting phone samples prior to analyzing their data. (...) the reliability of these new items has not yet been established." (page 5) Another report by the same AAPOR Task Force published in 2010 has adopted a more positive outlook on using cell phone surveying and representativeness: given the particularities of cell phone-only users in the United States (page 6 of the report mentions young adults, males, minorities, etc.), it concludes that using cell phones in a random digital dialing (RDD) frame "is very good for telephone survey researchers in terms of reaching more representative unweighted samples of the general public." (page 6) Nonetheless, issues with how to handle mixed households reached on cell phones as well as how to optimally screen for cell phone use and interview appropriateness remain unresolved.

Survey Costs

A paper published by the Canadian Marketing Research and Intelligence Association (MRIA) in 2007 predicts drastically increased surveying costs due to various factors linked with cell phone-only populations: need for an incentive, lower response rate, need to place more initial calls (because of more terminations linked with underage participants, busy or distracted respondents, etc.), need to abide by cell phone etiquette and leave a message and call-back number. According to MRIA, these factors, among others, would mean surveying costs 2.4 times higher for cell phone populations than for landline users (page 13); the AAPOR concurs, suggesting that the cost of cell phone surveys are generally between 2 and 4 times more expensive than landline ones (2010, page 11).

The 2010 AAPOR has drawn on experience from recent years to make a number of suggestions about cell phone surveying etiquette. Recommendations include:

The AAPOR remains optimistic, and concludes its 2008 report by stating that "ongoing research has demonstrated that conducting survey interviews by cell phone users in the U.S. is feasible, if also somewhat difficult and more expensive than conducting similar interviews on landline telephones." (page 3) The 2010 AAPOR Task Force report makes three recommendations, all of which are linked with researchers acting purposefully: "(1) researchers should explain the method by which the cell phone numbers used in a survey were selected, (2) if RDD telephone surveys do not sample cell phone numbers, then researchers should provide an explanation of how excluding cell phone owners might or might not affect the survey's results, and (3) researchers should explain the decisions that were made concerning weighting of cell phone samples, including why the sample was not weighted, if in fact that was the case." (page 11)

Ethical Considerations

The ethics of cell phone surveying should also be taken into consideration. The AAPOR Task Force, in its 2010 report, addresses this question and makes a number of suggestions. Recommendations include being mindful of the possibility that respondents not be in the time zone indicated by their area code (page 79), that special attention be paid to the frequency and timing of calling attempts, that detailed caller identification be disclosed (also to help increase the number of call-backs, as mentioned earlier), and that the survey providers maintain an internal "Do Not Call List", to minimize the chances of repeated burden. Moreover, the AAPOR Task Force suggests that special attention be paid to the safety and privacy of respondents. Over and above ensuring (potentially at the start of the survey and part-way through) that the respondent is not distracted by another activity (including, but not limited to, driving, walking, etc.), the survey provider should ensure the respondent's privacy by wording questions so that "respondents in public or semi-private places should not be required to verbalize responses that could (1) reasonably place them at risk of criminal or civil liability; (2) be damaging or to their financial standing, employability, or reputation; or (3) otherwise violate their privacy" (page 81).

 Return to the home page
 List all news

To reach us:

General address : service@circum.com
Benoît Gauthier : gauthier@circum.com, @BGauthierCEEQ
Tel. : +1 819 775-2620, Fax : (no more fax; sorry)
238 Fleming Road, Cantley, Québec J8V 3B4

Benoît Gauthier, Mastodon