JUNE 3, 2011

Are Telephone Survey Response Rates Problematic?

Jenkins Research Inc. recently published a new study on telephone survey response rates in Canada. The study analyzed the response rates of custom, general population, national surveys conducted by federal government departments between 2005 and 2009. It showed that response rates were gradually declining throughout the period of study, and highlighted an inverted correlation between response rates and interview length (the longer the interview, the lower the response rate). As would be expected, longer field periods were also associated with higher response rates. The article draws to a close by questioning the validity of the commitment to telephone surveys as a data collection tool and opening the debate as to what other variables (other than interview length and field period duration) could be controlled to increase response rates.

Jenkins Research Inc.'s study draws conclusions analogous to those drawn by Teresa L. Bladon in the most recent edition of the Canadian Journal of Program Evaluation (The Downward Trend of Survey Response Rates: Implications and Considerations for Evaluations, vol. 24, no. 2, Fall 2009). In her article, she uses comparisons drawn from annual household telephone surveys conducted by the University of Alberta's Population Research Laboratory from 1991, 1996 and 2002 to draw two major conclusions:

• There is a substantial decline in response rates, due to increases in the rates of both refusals and non-contacts; and
• There is an increased level of bias in variables related to respondent education. That is, those with a higher total number of years of schooling and those with a higher level of education are most misrepresented in telephone survey data.

While Bladon's article does not offer any tangible options for counteracting the decrease in response rates that has been observed since the early 1990's, it makes two suggestions for reducing nonresponse bias, namely:

• Adopting a random stratified sampling approach to try and reproduce the characteristics of the general population in the survey population; and
• Post-survey data weighting, which has its downsides: "In order for post-survey statistical adjustments such as weighting to be effective in reducing bias, the weighting variables must be predictive of, or related to, key variables of interest" (p. 148).

 Return to the home page
 List all news