Psychological Differences between Cell Phone and Landline users presents a challenge for polls

May 29th, 2012 by Ravi Iyer

When the NY Times or Gallup reports that Obama or Romney has a lead in the polls, how do they know this?  Typically, they pay people to randomly call people and they extrapolate from this sample, using established statistical methods, to make generalizations to the population.  Some groups won’t respond, especially young adults who often have cellphones and screen their calls.  Many people I know are like this.

Polling guru, Nate Silver, has written about this issue extensively, and Pew has researched it as well.   Cell phone users tend to be younger and more liberal.  Pollsters are used to correcting for such selective non-response (e.g. men selectively non-respond more than women) by weighting their answers.  However, this critically relies on having a variable that you can use to do this weighting.  If cellphone users simply differ on demographic dimensions, weighting should work, but if they differ on other dimensions such as Big 5 personality traits or values, then pollsters will be unable to weight their data.

Do cell phone users differ from landline users on psychological dimensions? The answer is fairly common sense as this is an issue that we all have lots of anecdotal data on.  Of course they do.  The below chart compares cellphone users to landline users based on visitors to yourmorals.org who answered a question about their phone usage, with traits related to landline use at the top and traits predicting cellphone use at the bottom.

Psychology of Cell Phone Users

Cellphone users value stimulation, achievement, and hedonism more.  They value tradition, conformity, and security less.  They are less conscientious, more liberal (especially on social issues), and are younger.  Some of these variables are things that pollsters can address by weighting their results (e.g. youth and liberalism), but other variables are things that pollsters do not measure and therefore cannot directly weight for.

Since some of these things vary by ideology, gender and age as well, we can statistically control for these factors and see if we get fewer significant predictors of cellphone usage.  Valuing stimulation and achievement are the remaining significant predictors with valuing tradition and being socially conservative as marginally significant predictors.  Other psychological variables such as being conscientiousness and valuing hedonism are accounted for by controlling for factors that pollsters likely can weight for.  As such, perhaps these psychological variables are less problematic.  It is worth noting that valuing stimulation remains by far the best predictor of cellphone usage (after age) in regression analyses controlling for demographic variables (beta = .13, p<.001).

The yourmorals.org sample is not a representative sample, but I think that might be better in this case.  Trying to measure characteristics of people who use cellphones, which I would assume correlates with screening calls and generally being less responsive to surveys, might be better done using non-phone means so that your measurement interacts less with what you are measuring.  The educated, internet savvy users who tend to answer yourmorals surveys are exactly the kind of people you might want to examine and be unlikely to poll via phone.  Further, we aren’t interested in whether the overall population has differences between cellphone users and landline users.  That could be a function of youth (the biggest predictor here).  Rather, we are interested in whether people who have the exact same demographic characteristics and vary only in terms of their cell phone usage may differ in meaningful ways as it is this variance that would confound pollsters.  Using a particular non-representative sample can actually be better for answering questions about the relationship between variables, as certain differences are naturally controlled for with the whole sample being generally internet savvy, educated, and white.  But certainly these findings (like all social science) need to be replicated by others in other datasets to have more confidence.

The take home message?  As noted by Pew and Nate Silver, polls will have to have cellphone samples in order to avoid bias that likely skews against liberal candidates.  Second, if my intuition that heavy cell phone users are unlikely to respond regardless is correct, then even pollsters that poll cellphones may have to start thinking about weighting for non-traditional variables that are a proxy for these psychological variables that predict non-response.  Silver suggests “urban/rural status, technology usage, or perhaps even media consumption habits“.  Third, the psychological profile of cellphone users (seeking novelty, being socially liberal and not valuing tradition) suggests that polls might exhibit more bias on social issues such as gay marriage, and other issues which could reasonably be said to correlate with being a novelty seeker.  These effects aren’t big, but in a world where a few percentage points is big news, they are worth considering when digesting poll results.

- Ravi Iyer

Posted in big 5, cellphone users, news commentary, ny times, obama, personality traits, political psychology, psychological differences, romney, unpublished results, yourmorals.org6 Comments »

6 Responses to “Psychological Differences between Cell Phone and Landline users presents a challenge for polls”

  1. Carol says:

    This article is based on the assumption that all cellphone users are the same. This assumption may be partly valid for smartphone users, but that is only half of all cellphone users. A significant number of cellphone users have prepaid cellphones. I would submit that Tracfone users are different to I phone users.

  2. l says:

    How many landline users under the age of 30 did you find? Did you ask them whether they live w/their parents?

  3. l says:

    Also, whether you make most of your personal phone calls with a landline or cell phone could be in significant part a function of your employment status. I make lots of personal calls from my office (landline), but outside of work I don’t even own a landline.

  4. sam says:

    Carol, I have both a prepaid phone and an iphone. Sometimes you need to buy apps and sometimes you need to buy drugs.

  5. Ravi Iyer says:

    Carol and L, those are all valid points and unfortunately data we don’t have. I don’t think any of those factors would change the basic findings here, but they certainly would add needed nuance, so we’ll take them into account if we move further down this research path. Or perhaps someone else will take this ball and run with our findings plus your comments in some new direction. Thanks for your interest.

  6. Cell phone users are more advanced than landline users. There are lot more challenging apps in cell phones than regular landlines where you got nothing.

RSS feed for comments on this post.

Leave a Reply