Call me old school or old fashioned, but I still believe in the power that telephone data collection has for public opinion research. In the Western world we see a change especially within market research from traditional probability sampling to non-probability sampling due to the large coverage of panels with an enormous amount of demographic variables included.
But what about developing countries or areas in which online panel or face-to-face studies are very difficult to establish due to geographical or political issues? Furthermore, I believe that the higher penetration of cell phone is not a risk for traditional research but rather an opportunity to gain extra contact moments not just via actual phone calls as with CATI surveys, but further provides an option to connect with either 1-way or 2-way SMS. Think about demographics that can be linked to a cell phone number as unique identifier (not an email address). By using online surveys additionally to CATI surveys, extra demographics like location data can be stored from the location setting from within browser.
For many social researchers especially, face-to-face research remains the gold standard and even looking at events like Turkey or Venezuela, researchers are very hesitant to move to CATI fieldwork. Besides the obvious cost benefit there are way more important advantages of CATI:
Fieldwork can be controlled easier at a single location
Possible to collect data from outside a country
Faster turnaround times
Mobile populations or slums can be included due to high level of cell phone coverage
Currently social researchers are still very hesitant to move away from the so-called "Gold Standard" of face-to-face interviews due to concerns such as:
High volume of non-working numbers
Using sampling frames with a large amount of cell phone share
Do sampling frames actually exists and if so, what about the coverage?
Might a transition from face-to-face to CATI bring a change to the collected data?
Phone penetration is not high enough
Some of these aspects are valid, some can be managed and some bear pretty much no truth. Let us go through some of the aspects.
High volume of non-working numbers is certainly true for some of the countries. Nevertheless, many dialer operators can fine-tune the recogniton of info signals and accurately mark numbers as non-obtainable which can be used for later response rate calculation using the AAPOR guidelines. Pricing for phone numbers can also be changes as cost per completed interview so that research agencies can calculate with a fixed cost per completed interview, which allows some grip on the sampling costs.
The next point refers to sampling frames. Consider the case of creating a multi-stage cluster sampling frame for a face-to-face study and compare this with a dual frame telephone frame. For the multi-stage cluster frame, amount of completes in most cases need to be a lot higher in order to have similar design effects when comparing to telephone frames. The telephone frames for both mobile and landline can be set up in such a way that every single number is included in the frame and additionally has the same probability of being selected within a stratum. If you require more information from vendors for a sampling frame, always request a sampling methodology document on the various numbering blocks assigned for both landline and mobile so that any coverage error can be prevented.
I believe the only larger region in which phone penetration level lacks behind for phone research would be Sub-Saharan Africa.
Looking at phone penetration rates, I believe the only larger region in which phone penetration level lacks behind for phone research would be Sub-Saharan Africa in which still a strong bias towards a more educated and wealthier part of the population exists, which is part of the conclusion from our white paper on SMS Data collection in Sub-Saharan Africa.
Lastly referring to the question whether data collected from face-to-face vs. CATI will be different – most likely YES! Looking at data from the Gallup World poll or other research on comparison of methods (face-to-face vs. telephone or even more face-to-face vs. SMS), even in areas where phone penetration provides almost full-coverage, differences in data will occur due to various elements: Interviewer effects, response rates and so on.
Feel free to get in touch with directly with the author, Carsten Broich, Managing Director at Sample Solutions via email email@example.com or request a call from Nebu to hear about our CATI system (fill out the form you can see on the right side, next to the article)!
In the meanwhile download your copy of our free ebook: A Guide to CATI Research.