blogbg-img.jpg

Nebu’s Market Research Magazine

Subscribe to Email Updates

The Evolution of Market Research Data Collection

From pen and paper to multi-mode and mixed-mode research

The Evolution of Market Research Data Collection

Ian Roberts
Posted on 16 January 2015 in Data Collection
by Ian Roberts
6 min

In the halcyon days of the market research industry – not quite when dinosaurs roamed the earth, but close – paper-and-pencil interviewing (PAPI) was virtually the sole mode of quantitative data collection: Referring to a hard copy questionnaire, the interviewer would ask questions verbatim (i.e., exactly as they were printed on the page, following the prescribed skip patterns), and write the respondent’s replies directly onto the page.

evolution

PAPI is a time-consuming and error-prone method of research data collection, relying heavily on the diligence with which the interviewer follows skip pattern directions and legibly records the respondent’s answers. With PAPI, data integrity can also be compromised if different regional accents mitigated how well the interviewer and interviewee understand one another.

In addition, PAPI surveys are not conducive to use if a client wants respondents to evaluate visual stimuli or lengthy blocks of text (as in complex product concept testing). And because PAPI makes it necessary to connect the interviewer and the respondent in “real” time, it’s necessary to either pre-schedule interviews or reach out to a slew of potential respondents (via telephone or a mall intercept, for example) with the hope that some of them will both qualify for participation and be willing to complete the interview.

In short, the days of PAPI-only interviewing were challenging, to say the least!

Computers:  The “Big Bang” of Market Research Survey Methods

If your involvement in the market research field only goes back a decade or two, most of these types of limitations and caveats are probably unthinkable. And even if you were in the market research trenches in “days of yore”, it’s probably difficult to believe that there was ever a time when a computer didn’t play an integral part in the research process. Indeed, with the introduction of computers into the research arena, the potential of survey research exploded.

Once computers were added to the mix via computer-assisted personal interviewing (CAPI) and computer-assisted telephone interviewing (CATI), it was a huge step in alleviating the aforementioned concerns. With CAPI, because responses are entered directly into the computer by the interviewer during the real-time interview, concerns about handwriting legibility are ameliorated; with CATI, concerns about handwriting are similarly obliterated, and it’s also no longer necessary to worry about whether interviewers are following correct skip patterns. Computer software has, in essence, taken on responsibility for more elements of quality control.

Market research goes on-line

Of course, even CAPI and CATI rely upon the involvement of an interviewer, who must read the survey questions to the respondent, and enter the interviewee’s responses. Enter the World Wide Web. Using research platforms that involve the Internet, Web-Assisted Personal Interviewing (WAPI) and Computer-Assisted Web Interviewing (CAWI) eliminate reliance upon an interviewer and allow for respondents to complete Internet-based surveys on their own schedules, at their own pace. Skip patterns are pre-programmed into the survey software, and recruiting is often conducted via email invitation.

The use of respondent panels has proliferated as the popularity of Internet research has blossomed. These panels typically offer an incentive to the panel member for each survey completed; panels are often built with an eye toward including very specialized respondent subgroups that allow clients to more easily recruit the specific respondent interest, demographic characteristic, or behaviors they are seeking (e.g., hospital-based Interventional Cardiologists; working parents of children under 13 years-old; affluent individuals who invest in mutual funds). This, of course, leads to enhanced incidence, enhanced respondent engagement, and enhanced completion rates.

“Phoning It In”: Surveys Go Mobile

As web-based technology becomes a routine part of daily life for an increasing number of consumers, the options for survey administration continue to expand, as well. The ubiquitous presence of smart phones and tablets has spurred the growth of a new generation of survey tools. Smartphone-Assisted Personal Interviewing (SAPI) and Tablet-Assisted Personal Interviewing (TAPI) have become the latest approaches to web-based survey research. In addition to affording scheduling convenience to respondents – as well as the enhanced data integrity that online research confers – these approaches also allow respondents to complete surveys wherever they happen to be. They are no longer tied to their desks, or even their laptops. Instead, surveys may be completed while commuting to work, at a coffee shop, or wherever else a respondent finds himself with a bit of extra time.

What are Multi-Mode and Mixed-Mode surveys?

As the term “multi-mode” implies, within multi-mode research the survey is created in several different delivery methods across the previously-discussed survey modes, and respondents are offered the option of completing the survey using any of the available approaches. This approach allows respondents to choose a survey “delivery method” that works best with their own available technology and preferences. While this approach enables any existing survey mode to be used, the respondent is limited to using only one of the modes; switching between modes is not possible.

Mixed-mode research – a sub-group within multi-mode research – also allows respondents to select from several survey delivery methods. However, with mixed-mode research, it is possible to switch between modes seamlessly while actually within a survey. For example, the survey could be started within CATI, with the respondent instantly sent a URL to enable him to continue the study in WAPI. This approach allows the research team to leverage specific advantages of different survey methods, and tailor each survey to use the methods that are optimal in light of the study objectives and target respondent audience for that particular survey.

The most typical scenario for mixed-mode research is moving from Computer-Assisted Personal Interviewing and/or Telephone Interviewing (CAPI and/or CATI) pre-screening to Web-Assisted or Smartphone-Assisted Personal Interviewing (WAPI or SAPI); however, moving from WAPI/SAPI to CATI (e.g., for a “call-me” request) could also be covered by mixed mode.

What’s so “dynamic” about Dynamic Switching?

To optimize mixed-mode research, it’s critical – from the vantage point of respondent convenience and survey integrity – - that the respondent be afforded a dynamic switch between modes. This requires that the respondent be able to move from telephone to web (and back) at virtually any point in the survey. Therefore, it’s not possible to simply insert a programming instruction to “switch” at a particular question or juncture in the survey; in true “mixed-mode” research, the respondent has the option of completing the survey in his own time rather than potentially losing the connection and requiring them to re-engage at a later time. This dynamic switching capability is certainly optimal. However, because it’s not a given that dynamic switching will be incorporated into the survey instrument software “mix”, the research team should verify that their survey instrument software includes this capability.

Not only should the software allow for dynamic switching – it’s also key for the software to enable ease of multi-mode and mixed-mode scripting. Each survey mode should only display elements of the script that are necessary within that particular mode, i.e., interviewer instructions are shown within the CATI and CAPI versions, but not within WAPI mode. This capability means that a consistent “language” can be used across all modes, versus requiring different software to accommodate different scripting between the various modes in use.

    

Like our blog? Subscribe now!

Submit a commment