Nebu’s Market Research Magazine

Subscribe to Email Updates


Sample and Survey Quality in Market Research

A list of helpful services to ensure sample quality

Sample and Survey Quality in Market Research

Eric van Velzen
Posted on 21 October 2016 in Sampling
by Eric van Velzen
4 min

Sooner or later, every market researcher meets the need to examine the quality of their providers' data - to have it confirmed that the participants are unique, real and engaged.

Luckily nowadays it is very hard to think of a challenge for which you can't already find a mature and approved solution out there. Several renowned key players in checking and ensuring sample quality are confirming this theory.

Sample and survey quality

In this post, I will introduce you to some of them, and explain how they work. Your client will not have any doubts about your research if you are using one or a combination of these services. Another great benefit of using these leaders' services is that their processes are not interfering with user experience and are consistent with privacy and data protection laws.

1. Imperium

Imperium provides a comprehensive  suite of technology services and customized solutions to verify personal information and restrict fraudulent online activities.

The world's most respected market research and e-commerce businesses rely on Imperium's superior technology and custom engineered solutions to validate their customers' identities, verify data accuracy, automate review processes and uncover the intelligence that improves profitability.

a. RelevantID

RelevantID is a proprietary technology that gathers a large number of data points from a respondent's computer, such as operating system version, browser version, plug-in, etc., and assigns a relative weight to each data point. The data gathered is put through deterministic algorithms to create a so-called digital fingerprint of each computer.

This unique digital fingerprint identifies duplicate respondents who take the same survey more than once from the same machine. RelevantID flags a computer each time a user tries to take a survey, so it can detect if multiple email accounts are being used to take surveys from a single computer.

b. Verity

Verity reduces fraud by confirming the accuracy of an individual's self-reported data against well-known databases that specialize in consumer demographics. These databases seamlessly integrate billions of records across thousands of databases. Leveraging this information, Verity validates the accuracy of a person's name, address and other demographics without any interruption in the user experience.

Long story short, with Verity, you can tell whether your respondent is lying about themselves.

c. Real Answer

Open-end survey responses provide unparalleled insight that companies rely on to add depth to their research. Real Answer uses a proprietary algorithm to evaluate the quality of open-end responses by comparing the lexicon used against several critical factors, including garbage words, vulgar language, similar prior responses, pasted responses, and robot-generated text, as well as the length of response.

d. Capture

Companies interested in designing an optimal user experience need to understand who is accessing their website and the technology being utilized to define the user experience.

Capture provides a comprehensive technical profile of website users, from browser settings to geographic location. Capture can be used to analyze web-user trends, ensuring an optimal web experience and thereby yielding higher lead conversion and ROI. It can be used in real time to route individuals based on the details of their environment, or saved for later segmentation and analysis.

2. TrueSample

TrueSample provides you with the tools to achieve a measurable increase in the confidence of your research respondents and results. A patent-pending technology, TrueSample's software delivers objective assurance that research respondents and results are real, unique, and fraud free.

TrueSample divides their services to 3 main categories:

  • Device: Device Fingerprint + Fraud Markers
  • Respondent: Sample Validation 
  • Response: Survey Validation

a. Device Fingerprint + Fraud Markers

Device Fingerprint analyzes a number of characteristics with sophisticated and fast algorithms to determine a unique device fingerprint. With this service, you can prevent duplicates in your sample.

Fraud Markers lets you check the device of your respondent upon entry to the survey, so you can make sure it isn't unique but also a "real" device, matching the expected device settings.

For more information, please check our blog post: How to Deliver The Highest Level of Data Quality.

b. Sample Validation

Instead of concentrating on the device, Sample Validation checks that your participants are real, unique and fraud-free based on information about them, including verifying names and postal address and checking names and email addresses against third-party databases of social sources.

c. Survey Validation

Survey Validation approaches the problem from a different angle - besides verifying that the respondent passed Sample Validation, this service also makes sure that your panelists can take a survey once no matter how many sample sources they come from or what devices they use.

Survey Validation also uses advanced behavioral algorithms to eliminate respondents that don't seem to be paying attention or giving thoughtful answers.

+1: evercookie

If you are new in the market research industry and don't want to spend resources on the above hotshots yet, there is always a choice of implementing your own solution. For that, evercookie, a free and open source tool, can be very useful.

Evercookie provides a JavaScript API for storing extremely persistent, so called zombie-cookies (because no matter how you kill them, they come back) in a browser, which you can for example check to filter out duplicates.

Evercookie stores cookies in thirteen different places and ways, with some of them even being able to persist cookies across different browsers of the same machine. This set of ways makes its persistent storage so effective that, in a leaked document, NSA also cites evercookie as a method of tracking TOR users.


Like our blog? Subscribe now!

Submit a commment