FREE! Subscribe to News Fetch, THE daily wine industry briefing - Click Here


SPONSORED BY:
<Future Wine Expo
SPONSORED BY:

Ardagh Group. Bottles Born & Backed in the USA. Locally manufacturing quality glass for more than 125 years. Find your perfect bottle.

How to tell if a survey is credible

Determining whether a survey is credible and the data valid as presented is often a difficult task.

 

Unfortunately, many surveys lack credibility because they fail to meet standards for design, implementation and transparency.

 

Why?

Standards used by credible news organizations for assessing data validity (see below for links) require that a study must (at a minimum) reveal:

 

  • The number of subjects in a study,
  • How the were selected,
  • How the questions were asked,
  • The exact wording of the questions,
  • When and where the questions were asked.

 

All of those affect the validity of the results and should be available whenever results are presented as valid.

 

If a study does not publish that information along with its results, there is no way to tell if it’s fake data or not.

Trustworthy survey: case in study

Screen Shot 2017-07-05 at 7.49.05 AM

Properly done polls — such as this recent one from Gallup always — ALWAYS — include the following specific information. Polls which do not include this should not be trusted.

Survey Methods

“Results for this Gallup poll are based on telephone interviews conducted June 7-11, 2017, with a random sample of 1,009 adults, aged 18 and older, living in all 50 U.S. states and the District of Columbia. For results based on the total sample of national adults, the margin of sampling error is ±4 percentage points at the 95% confidence level. All reported margins of sampling error include computed design effects for weighting.

Each sample of national adults includes a minimum quota of 70% cellphone respondents and 30% landline respondents, with additional minimum quotas by time zone within region. Landline and cellular telephone numbers are selected using random-digit-dial methods.”

 

In addition, trustworthy surveys, will include details allowing a close examination for validity: Gallup: View survey methodology, complete question responses and trends.

 

Again, surveys that do not include that degree of detail are always suspect.

Self-selected participant surveys among condemned practices

Screen Shot 2017-07-05 at 7.49.45 AM

 

This link — AAPOR Code of Ethics — is from the American Association for Public Opinion Research (AAPOR) which sets basic standards for surveys. Also: Condemned Survey Practices

Other sources that should be taken into account when examining the validity of poll data:

 

About this article’s author

The author of this article, Lewis Perdue, has been a journalism or communications faculty member at Cornell University, and UCLA.

In addition to his journalism experience, Perdue has also been a Managing Director in the global marketing and PR/Communications firm MSLGROUP, now a part of the multinational Publicis Groupe.

His professional journalism experience includes:

  • Wall Street Journal Online,
  • TheStreet.Com,
  • LA Times.
  • Boston Globe,
  • Washington Post,
  • New Scientist,
  • Marketwatch,
  • Gannett,
  • Forbes ASAP,
  • Barron’s, Business,
  • Washington Monthly,
  • InfoWorld,
  • PC World,
  • TechWeek,
  • Embedded Systems Journal,
  • Computer Currents.

He has been a Washington bureau correspondent (covering the White House and Congress) for:

  • Dow Jones/Ottaway Newspapers,
  • States News Service, and,
  • The Winston-Salem (N.C.) Sentinel

Perdue’s investigative reporting includes work with:

  • Jack Anderson,
  • ‘The Washington Post
  • Congressional News Syndicate