A column about writing in business
Surveys: uses and abuses
Surveys are an old standby for PR companies on slow news days. But they stink of dubious statistics and questionable objectivity. No wonder the public is increasingly cynical.
You've seen the phenomenon already. Every Christmas and Easter, someone will publish a survey claiming that chocolate is good for you. The media lap it up -- it's a good story. But who benefits? Needless to say, the people behind these surveys are chocolate manufacturers and their PR firms.
Surveys are part of the standard-issue PR toolkit along with press releases, case studies, launch events and buddy lunches. They are used to generate "news" when the client hasn't anything more newsworthy to say and to give the client a platform for opionating.
The public find surveys irresistible. We want to know more about ourselves and our world and they give us (apparently) hard data. In particular, we like them more when they confirm our own opinions: chocolate is nice, young people are rude etc.
Lazy journalists also find them irresistible. I saw a double page feature in a national newspaper recently that was nothing more than a flimsy survey spun up with some pictures and flannel copy. They are cheap copy, the writing equivalent of reality TV.
Why they are bogus?
- Conflict of interest. Did you ever see a press release that said "8 out of 10 cats hated our products, according to a survey?" No! PR-driven surveys always reflect the commercial interests of the sponsor.
- Lack of transparency. You very rarely see press releases that report, in detail, on the methodology behind the survey. Often, PRs will only supply "the full report" if you go through an elaborate mating ritual. Usually you get a press release overview of the executive summary.
- Slippery statistics. Read How to Lie With Statistics for more on this.
- Poor numeracy on the part of the PR, the journalists and the public.
- Lack of context. PR-generated surveys are (by definition) one-sided. They rarely supply any context for the survey - reasons, causes, solutions, controversies -- except for comment by the PR's own client. In other words, spin.
Lack of trust
The result is cynicism. In the UK, only 17 per cent of people believe official figures are produced without political interference and only 14 per cent say the government uses them honestly. Ironically, we know this because the Office of National Statistics carried out a survey.
How to write about surveys
These difficulties don't mean we have to avoid surveys altogether. Treated properly, they can be good copy -- in every sense of the word.
- Personalise the data. Find a way to illustrate the data with human beings. Interview people who like chocolate and who don't like chocolate, for example.
- Get the whole report. If PRs try to parcel out the stats to different journalists and won't give you the whole report, don't use it.
- Report the controversy. Don't just cite the source of the survey but, if there is a potential conflict of interest, report who paid for it as well.
- Ask questions about the methodology: sample size, question rotation, response rate, who paid for it etc.
- Get numerate (do as I say, not as I do!). Good places to start: The Economist's Numbers Guide: The Essentials of Business Numeracy and Journalism.org.
- The numbers aren't the story. Use them for illumination not support and go behind the numbers to find the reasons, surprises or tension.
- Graphs are better than text. If you can give the information visually or find a compelling metaphor, that is better than a dry recitation of the data. I like the way The Economist (which is essentially a numbers magazine) uses simple graphs to do this.
- Do your own research. Kudos to the Norman Nielsen Group who do a lot of hands-on research to generate their reports. Genuine, open-minded research that generates real insight is always going to be valuable.
So, in summary, surveys are often bogus but they're okay if you use them properly. I'm sorry that this isn't a self-consistent position. Like the economist who had his feet in the oven and his head in the freezer, on average I feel okay about it.