Doubtful data can do real damage to trade publishers

There's an awful lot of self-serving spin going on about the internet's displacement of traditional media. The so-called facts that come from these efforts gain currency often based on a press release from someone or some company with a fairly large axe to grind. As a result, the data gains a life of its own without the reader being aware of (or being made aware of) its source. And the drumbeat of negativity about traditional media becomes something of a self-fulfilling prophecy.

This is not to say that, for instance, trade magazines are not finding their business tough going with the growth of online business and trade news sources. Printed directories are becoming harder to sustain because online sources are so easily used and searchable. But such self-serving "data" makes the public and advertiser views of the trade press much, much worse.

As an arbitrary, but classic, example, take the case of a press release that went out over Canada News Wire on July 12. It was headlined: Internet Usage Continues to Grow Within the Canadian Industrial Sector. Not much to quarrel with about that heading, because it's probably true.

The story itelf, however, is based on a "Canadian Industrial Engineering Trends Survey" conducted by GlobalSpec, self-described as "the leading specialized search engine, information services and e-publishing company for the engineering, industrial and technical communities".

The press release nowhere contains any information about the methodology, though a link is provided to a summary on the company's website. The summary tells you that 2,466 people responded, but it doesn't say out of how many people polled. It is also coy about who these people are, beyond saying they are drawn from GlobalSpec's "registered user base of engineers, technical buyers, scientific professionals and other members of the industrial community". In other words, GlobalSpec's own customers.

We are never told what proportion of the "community" the survey's database or response represents or how the people polled were chosen or even if the survey is based on a sample.

And what do they say the results are? Among other things, "37 percent stated their use of printed trade magazines has gone down in the past year".

What the press release doesn't say, however, is that 63% of the respondents said their use of printed trade magazines has stayed the same or actually gone up (5%). Nor do they say that the research shows printed material (trade magazines, supplier catalogues and printed buyers' guides) actually match or outstrip reliance on GlobalSpec.

Now, I'm wondering whether people writing various news stories that are spun out of this press release will drill down and question the methodology, response rate or scientific validity of such a study. Or whether the stories will focus on that 37% figure, without asking where it came from. I suspect the latter.

(What's sometimes not realized is that CNW feeds go directly into newsroom computers of newspapers and other media and are sometimes used unchanged, as though they weren't public relations rather than news. Many of the secondary stories in newspapers are largely based on or simply reprints, of these press releases.)

I'm not saying there's no trouble in trade magazine publishing. Or that online search isn't a useful and increasingly used tool of business. But readers need to be sceptical about where this stuff comes from and trade publishers need to be more vigorous in holding their competitors to account for this kind of self-serving baloney.