The value of judgement and the human editor

Algorithms and search engines can roam far and wide with a speed that leaves ordinary people in their dust, but the one thing that technology doesn't have is human judgement. And there is an interesting item by Scott Karp* on the blog Publish 2.0 on current thinking about the role of trusted human editors in the dissemination of reliable stories and information.

( Why am I reporting this on the Canadian Magazines blog? Magazine people have always understood that the mediation of information is one of the great strengths of the medium; as more and more magazine publishers augment their traditional publishing with new tools and techniques, it is important to understand how this is to be accomplished.)

The value of judgement and human filters may seem self-evident, but consider social networking sites where, as more and more people contribute, there is a dilution of quality and trust because a lot of people who aren't very smart or savvy contribute to the dumbing down of the content.

Paul Graham of Hacker News, for instance, weights contributors' ability to influence future stories (using human judgement), based on whether they pick good stories now and did so in the past.

Of course, it’s easy to have a good site when you start out with a core group of smart users. How do you keep it good as more people find out about it? We think we have an answer to that. We’re going to have a group of human editors who train the system in what counts as a good story. Each user’s voting power will then be scaled based on whether they vote for good stories or bad ones. This should protect us against the arrival of users who vote up dumb stories. The worse stuff a user upvotes, the less effect their future votes will have. And vice versa: someone who consistently recommends interesting stories will be rewarded with a louder voice.

In a sense, this is what magazines and newspapers have always done, as editors select good stories by good writers and readers vote on the result with their attention, their subscriptions and the single copies they buy. The speedup of the process, however, means that the old and stately ways are becoming harder to maintain. (Note, I said harder, not impossible.) So it is in the interest of every editor and publisher today to burn up some mental energy thinking about how his or her mission can be adapted to these modern circumstances. For instance, by insisting on human judgement as an integral part of the process and on the right to mediate.

*Scott Karp and his colleagues at Publish 2.0 are self-interested in this; they are soon to launch a social networking site specifically for journalists and bloggers called Publish2. But that doesn't make what they're saying any less useful or interesting, but more so.