That’s the usual dictionary definition. But, I want to extend the meaning somewhat. I don’t think that a bias simplyinhibitsimpartial judgment, I believe that it alsoinformsit. And so for the sake of this discussion that’s the meaning that I’ll be using here.

Tunnel Walker. Toronto, November, 2006

Leica M8 with Voigtlander 21mm Skopar. ISO 320

___________________________________________________________

In late 2006 there was an incident which involved my review of theLeica M8camera, which has since caused me to think about a product reviewer’s responsibilities. Enough time (almost six months) has now passed that the dust has settled, and I have been able to give the issue some thought.

Briefly – I had seen a couple of problems with the M8, didn’t know what it was that I was seeing, and asked the company for their thoughts. At the time they didn’t understand what the problems were either, (or so they said) and suggested that until they and I did it might make sense not to put them in my review. And so, because I only saw these problems on a handful of images out of many hundreds taken, I didn’t comment on them in my review.

That was my mistake. With hindsight, I should have said that I did see some anomalies, don’t understand what they were, and neither did Leica, but when I (we) did there would be a follow-up.

As a consequence, when shortly thereafter many of the M8’s problems started to be identified, my feet were held to the fire by some readers. Anyone interested in the full story canread the reviewas well as my postscripts and draw their own conclusions.

Since then the incident has made me interested in the whole question of a review’s responsibility – to himself, his readership, and the company whose product is being reviewed. Hence, this essay.

___________________________________________________________

Reviewers – Who to Believe?

There is no such thing as an objective reviewer. If there were, they’d be boring. Boring to tears. A good reviewer, like a good teacher, has experience and insights. They’ve been there, and done that. They bring some years, even decades of experience to their craft, and one doesn’t live too long in this world without developing biases and opinions.

The trick for the reader is to identify what these biases and opinions are, and to then determine if a particular reviewer’s biases and opinions jibe with ones own. Whether it’s movie reviews, car reviews, book reviews, or yes – even cameras, one needs to know something of the mind-set of a particular reviewer before determining if what they may have to say will have any value –to you!

When I read someone say that I am biased, my first response (after I’ve rolled my eyes) is to say out loud –duhh!Yes, damn it, Iambiased. What person with an ounce of life experience isn’t? We vote Democrat or Republican, or Labour or Conservative. We prefer Shiraz over Merlot, or vice versa. You want someone unbiased? Visit the cemetery.

But, any particular bias (or set of opinions, to use a more polite phrase) needs to be tempered with experience and an in-depth knowledge of the subject. Then, when a reader becomes comfortable with a certain reviewer’s mind-set they can make an informed decision as to whether or not that review is to be regarded as credible – fortheir needs.

For example; I live in a city with four daily newspapers of which I read two regularly. The lead movie reviewer in one has, I’ve discovered over the years, taste similar to mine. When she writes that a movie is worth seeing I go to it knowing that 9 times out of 10 I’ll enjoy it. The other reviewer has his taste exclusively in his mouth, and I wouldn’t go to a movie that he recommended just out of spite.

Do you get my point? It’s a matter of opinion and taste. Of course this all presupposes that the reviewer is being honest with themselves in their opinions, and honest to the company and product that they are reviewing. If that’s the case then the reader can feel confident in deciding whether or not to give credence to the review.

___________________________________________________________

Responsibly to Oneself

I believe that the most important responsibility that a review has is to himself – to speak with a clear and consistent voice, and to be true to his beliefs and experience. If one one can look at oneself in the metaphorical mirror and know that one has been truthful, informative, and consistent, then when one errs, which as they say is only "human", then the only thing that’s lost is somefacerather than ones self respect.

There are many pressures on a reviewer, particularly one who writes for the web. Time scales have become compressed. It used to be that a new product would be announced, and then weeks or monthstime later review samples sent to the magazines. They would then send these out to one of their contributing editors, and a month or so later a review would be handed in. Anywhere from two to three months after that it would appear in a magazine on ones local newsstand.

Today new products are Fedexed directly to web reviewers, sometimes even before they are in actual production. Reviews are written over a few days or a few weeks, and then immediately placed online for anyone in the world to instantly read.

This acceleration is beneficial (I think) for readers, but problematic for reviewers. There’s little to no time to breath, and certainly there’s no editor looking over ones shoulder with questions and advice. For this reason what I do is turn to a form of peer review. I send my reviews to other photographers and writers whose opinions I value, and ask them to review my review (as it were) as a sanity test before publication.

But, then there’s the problem which occurs when an absolutely new product appears, and there’s no one else, anywhere, who has any knowledge or experience with it. Who to turn to then?

In some instances I will send reviews to the manufacturer just before publication for their comments. Not for their approval, please note, but for their feedback on any factual points that I may have misunderstood or misstated.

But herein lies the problem ofresponsibility to oneself, which is the heading for this section. If you ask someone for their opinion, they’ll give it to you. ‘Tis the nature of the thing. And once having asked and been told, one then has to integrate that information and opinion into the final review. This can be quite difficult when a colleague has pointed out an alternative perspective that appears valid, but is at odds with ones own perceptions. This can also occur when the manufacturer points out why something was designed the way it was, even though one may think that the approach is flawed.

In the end one has to take external input into account, but try and lift ones head above the rush of information and opinions and try and steer a course that is true to ones inherent beliefs. Not always an easy thing to do.

___________________________________________________________

Responsibility to the Reader

No one wants to be blind-sided. We all want as much information as we can get so that we can make intelligent purchasing decisions. This is so as to not waste our time or money. But, much of the time there is no one right answer. For some a Nikon D200 will be a better choice than a Canon 30D, or an HP9180 printer a better buy than an Epson 2400. Or the inverse.

A reviewer therefore has to tread a fine line between their own interests and biases, and the realization that others may not share them. For example – I never use printer manufacturer’s profiles. I have a profiling system and can make my own custom profiles in short order. But, most people don’t have these expensive systems, and so I have a responsibility to report on how good (or bad) the provided profiles that accompany a new printer might be.

I also shoot in raw mode 99% of the time. But, I recognize that many people shoot JPG, either because they really have to, or because they don’t yet understand what they’re losing by not shooting raw. When reviewing a camera it’s therefore necessary for me to test and comment on the camera’s JPG output, even though this isn’t something that I will likely ever need to use myself.

These are just a couple of examples, but the theme is pervasive when one is reviewing every sort of product. One has to keep in mind not only ones own interests and needs but those of a broader cross section of users.

___________________________________________________________

Responsibility to The Company

When a popular reviewer or review site holds a product under the microscope there’s a lot at stake. A new digital camera may involve millions or even of tens of millions of dollars in R&D and marketing costs. When a magazine or web site has half a million to a million readers, or more, the effect of either a positive or negative review on a company can be profound.

Since reviews are by definition a reflection of an individual’s experience and biases there is a huge responsibly that these are exercised in a consistent manner, so that from product to product no favour is shown based on anything other than adherence by that product to the norms that the reviewer has established. This is where self-consistancy comes in.

For example, for me handling is as important an aspect of performance as is image quality. If a camera doesn’tfall to hand, then it gets marked down, and some otherwise well-regarded cameras have been so judged on this site over the years. I live in a climate that’s cold 6 months of the year. This means shooting outdoors with gloves on. If a camera has such poor controls that it can’t be used with gloves on, then it gets savaged in my reviews. Live in Panama or Australia? Fine – ignore those comments. But if you shoot outdoors from New York to Berlin, then this may be of significance to you.

I think that any camera that can’t write a raw file in less than a few seconds, or which has such a small buffer that it can’t shoot more than 2 or 3 raw files in quick succession has been poorly designed. I got an email the other day from someone that chided me for criticizing a digicam that takes 15 seconds to write a raw file, because it takes that long to change the dark slide on a sheet film holder. Huh!? (Really – he said that).

I also believe that sometimes companies honestly don’t know that they have a problem,. One example is thePanasonic L1. When I got an early review sample I was appalled to note that it had no dynamic buffer. I phoned a senior company executive as soon as I saw this problem (on the second day that I had the camera), and reported it. He said that he was unaware of the issue. Indeed, the camera started shipping shortly thereafter and it was about three months before there was a public announcement of a firmware fix for the problem. I reported the problem because I believed that it seriously compromised the cameras.

I write this not to vindicate myself over other issues, or seek praise, but rather to make the point that reviewers have an obligation both to their readers and to the companies whose products they report on. When there’s a problem found, especially when I’m one of the first people outside the company to ever see the product, I want to understand what the problem is so that I can write about it intelligently. If the company doesn’t yet know of it, or understand it, then that has to be dealt with as well.

Some have written to me since my initial Leica M8 review to say that as a journalist I should not be in touch with a company when I find a fault. Nonsense. No one would benefit from that. This isn’t politics, where one wants to smear the opponent. This isn’t an adversarial game. This is about trying to provide consumers with usable information as well as a perspective on how a certain product might perform its intended task, hopefully by someone with the experience to provide a worthwhile evaluation. Withholding commentary from the company beforehand would be to blind-side them unfairly and without purpose. But withholding information from readers is also perilous, even if well intentioned.

___________________________________________________________

June, 2007