There are about a half dozen first-rate digital camera review sites on the web. They are written and published by some very conscientious people who dedicate themselves to the task.

This site isnota digital camera review site, though I do from time to time review digital cameras. But I also review film cameras, including medium format, large format, panoramic and underwater cameras, as well as software, scanners, printers, films, papers, monitors, lenses, tripods, flash units, ball heads, bubble levels, books, and maybe a dozen other accessories, peripherals and gadgets associated with photography. I also write tutorials, essay, travel pieces and interviews — on virtually every aspect of photography that interests me. I make my living, not as an equipment reviewer, or though product advertising, but rather as a fine-art photographer, writer, teacher and lecturer. My camera reviews, whether they appear here on this site or in a magazine, are done because the product catches my interest, not because my business is covering every last model to appear out of the Far East.

I’ve been writing professionally for about as long as I’ve been involved in photography, some 30+ years in magazines, and for the past six years on the web as well. I’ll be the first to admit that my reviews are not as comprehensive as those found on some sites. I have no interest in making them so. I also have no interest in detailing every knob, function and button on a digital camera. This information can be found on the manufacturer’s web site or on some of the review sites that do it so well.

My interest is in reviewing cameras from the perspective of a photographer who might use it to shoot anywhere from several hundred to several thousand frames a month. I do so by testing cameras in the field on real-world shoots. For me this means using them in all kinds of weather and under adverse conditions, from well below freezing in the Arctic, to desert heat; in rain, snow and blowing sand.

Lupine Church — Iceland, July, 2004

Contax 645 with Kodak DCS ProBack 645C and Hartblei 45mm f/3.5 Super-Rotator lens

How well does the camera handle with gloves on? What are the cameras’s ergonomics like? How many button presses does it take to activate a required feature? These are the things that I care about, and these are the things that I report on. And, as I’ve written on these pages before, a detailed evaluation of image quality comes only after I’ve satisfied myself about handling. If a camera fails badly in this area, the fact that it may produce high quality images is almost irrelevant to me.

A case in point was one of the 8 Megapixel digicams that I reviewed in the Spring of 2004. It was so poorly designed from a user interface perspective that I found it almost unusable. And I said so in my review. Yet, this same camera tested among the top of its field in image quality, and almost every web and print review that followed ranked it very highly. Some of the reviews made mention of a few of the design defects, but didn’t find them to be that serious. Other reviewers didn’t even mention them at all (which only proves to me that these reviewers can’t be active photographers, or else these flaws would have become readily apparent).

So who’s right? This isn’t really a meaningful question. For someone for whom their camera is a techy toy, who takes pictures on the weekend or on vacation, my comments about design flaws and ergonomic black holes simply aren’t terribly relevant. The camera test sites that concentrate on detailed lines/mm charts and feature comparisons will be much more useful. But I get sufficient (and often voluminous) feedback from active and professional photographers who tell me that my hands-on field evaluations are both informative and useful to them, that I continue with this, my chosen approach.

It’s The Prints Stupid

I evaluate a camera or lens’ technical ability by looking at prints. As a fine-art photographer, what is important to me is whether the device delivers images on paper that meet my (demanding) technical requirements. But the operative word here is "prints". While I look at images on-screen at 100% magnification I realize that this can allow me to see things that are essentially irrelevant in the real world. My written evaluations are therefore based on what I see on prints, not what I see atactual-pixelsmagnification on screen.

Photographers understand this, but regrettably the gear-oriented contingent doesn’t. This is what leads to so much misunderstanding and name calling. One person looks at an A3 print produced with a certain camera and is pleased with it. Another looks at a 100% magnification of a small segment of that same image on-screen (at a size equivalent to a print 3 feet [a meter] across), and sees flaws.

Who’s right? Both are, of course, and regrettable this is what leads to so many on-line punch-ups.

DxO Analyzer

It needs to be said though that this type of close examination does have a role to play. Indeed that’s why I’ve adopted the DxO Analyzer system. DxO has also been adopted by at least two of the world’s largest photographic magazines for generating their test reports, and also by several camera markers for use in evaluation of their own and other’s products in the development lab. People that diss DxO do so either out of malice, or misinformation, usually because its results don’t jibe with their desired outcome.

For me the use of DxO is as an adjunct to my hands-on testing. I always go out and shoot with a camera or lens, for either days or even weeks, before putting it on the DxO test bench. This allows me to draw conclusions about image quality that are independent of the numbers. I then run the test system and generate the charts and graphs, and compare them to what I see. Most of the time what is measured is in accord with my visual evaluation. Occasionally it’s in contradiction.

Why? Because often the things that are measured fall below the ability of the unaided eye to see in normal sized prints. It’s for this reason that I make my evaluations prior to bench testing. I don’t want to allow the numbers to unduly influence my opinion.

Seeing vs. Measuring

This brings us back to the upset that some people express when they see that one camera tests say 0.5dB lower than other, and yet I’ve written that they look the same. Their attitude is that if it measures worse, or if it looks worse at 100% magnification, it must be worse. Well, that’s not the way the real world works. Experienced photographers know this, and move on. The gear-heads thrill at calling names (I guess I’m now doing the same thing. Oh well…) but willingly fail to understand that perspective and judgment are needed when making evaluations, not simply engaging in pixel counting and numeric comparisons.

There’s an analogy from the world of audio. Some amplifiers measure as having 0.005 total harmonic distortion (THD), which is a standard measure of how well an amp handles an audio signal without distortion. But some amps only measure .05 THD, or even worse; yet paradoxically to some, still sound better. This is especially true of tube (valve) amplifiers, even though they measure quite poorly compared to the latest IC amps. The reason is that tube amps produce large amount of even-order harmonics, while IC amps produce much smaller amount of odd-order harmonics. The human ear though is quite accepting of even-order harmonics (it hears them as musical) while it is extremely sensitive to, and irritated, by even the tiniest amount of odd-order THD. In other words, high-end audiophiles know and understand that specs and measurements tell one precious-little about how a piece of quality audio gear will sound. That doesn’t mean — don’t perform the measurements. But it does mean, learn how to interpret what they mean in the real world. The way to properly audition sound equipment is simply tolisten. Photographers need to learn a lesson from this, and spend timeseeingas well as measuring. And hopefully, that’s what my camera reviews are about.

Michael H. Reichmann
October, 2004