Background

In our highly completive consumer society sometimes we are presented with so much choice as to make purchasing decisions difficult. This is very much the case when it comes to digital SLRs. The technology is so fast-moving, and also complicated to understand, that even for experience photographers choosing the most appropriate tool for one’s needs can be a daunting process.

DxOhas been in the business of making digital camera testing systems for a number of years. I was involved with them in their early days as an alpha tester of the DxO Analyzer system, and published quite a few camera and lens reviews using it. After a time though I stopped, because of time constraints as well as the feeling that my efforts in camera reviewing would be best applied to subjective evaluations.

DxO then brought out their DxO Optics Pro image correction and raw processing system. This has proven to be very popular and many photographers find it to have become an indispensable tool in their raw workflow, particular with regard to lens correction.

What people may not have realized is that in order to be able to produce analysis of camera / sensor / lens combinations for Optics Pro, DxO Labs has needed to have a rigorous and comprehensive analysis system – one based on their original DxO Analyzer.

DxOMark

Now, in a move that will be hailed by photographers around the world, DxO Labs has released DxOMark, a free web site offering the ability to compare test results from more than 45 current amd past camera models. These test results are taken from the raw images produced by cameras, and are directly comparable between cameras.

Test results are provided in both numeric and graphical formats, and cover metrics associated with Low-Light ISO, Dynamic Range, Color Depth, and DxOMark – which is an aggregate of the other measurements.

Immediately above is an example of the current (November, 2008) Sensor Ranking scores. Only the top 4 cameras are shown, but by scrolling down the list more than 45 other cameras are displayed in descending order. Both current as well as older discontinued models are shown.

To the right of the scrolling list is a graph showing the camera’s scores on the vertical axis and the camera’s date of release on the lower axis. Click the dollar sign though and this graph now shows the lower axis as price, or similarly megapixel count. Pass your mouse over each data point triangle and the camera associated with it will light up on the left hand scrolling window.

In the above example we see that the Nikon D700 scores the highest in terms of DxOMark, offering the top overall ranking along with the lowest price, though it is the number two camera overall.

If we wish to do a similar comparison for colour depth though, it now shows us that the Sony Alpha 900 is the price / performance winner.

DxOMark also allows us to compare two or three different cameras at once, looking at their scores in each of the five areas as well as comparing other salient factors.

Thanks DxO

DxO Labs is to be commended for having created this comprehensive resource for photographers. Providing it free to all is a real bonus.

I would caution though that these numerical comparisons come with two inherent dangers. The first is that reducing performance characteristics to simple numeric values carries with it the risk that unsophisticated readers will use these as gospel. For example – the DxOMark rating for the top camera at the moment is the Nikon D3 at 80.6, while the number two camera is the Nikon D700 at 80.5. What does this 0.1 difference mean? Is it visible? Is it even relevant or just sample to sample variance, since both cameras use the same sensor and support electronics?

These numbers also don’t take into account factors such as autofocus speed, metering sophistication and accuracy, handling and ergonomics, lens choices, frame rates, and a whole lot more. Reducing selection of a camera to a numeric comparison is like choosing a stereo because it scores .001 THD better than another, or a car because it has a 0-60 time 0.2 seconds faster than another. Essentially meaningless.

Don’t get me wrong though. DxOMark will fill a very important role for equipment purchasers – if interpreted and used intelligently. Of course it will also provide wonderful fuel for the flamers and self-appointed pundits on web forums who will create a storm of on-going rancorous debate based on these results. (As for me,I’m heading for the hurricane shelter).

So – thanks to DxO Labs for creating this terrific resource. I would only request though that for it to be truly valuable in this fast-paced industry new cameras be added as quickly as possible so that the database’s usefulness is updated and maintained in a timely manner.

DxOMark

November, 2008