I have removed the article on high ISO Nikon D2X noise, which appeared here briefly on Feb 22.

My review showed a comparison of high ISO noise performance done between a Nikon D2x and a Canon 20D, shot at the PMA show. The Nikon appeared to perform quite poorly in this area, though, as I wrote in my show report, I was very impressed with the camera’s low ISO performance.

While I believe that the results presented in my test were by and large accurate, enough errors in my methodology were pointed out to me that I’ve decided to withdraw the piece until a more rigorous and comprehensive test can be undertaken.

I regret any inconvenience that this premature publication may have caused.

Michael

________________________________________________

An Explanation

or

Though I would rather that it just went away by itself, I do owe an explanation to the several thousand people that read my D2x noise review before I pulled the plug. If you didn’t read it, there may still be a lesson in how easy it is to trip up over faulty methodology, and when working in a hurry.

I made several errors, none of which by itself invalidated the test or its results, but which when combined meant that my conclusions were suspect, and ultimately made me look like a horse’s ass.

The concept was simple. Take four shots with both the Nikon D2x and the Canon 20D, of the same subject, at the same time. Because of the small pixels and high density of the new Sony CMOS chip in the D2x, and because early samples showed there to be some considerable noise at high ISOs, it seemed like the kind of thing that could easily be tested and compared, even in the uncontrolled environment of a trade show floor. Wrong.

My first mistake was that the lens used on the 20D was the Image Stabilized Canon 17-85mm f/4.5. I’d been walking the show the previous day doing many shots hand-held at 1/20th sec of so, even 1/10th sec at wide angle settings, and thought nothing of it. With stabilization, these hand-held shutter speeds are not a problem. I set both cameras to f/6.3 in Aperture Priority mode, to ensure that these relatively slow lenses weren’t at their widest aperture, and therefore either vignetting or too soft.

As you might imagine, the shots with the 20D were very sharp, while the ones with the D2x weren’t. Nothing wrong with the Nikon, other than the fact that it was trying to compete with one arm tied behind it’s back. Of course I should have been aware of the issue, but frankly it wasn’t till after the review was online and I started to get e-mails wondering what the hell was going on that I realized my error.

The second mistake was that I assumed that both cameras would have similar metering capability. It turned out that, whatever the reason, the Canon’s exposures were anywhere from a half stop to a full stop brighter than those from the Nikon. This wasn’t immediately apparent, because with the very contrasty lighting, both camera’s histograms went full range. In other words, there were hot highlights and deep shadows, and so the D2x’s apparent underexposure (at least compared to the 20D), went unnoticed.

It would appear that the Nikon ere’s on the side of not blowing the highlights (at least with the sample camera I used), while the Canon, by comparison, is biased toward opening up the shadows.

Of course this gave a very unfair advantage to the Canon, since noise increases considerably with underexposure.

Finally, I messed up with the files. Because I was seeing results that were so far off what I expected to see from the Nikon, I started playing with the files to see what was going on. This meant my saving many different versions of the files for reference, some with exposure compensation in Photoshop adjusted to open up the shadows by a stop or more. Inadvertently those got mislabeled and put online as the "as-shot" files. I also posted one of the files twice, once as ISO 400 and the second time labled as ISO 800. I then briefly reposted the correct one, only to discover in my haste that this too was the incorrect file. At that point, I poured myself another scotch, and pulled the plug.

By way of excuse (and there really isn’t one), I was working in a hurry because I was eager to start a vacation with my family on the beach in south Florida, after attending the PMA show. I wasn’t as diligent as I should have been. Blame it on middle aged brain fade caused by two tiring days at a trade show.

I could go on, because I made even more errors than this, but at this point I’m annoyed enough at myself to put an end to the self-flagelation.

So. I suppose that the price that I’ll pay from now on for these errors is that every time I publish a review which someone takes exception to, the refrain will be –Oh well, you know Reichmann, the guy that screwed up the D2x noise review. Fair enough. I suppose I’ll just have to live with it.

But, based on the results that I’ve analyzed without these errors, I am confident that when other reviewers start testing the camera at high ISO, it will indeed be found to have noise levels higher than some competitors. So, while my methodology was flawed, it’s likely that my conclusions weren’t. Time will tell.

In any event –mea culpa.

Michael

________________________________________________

Further Thoughts (Feb 25, 2005)

The problem with doing product comparisons, even when one is satisfied that ones results are valid and meaningful, is that of convincing others of their validity. This is especially true when one makes mistakes, which I most definitely did when I produced my quick noise tests on the floor of the PMA show.

But while, as I’ve written, I regret my errors, I stand by my results. As I wrote at the end of the section above –"…based on the results that I’ve analyzed without these errors, I am confident that when other reviewers start testing the camera at high ISO, it will indeed be found to have noise levels higher than some competitors. So, while my methodology was flawed, it’s likely that my conclusions weren’t".

And, what were these results? Simply, that at high ISO settings (800, 1600 and 3200) the Nikon D2x has higher noise than the Canon 1Ds MKII. This is not an unexpected result, from the Nikon having considerably smaller pixels than the Canon. (Because that’s what needs to be done to got that many photosites into such a small sensor).

Because the publishing embargo on reviews by those that have been testing the Nikon D2x, has been lifted as of today, we are now starting to see results. As expected, this is a very fine camera indeed. One that Nikon has owed its loyal customers for nearly two years.

But, what is also starting to be seen, is that as many suspected, and I discovered with my now withdrawn test earlier in the week, the D2x is indeed noisier than the 1Ds MKII at ISO settings of 800 and higher.

Acomprehensive test reportby respected Nikon photographerBjørn Rørslettgives a good overview of the camera, but there is another which appeared today byAlexander Tuftewhich appears to show virtually the same things that I saw.

One difference is that while I attributed the blocked-up shadows to being because of metering differences, it now appears that it may be reduced dynamic range on the Nikon that is causing it. These comparison imagescan be found here.

(Update:Well, it appears that the test was withdrawn just hours after it first appeared, because they were done with an uncalibrated preproduction D2x. Guess we’ll have to wait a little while longer for more definitive results).

At this early stage it isn’t yet clear what exactly is being seen, but evenBjørn Rørslett,who can not be accused of any anti-Nikon bias, agrees that the D2x appears to have reduced dynamic range as compared to the Canon 1Ds MKII.

Regardless, the D2x clearly is a great camera, and I look forward to seeing further test results, especially from those who are not in one "camp" or another. There’s already too much noise online from both the faithfulandthe bottom feeders.

Update:Hardware Zonenow has a well written and comprehensive review of the D2x online.

As the saying goes – the truth is out there– and in the end that’s what matters. Truth.