In this thread: http://luminous-landscape.com/forum/index....showtopic=24375 Panopeeper showed how expanded ISOs do not correspond to real electronic ISO gains.

I thought however that intermediate ISO values (like ISO640) were also fake coming from software adjustment over a real ISO400 (or maybe ISO800) shot.
But I have been given 4 RAW files of the Nikon D300 and calculated its histograms:

14 bit / ISO400
14 bit / ISO640
12 bit / ISO400
12 bit / ISO640

I was surprised to see that ISO640 RAW histograms (i.e. only non-interpolated RAW data, just scaled from 12/14 to 16 bits without any additional WB scaling, saturation scaling of offset substraction) look the same as ISO400, so I don't have any evidence to suspect ISO640 is a fake when compared to ISO400:

14 bits

ISO400


ISO640




12 bits

ISO400


ISO640



In addition to finding no differences between ISO400 and ISO640, one can see that:
- In 14 bits, R and B channels have holes in the RAW data. Could this slight expansion be related to some WB in-camera adjustment? or just and adjustment to reach the saturation level? if so, why G channel is not corrected the same way?
- In 12 bits, again G channel looks soft, but R and B channels seem to suffer of some periodic level aggregation. Any explanation for this?

The RAW files are available here for download:

14 bits/ISO400 RAW

14 bits/ISO640 RAW

12 bits/ISO400 RAW

12 bits/ISO640 RAW

I too was surprized to see, that the 1/3 stop ISOs appear genuine, with both the D3 and the D300. The Canon 40D derives these from the closest full stop ISO.

Good for the Nikon JPEG shooters.

The contradiction between the green and the other channels is an interesting phenomenon. Stretching of the red and blue obviously purports to fill the numerical range. The question is, why they need to be stretched - or why the green does not need to be?

Correct. Nikon seems to implement all ISO's in a uniform manner with a single variable gain amplifier. Canon uses a two-stage amplification scheme, one amplifier for the main ISO's of 100-200-400 etc; and a second stage of analog amplification for the intermediate ISO's for the 1 series and 5D, or simply software multiplication for the xxD series. Nikon's implementation is superior IMO.



Nikon seems to feel the need to do this "raw white balance", scaling the R and B channels. On D300 sample I was analyzing some months ago, the B channel was scaled by 23/20, the R channel by 13/11. The gaps are then created by the quantization of the rescaled levels when doing this multiplication in integer arithmetic. The motivation for this is unclear; WB in raw conversion would do the same thing anyway, why introduce additional roundoff errors by doing it partially and by a fixed amount in writing the raw data, and then again in the raw converter? I presume the G channel is unaffected since that is ordinarily the way white balance is done -- rescale the R and B channels keeping G fixed.



It is as though the same calculation is being done in 14-bit arithmetic, with the 12-bit data loaded into the MSB's and the two LSB's random. One possible explanation: the Expeed chip in the camera calculates with 16-bit precision according to Nikon press releases. Apparently though the 14-bit raw data is loaded into LSB's or else there would be no gaps in the stretched R and B channels of 14-bit capture. If the 12-bit were loaded in to align with the 12 MSB's of 14-bit data, but the last two bits were not zeroed out and effectively random, then the same stretching by the same factor and then written out as 12-bit data would have the observed sawtooth pattern, as the gaps only occur about every 5 to 7 14-bit levels -- so when truncating to 12 bits, sometimes 4 populated raw levels are binned together, sometimes 3 populated and one empty and the histogram for the binned level is only 3/4 as high. If the stretching calculations were being done with 12-bit accuracy, they would have complete gaps rather than a sawtooth 1-vs-3/4 pattern.

The answer is WB gain applied, it is 1.18x for the red channel and 1.59x for the blue channel. Using 12 bits instead of 14 bits appears to have stretched the data more (although probably not visibly) and this may be further exacerbated by mapping the data to sRGB color space.

And also a good one for Nikon RAW shooters using Auto-ISO, don't you think so?

Thanks for the explanations ejMartin. These machines are getting more and more complicated to analyse everytime, I am loving the simple and honest RAW files from my little 350D more than ever.

The reasons for Nikon to apply any in-camera WB to the RAW files are very unclear to me.

I am unconvinced that Nikon has done this. I believe the WB gain is recorded as an EXIF header and it is up to the RAW converter to interpret it. I often use "uni-WB" on my Nikon DSLRs (D200 and D300) and I get the same colors using that as I do with Auto and Preset WB settings if I use a gray card to reference WB during conversion -- although this depends on the RAW converter, ACR seems to screw it up and NC4 sometimes gave me odd results with my D200; but Raw Magick Lite with the D200 and Capture One LE and NX with the D200 and D300 have always given me consistent results.

The histograms I displayed are pure RAW (obtained from DCRAW through the dcraw -D option), so EXIF is completely ignored since no processing is done to the RAW data which is just read from the RAW file and decompressed if needed.

Commercial RAW developers are simply a pain, and most times unsuitable for any serious analysis.

Should I use NX for my D300 Raw conversion rather than ACR or LR?

Steve

Think of it as a "pre-WB". It's just a fixed rescaling (that according to Panopeeper, seems to adjust with the firmware version) of the raw data coming off the sensor before it is written to memory card. Any white balance done by the raw converter is on top of this and independent of it, and of course varies with the illuminant (light source) and its color temperature, for which the camera records its best guess into the metadata.


... and so Steve, use any raw converter you like.

There is no such thing as WB gain without actual white balancing. The stretching of the red and blue channels is constant, it has nothing to do with the actual light.

Upon closer review of the original 12 bit ISO 400 file I see that there was no WB gain applied to the histogram. Therefore, it appears that bit depth is the culprit since it is the only thing that is actually different in the 14 bit file that does not have the spikes.

This is like asking which film is best; there is no one answer to that, often not even for a single person no less any two people. Personally, I prefer the results of NX D2XMODE2 conversions most of the time with my D300. I have to say that I'm very frustrated right now with Raw Magick Lite because it's been several months since the D300 came on to the market and it is still not supported -- with my D200 I consider Raw Magick Lite conversions the best, albeit the precision comes at a huge price in terms of a cumbersome UI and the conversions are slow (they take 3-5 minutes on my computer), but I often can be editing one file in Photoshop and browsing the web while the next one is being converted which suits me just fine.

I have never been happy with the results of ACR for my Nikon cameras. The colors seem off and my D200 files were much noisier at higher ISOs than what I got from every other converter I tried. I also found myself spending an inordinate amount of time tweaking the conversions and the resulting files. The speed that NX processes the files is just as fast with NR turned off and I spend far less time making adjustments. Finally, the costs of upgrading CS2 to CS3 or to start using Lightroom to handle my D300 files was completely unjustifiable for me since I got a free copy of NX and Capture One LE upgraded for free -- it is just not worth paying $300 to use ACR with my D300 and I can't stand DNG (the colors look awful).

Converting to DNG shouldn't have any impact on the output from ACR compared to using the original NEFF. However, it's clear that different raw converters handl;e colour differently and some better than others:) I found that LR/ACR is much improved by calibrating for the particular camera (canon user though so...).

Mike

I bought a Color Checker to calibrate my D300 for LR/ACR, but the script in CS3 to use the color checker image produced large deviations to the Camera Calibration settings which resulted in worse color rendition. Am I missing something. Is there a fool proof method to calibrate in LR?

This WB pre-conditioning is not a new thing for Nikon. They have been using it for long. It exists also in the D2 series cameras (and I believe in D200 also).

Maybe one reason might be to attempt to present a more 'uniform' sensor response (sensor metamerism differences not withstanding of course) to the RAW converter / jpeg engine independently of camera model and sensor technology used?

Experience certainly shows that Nikon appear quite consistent in colour response across their model range. Additionally, use of the same colour curves or camera presets seem to provide quite a consistent look across the model range.