I see. Your role is to quote and link without any personal understanding.



Really? A discerning reader would look at the logic and quality of the arguments, critically. Test these things for yourself. Why would you want to trust someone else for something you can test yourself?



That doesn't impress me to the same level it apparently does you. The world is full of people who fudge their way through their careers. Statistically speaking, he is likely to know more than someone who has not taken such a career path, but it does not license him to guaranteed truth.

I am impressed by quality of argument, and depth of thought. Do you think that there are no errors in 179 scientific papers? Do you think that no one objected to any of his ideas?


The quality of my arguments is my qualification. There is nothing mysterious, either in what I say. You can test most of it for yourself.

If you can't understand them, then go idolize an "expert" who jumps to conclusions.

ACR does not do any highlights recovery. Correcting the Exposition down and therefore apparently recovering blown areas is not recovery at all, is simply compensate the blowing effect of white balance scaling, with the scaling provided by Exposition compensation to the left.

Your camera's histogram is pesimistic, as it will show blown areas where in the RAW file they have not still begin to blow. Reason? it's an histogram calculated on a white balanced RAW file using greater than 1.0 multipliers.
I see you use DCRAW to develop, just do this experiment: take the first RAW file in your series that started to appear blown on some channel in your camera0's histogram. Now develop it with -H 1 or better -H 2 options in DCRAW that ensure through smaller than 1 multipliers not to blow anything that was not already blown in the RAW file. Surely there will be nothing blown.

So if your camera's histogram shows nothing blown, no channel is. If it does, you don't know what you will find in the RAW file until you develop it in your PC.

Maybe we have to define what highlight recovery means.

ACR can build data in highlights IF one of the three color channels has data. If all three are blown out, nada.

John,

Thus far all the data you have presented is one graph showing noise characteristics of a camera with results similar to those obtained by others. Then you draw conclusions that, in my view, are not supported by the graph. Any scientific paper has a materials and methods section where the experimental technique is described in sufficient detail to allow others to replicate the experiment and confirm the results. If you look at Roger's web site, he describes his methods in detail and gives references in the scientific literature to support his conclusions. Please direct us to an explanation of your work.

Science is not logic and the validity of a hypothesis can not necessarily be determined from the quality of the argument and the depth of thought. You need data and need to show how it was obtained.

I have worked through many of Roger's methods with my own camera. However, thus for you not described any methods that I can verify. Thus, while you seem to know quite a bit, you have less credibility with me than does Roger. Let other forum members draw their own conclusions. It they want to set their cameras to ISO 3200 and lose dynamic range.

Bill

If you have an RGB histogram, and nothing is blown in it, then probably nothing is blown in the RAW, but if the histogram is a luminance one, blown red and blue channels can easily be missed by the histogram, as they have weak luminance weighting.

Whether what ACR does should be called recovery is a matter of semantics. With daylight white balance, the green channel can show severe clipping of the highlights when the blue and red channels are still intact. These tones in the green channel are lost and ACR does more than just scale back the green channel, but rather rebuilds it from intact data in the other channels using an algorithm to prevent color shifts as much as possible. This feat was demonstrated in the analysis I did on Digidog's overexposed file. Bruce Fraser and some other knowledgeable and articulate people have called this recovery. Your mileage may vary.



With Nikon cameras it is a simple matter to upload a custom white balance to the camera with the red and blue multipliers set to 1.0. The resulting RGB camera histogram gives a good preview of what is in the raw file without resorting to one's computer. If the camera has several banks in which to store camera settings, one of these can be used for the UniWB (coined by Julia Borg, a Nikon guru). This UniWB does mess up Adobe Camera Raw; when it sees Red and Blue multipliers set to 1.0, it thinks it is dealing with a multiple exposure. This problem can be addressed by making the coefficients nearly 1.

Bill

Let's put it this way. Blown highlights was more of a problem for me before I started using ACR several years ago. Canon's ZoomBrowser and othe popular RAW converters like BreezeBrowser could not do nearly as good a job as ACR in 'so-called' highlight recovery.

I remember being amazed at how much more detail in a grey sky I could recover using ACR. BreezeBrowser at that time could not compete, in my view. I don't know what BreezeBrowser is like now. I'm sure it has improved.

One RAW converter I still use is Raw Shooter Premium. I like the ease with which I can get solid, vibrant colors with almost a painterly effect, yet still retaining full detail. However, when it comes to recovering detail or color in a blown sky, ACR is marginally better than RSP.

It might well be true that DCRAW is even better at reconstructing lost data. I'm led to believe it is, from comments from John Sheehy and others and yourself.

But it's not a programs that's easy to use, is it? I get the impression it's a program that appeals more to computer programmers.

Yes, this is what you can read almost everywhere about ACR, but I never saw an example where it was clear that ACR recreated a blown channel (mean blown even in the RAW data) with the information of the other two. "This feat was demonstrated in the analysis I did on Digidog's overexposed file"-> where exactly can I find this analysis? I would be interested in looking at it.

What DCRAW calls recovery, despite the results can be better or worse, is real interpolation of values in a channel according to values in the same channel but in non-blown surrounding pixels.

Something (very argueable) that makes me think DCRAW does something more than ACR is that if you activate recovery on DCRAW you clearly see an increment in the processing time (in fact DCRAW tells you when the recovery stage begins), while ACR always seems to take the same time to develop a RAW, no matter how much highlights were blown or not.

This is a sample of an image with important enough real blown areas (in the RAW data) in the green and blue channels:

(developed with -H 1 in DCRAW to avoid WB clipping and be aware of real blown areas):
.

.


ACR produces (look at the water fall and the shiny rock) disgusting final tones. Shouldn'd it recover the blown channels from the red one?
On the other hand -H option in DCRAW produces a pleasant highlight recovery specially in the water fall. In this case -H 9 was used; with -H 6 the tone on the rock becomes more pleasant (less red).

ACR with -3EV exposure correction to minimise highlight blowing:


Developed using DCRAW and -H 6 recovery option:




The 1.0 EB multipliers on the Nikon is a cool option. With no doubt I would make use of it.


Best.

Have you tried ACR 4.1 with either Photoshop Elements (4.01/Mac or 5.0/Windows), Photoshop CS3 or Lightroom 1.1?

Already in Lightroom 1.0, the "vibrance" slider seems somewhat similar to a Raw Shooter feature with the same name. I can't say whether it works in exactly the same way, because I stopped using RSE shortly after starting to use it.

(I'm not saying that ACR 4.1 replicates what you like about RSP, but I'm wondering whether it does.)

Jani,
I have Lightroom 1.0 but don't use it, and I tried the CS3 demo until it expired. I'll eventually upgrade to CS3. I found the vibrancy slider very tame in ACR in CS3, however.

There's something about that combination of 'hot pixel/pattern noise suppression', 'noise suppression', 'detail extraction' and 'vibrance' in RSP that's difficult to emulate in ACR. There's a quality there that seems just right for some images, but not all images.

My analysis of Andrew's ETTR files was posted earlier in this thread. One can use the same images to demonstrate highlight recovery in ACR.

Here is a histogram of the green channel of Andrew's image taken at 1/50 sec @ f/16 from the white portion of the target. The image was rendered with DCRaw and the histogram is from ImageJ, which supports 16 bit histograms. The highlights are totally blown.

Click to view attachment

Here is the histogram after Andrew's highlight recovery. It is apparent that the green channel has been reconstructed in the highlight area.

Click to view attachment

Bill

I agree that a reliance on popularity can be misleading. But the fact is that arguments are not enough. Science is based on data. I've read through your data post a few times and must admit I have a hard time deciphering it to reconcile with your conclusions (despite a medical degree and a neurobiology PhD that was founded on digital data acquisition).

Maybe you could expand on the data you collected and explain (or at least link to) how you reached the arguments that you now propound.

Without *any* data from myself, I can still challenge Roger Clarks' conclusions, based on his data. His data does not support his conclusion, IMO, except as weak circumstantial evidence. He sees a tapering off in improvement in read noises as measured in electrons, going from ISO 800 to 1600, and then no improvement at all for ISO 3200 (no surprise there, as the camera doesn't really have an ISO 3200 amplification). Since 1 electron is approximately 1 ADU somewhere just below ISO 1600, he concludes that it is because of "unity gain", a point past which there is no further gain from more amplification. He supports this with "logic" that says that it makes no sense to count beyond unity gain, because electrons are discreet units and you only need one unit to count one. That logic would be fine, if cameras were actually counting photons-turned-electrons, and I wish that were true, but THEY ARE NOT COUNTING ELECTRONS. They are digitizing an analog amplification of the discreet electron counts, blanketed in camera-generated noises with an infinite number of in-between analog values from at least a couple of sources before the signal becomes binary data.

The variance due to read noises in apparent electron counts is tremendous at the ISOs near "unity gain". The situation is not even remotely close to "counting electrons" with any degree of accuracy, at the pixel level. Here is a histogram of a blackframe from the 20D at ISO 1600:



The variance here is due to read noises picked up throughout the signal chain, including noise introduced by the ADC unit. The noise introduced by the ADC unit, alone, would be cut in half, relative to signal, if there were a true gain-based ISO 3200.



I can only guess what you're refering to, without your being specific. I have said many things.

The point of my chart, if that is what you are refering to, is that the values at ISO 3200 are *NOT* a natural progression of the curves from ISO 100 through ISO 1600. The curve for "total noise" from 100 to 1600 is the closest one to actually suggesting the ISO 3200 value, but the 3200 value is still an abrupt flattening of the curve by a small degree. Everyone who has ever tried to use the shadows of Canon DSLRs at high ISOs knows that one of the most visible aspects of noise is the banding or line noise. Its statistical strength is very small, compared to its visible strength. Subtracting the banding component from a blackframe decreases the standard deviation by almost nothing. With my 20D, it lowered the standard deviation at ISO 1600 from about 4.7 to about 4.6, yet the visible improvement is much more than the numbers would suggest.



It's more a matter of *not* jumping to the conclusions that Roger Clark does, IMO. My interpretation of his data and mine combined suggest to me that it may be possible to get less read noise in electrons with greater and higher quality amplification. None of the data contradicts that possibility. The "unity gain" theory totally ignores the fact that the real limitation to low signal quality is camera-generated read noises, which have been demonstrated to decrease, relative to absolute signal, at higher amplifications and with improved technologies. It is currently of no practical value, whatsoever. If cameras *were* actually counting electrons, then of course, there would be no point in amplifying to an extent, or digitizing to such a depth, where each electron was represented by more than 1 ADU, but this is the real world, and read noises are a real obstacle to counting electrons.

Your histogram of the dark frame (bias frame) is similar to that determined by others, for example Christian Buhl. However, the bias frame does not represent read noise, since it also includes fixed pattern noise derived from non-uniformity of the light field (e.g. due to vignetting in the lens), dust specs on the sensor, pixel to pixel sensitivity variations and other effects. To get the true read noise, you must subtract the fixed pattern noise as Christian points out in the above link. The horizontal and vertical banding noise on which you dwell is one type of fixed pattern noise. Since you do not describe your methods, I have no idea of what you are doing and whether or not any of your conclusions are valid.

The primary component of read noise is is from the on chip amplifier (see here, read noise). If you increase amplification, this noise would most likely increase also and you might not gain much. The noise introduced by the A/D converter is relatively minor and can be reduced by increasing the bit depth of the A/D converter (as Roger points out for the new EOS 1D M3, which uses a 14 bit device).

The relative contributions of photon noise to read noise at unity gain vary with signal strength. With higher signals, the noise source is mainly photon noise, while read noise predominates at lower signal strength. You did not state what signal strength was involved in your statement about read noise at unity gain. Of course, with a dark frame with a short integration time, the noise is almost entirely read noise, since there would be no photon noise and thermal noise would be minimal.

To use Roger's data for the 1DM2, full well at ISO 50 is 80,000 electrons. At ISO 1600 (unity gain), full scale on the A/D converter would represent 2500 electrons. The variance of photon noise would be 2500 and the variance due to read noise would be 15.2. If you look at noise in the shadows, for example 6 EV down, one would collect about 40 electrons. The variances of the photon noise and read noise would be 40 and 15.2 respectively. Photon noise still predominates.

Furthermore, when one determines read noise, one is measuring total noise and the various contributing components can not be separated out, and your statement "The noise introduced by the ADC unit, alone, would be cut in half, relative to signal, if there were a true gain-based ISO 3200" does not make much sense since you can't separate out amplifier noise from ADC noise.

Finally, the proof of the pudding is in the eating, and you should measure noise at various ISOs rather than theorizing at what they might be. Astronomers who have done their homework apparently do not bother with amplification much above unity gain, since it has no advantage with respect to noise and merely reduces dynamic range. The unity gain of the Canon cameras is relatively high, and the differences might be demonstrated more easily with a camera with a lower unity gain (such as the Nikon D200 with a unity gain of 800).

mmmm I see, you are totally right. ACR recovered both the red and green channels from the blue one. I thought highlight recovery in ACR was a fake from Adobe, glad to see I was wrong.
Thanks for the info Bill.

"Light field"? Hello?



Vignetting in the lens? Hello?



How does the sensor know that the specs are there? Are they telepathic?



Sensitivity to what? Do you know what a "blackframe" is? It is the noise of having to read a sensor, with no light having exposed it.



No, the read noise of a black frame is the noise of the blackframe. The *RANDOM* component of a blackframe, is the blackframe minus any fixed pattern noise. The Canon 20D has no significant fixed pattern noise in 1/8000 second black "exposures". Stacking 16 of them yields 1/4 of the statistical noise, and no non-random patterns appear. I've taken 16 exposures interleaved with 16 blackframes, stacked all the exposures together, stacked all the blacks together, and subtracted the black result from the exposure result and all I got was a 40% increase in random noise. It takes a couple of seconds at ISO 1600 for fixed-pattern noise to become measurably higher.

The problem with your understanding is that you are not familiar enough with these issues to have a clear picture in your head of what the terms are referring to.

You see me write the word "blackframe" and the only thing you can think of is astrophotography, with its long exposures, and reply as if I were talking about stacking many long exposures, subtracting fixed-pattern noises, and flat fields. None of that has anything directly to do with how much amplification is worth using for readout. In fact, if other noises were present which I needed to subtract out, then the curves in my 20D noise profile graph would be *STEEPER*, not flatter, as the curves approached ISO 1600. Do you understand that? Noises in the sensor itself have the same intensity in "electrons" at all ISOs; unsubtracted fixed pattern noises would compress the differences in measured noise from single frames at different ISOs, as measured in electrons!

Roger often mistakes line noise for fixed-pattern noise; the line noises I speak of *DO NOT* repeat from frame to frame; they are in a different place with a different pattern, in *EVERY* frame. No matter how many times I have told this to him, he goes on again to call the line noises "fixed pattern" noises. They are random 1-dimensional noises. In some cases, there is a periodic component, but the phases are totally random.



I have described many times to you what I do. I take blackframes (NOTHING TO DO WITH ASTRONOMY!) at a short exposure at all ISOs, load each RAW file into IRIS, window the display to 0 to 255 ADU, save out into a BMP, load into photoshop. I get the standard deviation of the blackframe from the histogram tool. That is the "total read noise". Then I make a copy of the blackframe, and resize it with bicubic so that it is one pixel wide. Then, I resize it again to the original width (not really necessary, but I may want to subtract it from the original), and get the standard deviation. This is the standard deviation of the horizontal line noise. Do the same vertically for the vertical line noise. Very simple. I would have included noise data for the original with the line noises subtracted, but they are almost exactly the same as the original, and would not be distinct on the graph.



That's accurate with older CCD cameras. The cameras Roger draws his conclusions from are not older, or CCD.



We are discussing at what point it becomes worthless. Saying that it will become worthless at some point is stating the obvious. I am saying that there is no evidence that read noise can't be reduced further than it is now by higher and/or better amplification.



There is no evidence of this happening in the 1Dmk3. The Ratio of max RAW signal to read noise is almost exactly the same on the mk2 and the mk3 at ISO 100. Low ISOs are the ones with the largest percentage of their noise contributed by the ADC, and ISO 100 has no improvement. That means that it is likely that there is no real improvement in ADC-contributed noise in the mk3. ISO 3200 on the mk3 has a read noise of 6.0 ADU compared to 9.4 ADU on the mk2, but the benefits are fully there even if the RAW data is quantized to 12 or even 10 bits.

Your post is very difficult to read, since you did not get the quote/unquotes right.

Anyway, I will respond to a few of your points. If you are determining the total noise in an image, you need to take the shot noise, read noise, and dark current (thermal noise) into account. With short exposures (up to several seconds), the dark current is negligible and may be ignored. You dwell on read noise, but photon noise is dominant in most situations. Photon noise may not be a wild card, but it needs to be taken into account. If if overshadows read noise by a large margin in a particular situation, read noise can be neglected for practical purposes, just as we are ignoring thermal noise.

To analyze the photon noise, you need some light falling on the sensor. If you merely take a picture of a supposedly uniform field, the light falling on the sensor may vary due to the factors I mentioned (taken from Roger's web site). These factors are fixed, and can be eliminated by subtracting images taken under identical conditions as Roger explains. For read noise, lens vignetting, dust on the sensor, etc do not apply since the image is taken in the dark with the lens cap on (and I should not have mentioned them in the context of read noise).

If you are analyzing a black frame for read noise (yes, I know they are different from a dark frame to determine the dark current in the long exposures used in astronomy), you must subtract fixed pattern noise. Your understanding of read noise is different from what Roger uses or from Christian Buhl's concept: "Subtraction of an offset image and of the fixed pattern structure for the EOS 10D. It is the true readout noise "image" of the camera." You allege that I don't have a clear concept of the terms involved, but you make up definitions on your own.



This is the first time I have seen an explanation of your methods in such detail. I don't know why you save in 8 bit format, since you lose precision. I multiply the results by 16 to convert 12 bit data to 16 bit format. The standard deviation of the black frame is not the read noise, since it includes fixed pattern noise. To get read noise, you must subtract a bias frame as Christian explains. That is also what Roger does.



You jumped to the wrong conclusion.



Yes, I was using only the green channel to simplify things. A full analysis would require all channels to be evaluated.



Not really. The ADU is related to the number of electrons as described by the camera gain, which is in units of electrons/ADU.

What do you guys think of this?

Click to view attachment

Top, minus 1 stop over flash meter recommendation (ISO 400)

Middle Normal

Bottom Plus 1.5 stops, normalized with exposure slider in Lightroom (-1.46).

Oh, I should add, small section of a big image with many different targets. New images, shoot with strobe (very even lighting according to flash meter). Did ISO 100/400 and 800 (this is but one small example). Bracketed with strobe back NOT aperture (that affected the results too much).

On my 5D, 2 stops over was blown out, couldn't recover less than 100% on whites with exposure slider in LR. 1.5, over, no problem.

Andrew,
I've always found ACR can do a spectacular job recovering greys. A grey sky or a grey concrete wall with fine texture seems to be able to accept more exposure before clipping than a scene with color. A particularly common indication of clipping is a deep blue sky that shows an obvious shift towards cyan.

It was mentioned earlier in the thread that ACR always applies a white balance to the conversion and that this apparently is responsible for a degree of clipping. There's no option, as far as I see, for 'no color balance' in ACR.

I'm not sure if the following is relevant, but recently whilst experimenting with a calibration of my scanner with an old IT8 target using Vuescan software, I noticed a dramatic change in the histogram when selecting 'no color balance'.

The following 2 previews of a slightly overexposed Ektachrome compare 'no color balance' with 'white balance'. Whatever characteristic of WB I select, neutral or landscape etc, the histogram is always pushed to the extreme right, with this particular overexposed slide, but with no WB the histogram is very far from the right.

Click to view attachment Click to view attachment

This is the second half of a reply I made last week; for some reason, the Quote feature won't work with the entire post, but does broken in into two halves. Apparently there is a limit on the number of Quote delimiters in the software.



Perhaps that's because it is irrelevant. Shot noise is not a wild card; shot noise is dependent only on the geometry of photon collection and quantum efficiency, including the effects of filters. Read noise is the thing that varies greatly from one camera to another, and its relationship to ISO varies greatly on any given camera. None of the things that you do to improve read noise have any effect on shot noise. The less read noise you have, the more usable your shadows become.



You've just described my blackframes! Why all the confusion at the beginning of your reply?



No; that's not Roger's data. Roger is well aware (pun intended) that RAW saturation at ISO 100 is not half as many photons as well saturation at ISO 50. ISO 50 is really only capable of about ISO 70 to 75 on the 1Dmk2, but is exposed like it is actually ISO 50, leaving less highlight headroom than the other ISOs, relative to metering (a major engineering blunder, IMO). RAW saturation at ISO 1600 on the 1Dmk2 is approximately 52300 (by his data) divided by 16 = 3269 electrons. I don't know if his values are correct, though, because he may assume that the 1Dmk2 has 4095 RAW levels to use, but signal is only represented by ADU 128 through about 3700+. He seems to derive either ADU:electron "gain" or max photon count from the other, using the uncertain figure.



No. Six stops down from saturation would be 3269/(2^6) = 51.1 electrons, which would have a shot noise of 7.15 electrons. Total read noise is about 4.19 electrons. Read plus shot would be (51.1 + (4.19)^2)^0.5 = 8.29 electrons, or about 0.21 stops more noise. How far down is Sat-6EV, though? There is about 3.5 stops of headroom above metered grey in most Canon DSLRs (4.0 for the XTi). That puts Sat-6EV about 2.5 stops below middle grey; not very far into the shadows at all, and that's just the green channel. A chain is no stronger than its weakest link, and the red channel is 1 stop less sensitive in daylight WB, so Sat-6EV is only 1.5 stops below "middle red". Change the WB to tungsten, and the blue channel is only 0.5 stops below "middle blue", with Sat-6EV!

Now, let's talk about some *real* (but still not extreme) shadows: 4 stops below middle blue in tungsten lighting - that's Sat-9.5EV. 3269/2^9.5 is 4.51 electrons. That's a shot noise of 2.2, against a read noise of 4.19. Total noise is (4.51+(4.19)^2) = 4.7, or 1.09 stops more total noise.

Not to mention the fact that the read noise has highly-perceptible line noises not present in shot noise of equal intensity.



It is not clear how much of the noise is due to the ADC, but ADC noise is real, and whatever part it is, would be reduced 50% relative to signal, a small, but real improvement. Total noise at ISO 3200 would be (non-ADC-1600-noise^2+(ADC-1600-noise/2)^2)^0.5. You don't even need any more at-sensor gain to get this small benefit, so the issue of the point of diminishing returns of sensor-read amplification is another issue. Some manufacturers use amplification of some kind for ISO 3200; the Minolta K7 a case in point; it's ISO 3200 has less noise than 1600 under-exposed by a stop.



I did; what are you talking about?



That's a very vague statement, and it may apply to old technology, and/or equipment that doesn't have any gain above unity gain, because the manufacturers believed the same myth, or technology wasn't ready.

The bottom line, which you seem to ignore, is that the steps of the ADU are arbitrary relative to the analog equivalent of an electron.





Go ahead, demonstrate it, and I will offer alternate explanations for your results.

It is fixed now. I had to break the post in half for the Quote feature to work properly. I did have a pair of quote delimiters reversed originally, but even after making sure they were all correct, it still wouldn't post correctly.



I was determining the total noise of a blackframe at a short exposure. There is no shot noise, and there is infinitessimal dark current noise in a short exposure.



Photon noise is dominant in the highlights, and read noise is dominant in the shadow areas. Read noise is dominant in determining ultimate DR.



Dark current noise is infinitessimal in a short exposure, compared to read noise.



To measure read noise in a blackframe? Blackframe read noise is a noise that has no relationship to signal, whatsoever, other than the fact that clipping the signal will also clip away the read noise (and any other noise).



Only the 8 LSBs are used in a 20D blackframe. You could probably count on one hand the number of pixels above 255 ADU in a blackframe from a 12-bit RAW.

BMP is the only lossless format that IRIS outputs without loss in a file format understandable and unmangled by photoshop.



In the Canon 20D, fixed pattern noise is infinitessimal in short exposures. That is one reason why Canons are so relatively noise-free in the shadows at high ISOs; the readout circutry is aware of any *consistent* significant offsets from black.



Does for what?

I've been playing with Canon RAW data long enough to know that there is no fixed pattern noise in any short exposure that has any visible effect, even in very under-exposed images pushed.



You can measure any two quantities and calculate their ratio, and give that ratio a name. That doesn't mean, however, that there is some special significance to "Unity" .

Here are some parallels:

"It's not worth driving at a speed of less than a mile a minute; you won't get anywhere that way."

"All resistances in the circuit less than 1 ohm should be removed, as they offer no resistance".

I could go on and on, but the point is that there is no empirical relationship between ADU steps and amplified electron counts clouded by analog read noise. All you would need to do is change the bit depth of the camera's RAW output to determine the maximum usable amplification at the initial readout. The 1Dmk3 does just that; it gives a 4x boost to the "ADU:electron gain". If "unity gain" was at ISO 1200 on the 1Dmk2, then it is now at ISO 300 on the 1Dmk3! If you think about nothing else at all, think about that! By the same token, just dropping bits off of any ADC would make it worthwhile to use greater amplification at the inital read!

There is nothing about the amplification levels at the photosite (CMOS) that has anything directly to do with electrons per se, nor with the gain of the ADU. Somehow you think that the camera knows about electrons, and their signal magnitude. The ADC is only taking an analog signal which has no spiking in its virtual analog histogram, and digitizing it. In order for the original signal to be preserved as best as possible, the noise needs to be so small that individual electron counts would have spikes; then we could talk about actually counting electrons. Since we can't count electrons, what we need is amplification that gives the least noise, no matter how many ADU that turns out to be.

Unity gain is an arbitrary ratio with current levels of analog noise.

Please don't, since your parallels are not particularly well chosen



You do not seem to understand the concept of unity gain. As Roger explained, if you use unity gain to compare the sensitivity of sensors, you must express it in the same bit depth.

Yes, with a unity gain at ISO 300 with the 1D M3 you could leave the ISO at 300 and shoot in dim light without any need to adjust the ISO to a higher value. If you increased the bit depth to 17, you wouldn't even need to adjust the ISO at all, since you would have captured all the information at base ISO. This assumes that the read noise at base ISO is kept under control. With the 1DM2, Roger's data show that read noise increases considerably at at base ISO as compared to ISO 1600. With the Nikon D200, the increase is less marked (up to 10 electrons from 7.4).

First off, I'd like to point out that John Sheehy is absolutely correct in that the camera sensors mentioned by Mr. Clark do not count individual electrons.

Counting single electrons is so hard that it was first done just over two years ago by Swedish physicists.

If John Sheehy doesn't understand the concept of unity gain, it may be because it's practically impossible to have a setting in the camera where it can count single electrons.

If bjanes wants to prove Roger Clark right (or he wants to do so himself), then my guess is that there's a Nobel Prize in physics waiting; doing this at room temperature would be a significant leap forward.

I expect lots of density in discussion of things like politics or religion, but stuff like this is puzzling. How can anyone, aware that the noise in the cleanest of systems varies in a smooth, analog manner over a couple dozen electrons, think that counting of electrons is occuring.

On top of the lack of any logic in the concept of unity gain as a limit in a noisy analog system, the 1Dmk3 totally contradicts the concept of unity gain. Even if all ADUs must be converted to 12-bit to qualify, the 1Dmk3 at ISO 3200 has an ADU_12:electron ratio ("gain") of 2.18; far above "unity gain", yet the amplification is immensely useful.

All they are chosen for is their demonstration of how arbitrary unity can be. They are good examples.



I do understand it. I understand it to be false and simplistic.



So, what's the magic bit depth? 12, because it's one of God's favorite numbers?

Maybe we can get back to ancient Greek cosmologies while we're at it; Pythagorean harmony between bits and electrons.



The point that you and Roger seem to ignore is that it is the increased amplification at the photosites that reduces the noise in electron units.

Obviously, just counting electrons would be the ideal; ISO would only be a way to tell the camera how to expose, and how bright to make the review image or the default JPEG.



You could double-check yourself. Just shoot short blackframes and load them into IRIS, and issue the "stat" command. Look at the Sigma value. That is the standard deviation of clipped black. To get the true noise, multiply by 1.725, which will result in what a symmetrical, unclipped standard deviation would be.

Take each of these values, and multiply them by the number of maximum electrons at RAW saturation, and divide by 4095 (yes, the D200 uses all 4096 values). I don't know the exact saturation of the D200, but I would guess about 30000 electrons at base ISO. The correct number is not required to compare ISOs, though.

What is the origin of this number?

I took a bunch of symmetrical gaussian noise samples (and fabrications in PS), measures their standard deviations, clipped them in the center, and measured again. 1.725 was typical of the ratios in the useable range.

I've just been looking at some D200 RAW files, ISO 100 and 1600, and it doesn't seem like those values are likely. It would be interesting to see what values you get for blackframes from the D200.

Or anyone with a D200.

No one has claimed that the camera is counting electrons individually. However, at high ISO we are working with relatively few electrons in the shadows. As an example, the table below is for the Nikon D200 and is derived from Roger Clark's sensor analysis of that camera.The table shows data for ISO 1600 from highlights to shadows in decrements of 1 f/stop. The number of electrons collected, the shot noise, the read noise, and total noise are shown along with the signal to noise ratio. All noise values are in electrons.

The full well capacity of the camera is about 33,000 electrons. However, at ISO 1600 the full well is not utilized and only about 2043 electrons are collected at full scale on the ADC. (output value = 4095 or thereabout). Seven stops down in the shadow area, only 16 electrons are collected. The total noise is 8.4 electrons and this represents the noise fog to which John Sheehy refers. Since the gain is 0.5 electrons per 12 bit ADU, the ADU output would be 16, corresponding to an 8 bit gamma 2.2 value of 28. This is Zone 8, and near where the blackpoint would be set in a normal photograph. We are not counting individual electrons, but are coming close. Since unity gain for the D200 is at ISO 800, we could have left the camera at ISO 800 and increased exposure in the raw converter. That wouldn't win a Nobel prize, but it might us to make better use of the camera.

That would be necessary, though, for Unity Gain to be fully meaningful. With a great deal of uncertainty in the counting of electrons, the only thing that should limit the usefulness of amplification is no further decrease in analog noise, as measured in units of electrons.



It would be nice if someone measured D200 blackframe noise at all ISOs in IRIS, which loads literal RAW data. I am skeptical about the software and flow that Roger uses for measuring RAW statistics.

10.0 to 7.4 electrons seems pretty low; that would make the D200 as clean as the 1Dmk3 at black at ISO 100, and I just don't see that in the RAW images. As I zoom into the D200 shadows, I see the noise become visible much quicker, and in out-of-focus deep shadow areas, the read noise seems to be about 3 ADU, which is 33000*3/4095 = about 24.2 electrons, a more typical value for a DSLR at ISO 100.



I don't see how you come to that conclusion. Do you know what a sigma of 7 or 8 electrons means? It doesn't mean that most counts are right, and occasionally one will be 3.5 - 4 electrons off. If the average signal is 15 electrons, you will have many pixels clipping at zero, and going as far off as 40 or 50 electrons worth of analog signal. It's a big mess.

And as I've said earlier, these RAW levels are not what you might think they are. The top stop isn't even used in the green channel with matte subjects in even lighting, at the rated ISO, and even more isn't used in the blue and red in daylight, and three stops get clipped away normally in the blue channel in incandescent. That puts middle blue at 6.5 stops below saturation in incandescent light. Sure, you can expose a stop to the right, but you are no longer actually at the stated ISO. Every camera has a highest ISO, and to use Av and Tv values that are needed may mean shooting way down in the RAW signal, in at least one color channel.



You could probably push ISO 400 to 1600 as well, with little loss, but the converter must not treat the midtones as shadows, as ACR does, for example, if you leave the Shadows slider at 5.

I don't see "unity gain" explaining anything here. If the results are the same (except for mild quantization with the pushes), then the real gain at the first stage may very well be the same for all ISOs, and the D200 doesn't do unity gain, but only 1:8 gain.

Within limits I think expose for the shadows and dev for the highlights works best for me.

Kevin.

Back to the beginning .

I think you're asking whether shadow detail can be brought out, in an underexposed image, by using the +/- exposure sliders in ACR.

I think the answer is both yes and no. The appearance of the shadows can be lightened and more detail made visible by using the Exposure Compensation slider during conversion, just as it can with many other techniques in Photoshop, such as use of curves.

But as far as I know, there is no recovery of shadow detail in the way that a minus adjustment of the EC slider can bring out highlight detail.

In order to get the most shadow detail in the RAW conversion it's necessary to have the 'shadows' and 'contrast' sliders at zero in order to minimise shadow clipping, and of course convert into 16 bit. Having done that, it doesn't make much difference if the EC slider is at +1 or -4. All the shadow detail that's there will be converted in both cases, but the conversion with a -4 setting will need to be lightened by other methods, such as use of curves or the shadow/highlight tool.

(Notice I wrote it doesn't make much difference. It probably makes some difference as a result of quantization issues, but nothing outside the realm of extreme pixel peeping, that I can see.)

Exposing to the right is basically just a technique of making the most of the dynamic range of your camera. If the scene you are shooting is of low contrast, it's not such a big deal. If the scene has a greater dynamic range than that of your camera, then it's very important to correctly expose to the right in order to minimise shadow noise. The alternatives would be exposure bracketing on a tripod and digitally blending the different exposures.

My take is this. Expose to the right places the most actual data in the last stop as the camera can record instead of noise. I'm not sure how that equates to dynamic range which I would have to believe is fixed.

If you have a camera that can record 6 stops from shadow to highlight and you have a scene that's less than 6 stops, this is pretty easy to capture, however, you still want to place as many levels in the last stop as possible. So with a 12 bit file, that means 64 levels can possibly be utilized for data. If you under expose, you'll get less than 64 levels and if you go way too far, you'll lose shadow detail of course (and get muddy highlights you'll have to adjust).

IF you have a 6 and a half stop range, well you better decide what half a stop you're not going to capture on one end or the other. Or play with fill or other lighting techniques to adjust the scene dynamic range. If shadow detail is key, you're going to lose ½ stop of highlight detail; you can't fit the entire range, the sensor can't handle it. If you go for the highlights, you lose half a stop of shadow detail. But in either case, you want to expose to get as much data into that last stop, which means not under exposing or all the levels get shifted into the wrong direction.

Does this sound reasonable?

Sure does. I was just trying to simplify matters. If your camera has a dynamic range of 7 stops and you give 2 stops less exposure than you could have given (without blowing highlights), then it's equivalent to using a lesser camera with a lower dynamic range of 5 stops in which the shot has been correctly exposed to the right.

Dynamic range limitations are often a major concern with digital cameras. If you don't expose to the right, you are not making full use of the dynamic range capability of your camera.

Yes, I do know what a standard deviation is, even though you seem to think you are the only one who understands anything. First of all, your method of determining the read noise is non-standard. The standard method used by Roger and also illustrated at RIT is to subtract the dark frame from a bias frame, determine the SD, and divide the result by 1.414. This eliminates any clipping at black and your crude corrections for clipping are not needed.

For example, with the D200, here is a dark frame of the green channel at ISO 1600 as produced by Iris:

Click to view attachment

And here is the result of adding 200 to one dark frame (the bias) and then subtracting a second dark frame. Note that one gets a smooth symmetrical bell shaped curve as one should for a normal distribution.

Click to view attachment

Note that in both cases the standard deviation is 22.63. In the second case, the standard deviation of the subtraction is 32.304 and dividing by 1.414 gives 22.84.

In the first case, the mean is 20.469 and the standard deviation is 23.881; the usual 95 confidence interval is the mean ± 2 SD or 20.369 ± 22.84 * 2. This means that 95% of all determinations will be within this interval. I did not determine the camera gain, but using Roger's data, it is 0.5 electrons/ADU. Therefore, the SD expressed in electrons would be 11 electrons, which is close to the value Roger obtained in his analysis. 20.369 ± 22.84 * 2 gives negative counts, which is an impossibility, and results in "clipping". However with the bias frame subtraction frame one has all positive numbers. Your fudge factor does not seem necessary.

See my Post for comments on this fudge factor.