The application I mentioned was Iris. See the tutorial

BTW, in your example, you are not viewing the unmodified raw data, which are in the range of 0..4095 for a 12 bit camera chip. The image would be so dark you could hardly see it in Photoshop 16 bit (15+1). In my Iris example, I multiplied the values by 15 to bring them into the 0..32767 range expected by Photoshop.

Jeff, don't you get it? The images I posted just above are from the raw file, NOT DE-MOSAICED, only mapped with the sRaw function.

The mapping has no relevance in this question. Here they are in linear mapping, with and w/o WB.

I repeat it: no de-mosaicing has been involved. The three raw channels have been used directly as R, G and B (again, the two greens averaged in the RGB green).

Uh huh. . .so, like a bumble bee can't fly, Camera Raw can't interpolate the color data from a grayscale Bayer array? I think that was the point of demosiacing, to interpret the grayscale CFA file into RGB color?

The SPLIT_CFA gives you 4 separate files...ok, but that's not too useful unless you want to run processing on the individual files considering the size of the files is reduced by a factor of 2...but the CFA image is still a single channel image after you do a MERGE_CFA command. It's the demosiacing that make the single channel grayscale file into an RGB file...right?

You are half right: a digital camera does not have a gamut, but the raw file does have a color space as documented in the DNG specification. Have you read Chapter 6 of that spec, and what is your take on it? However, these issues are not the topic of the discussion. IMHO, you brought them up to impugn the knowledge of the poster and score debating points, rather than contributing to the topic being discussed.

No, I'll admit that I took the linear grayscale (in a linear gray gamma) and transformed it into sRGB...

Here's the actual linear file...in linear gamma.


There, that better? Course, ya can't hardly see anything in the linear gamma.

So, you are relying on the mapping function to interpret the color data from the grayscale file...in leu of demosiacing.

The point is that the SPLIT_CFA command separates the color information into four files, and it can be viewed much as you would view the color channels (R, G, B ) of a TIFF file. The color information is in the raw file and does not need to be extracted by demosaicing. The files are smaller, since there is no interpolation of the missing two colors in each pixel. You could leave blank spaces in the split files, but that would be unsightly.

That's really funny. Somehow I had the feeling, that you will mix up things, therefor I posted two images with linear mapping.

The mapping occurs within the channel. It is not a substitute for de-mosaicing. In these images the red channel of the RGB is from the red filtered pixels, the blue is... etc.

Uh huh. . .and WHAT is interpreting the red colored pixels to display as representing RED? A magic wand? Because, without something being told that this pixel in this coordinate represents red, it would still be a grayscale image file.

I could be mistaken, but I was under the impression that Panopeeper posts on the Adobe Camera Raw forum as G Sch and he has written an interesting program that anyalzes and displays raw image files without any demosaicing. If so, he is not an imaging moron, even though he doesn't use gamut in the sense that it is used in digital cameras

http://www.cryptobola.com/PhotoBola/RawnalyzeGuide.htm

This is his definition, not the definition.



According to this explanation, the gamut of sRGB is the same as that of ProPhoto: one can input the ProPhoto in the sRGB encoding. Although some colors can not be reproduced any more, that does not matter, right?

Look further for definitions, and you will find, the the reproduction of the colors is important.

Example:

Color gamut is the subset of colors, which can be accurately represented...

Depending on the cameras (sensors), certain colors can not be accurately represented (one can not distingush between the colors; in other words, when you see a certain color from the sensor, you can not tell, what color it was originally). This is the reason, that the gamuts of cameras are different.

Uh huh...and I've used Rawnalyze to look at a raw file before Zalman told me how to output a raw file from DNG Verify...it doesn't alter the fact that something, somewhere has to be able to determine that a specific pixel in a specific coordinate represents R, G, & B. And it's that interpretation of the Bayer array that then represents the grayscale as color data instead of demosiacing. But the original capture is grayscale...

So, Bill, you like the true linear image better? I actually have created both a linear gamma working space in Photoshop as well as a ProPhoto RGB with a linear gamma so I could work with linear gradients in grayscale and color. But the original posted files WERE originally in linear until I transformed into sRGB for the original posting.

Actually, Rawnalyze works with the camera's color space in the sense, that not only no de-mosaicing takes place, but the mapping from camera colors to CIE XYZ is not carried out either. (Of course, this is not correct, because the raw channels do not represent single colors almost the entire spectrum).

Btw, the wording in the DNG specification (written by Adobe) is:

Mapping Camera Color Space to CIE XYZ Space

So I sincerely hope that camera manufacturers a.) and Adobe Camera Raw b.) are sharing the same definition for CIE XYZ:

a.) to shape the spectral response of the sensor in a way to represent a linear combination of CIE XYZ matching functions.

b.) to facilitate a somewhat correct interpretation of the camera’s gamut by assignment of a matrix space representing an appropriate subset of same CIE XYZ master.

Peter

--

Jeff, you already know most of what I am saying below, but others may be interested in the information.

The linear files do not display well without color management, because a gamma of 2.2 is assumed, at least with Windows. I don't know that much about Macs, but I understand they no longer use a gamma of 1.8. If you assign your custom ProPhotoRGB with a gamma of one to the linear file when you open it in Photoshop it will display with normal tonality, but the colors will be off unless your original has ProPhoto primaries. If you take a file with a bit depth of 12 and dump into a 16 bit linear space, it will still look very dark.

An alternative way of viewing a linear file in a gamma 2.2 or gamma 1.8 space is to apply a curve in Photoshop that undoes the gamma encoding. Shown below is a gamma 2.2 curve

Click to view attachment

Here is the flower image at gamma = 1 on the left (in sRGB), with the gamma 2.2 curve applied in the middle, and with a contrast boost via an S-curve on the right. Simply applying a gamma of 2.2 results in a very flat image.

Click to view attachment

Most HDR encodings use linear gamma without any tone curve (scene referred) and they display normally in Photoshop. If you want to reproduce a scene exactly, there must be a one to one relationship between the source (the scene) and the reproduction. When you view a gamma encoded file or print it, a reverse gamma function is applied but any compression is not undone, or else the image would clip due to limited dynamic range of the print or screen.

Exactly! And this has been discussed by the color scientist at Munsell (quote above) when he writes that the camera has no gamut. You can't define the gamut or a color space until you place specific coordinates of RG and B. Bruce said this as well.

This is getting to be a big cluster-fu*% of semantics but I'd far prefer to take the side of the color scientists and Bruce, among just a few who state I believe correctly and for the last time, the Raw data is essentially Grayscale and the capture device has no gamut.

And yet another quote from Phil Green:



And again:

And here is a demonstration of a color image from raw without any demosaicing. One takes the raw file and splits out the red, blue, green1 and green2 portions of the CFA with the Iris SPLIT_CFA command. This gives four files: red, blue, green1 and green2. The files are in the FIT format, which Photoshop can not read, so they have to be converted to TIFF, which I did with the freeware program ImageJ. The result is three gray scale files. They can then be loaded into Photoshop and merged into an RGB file. I used only one of the green separations. This proves that color can be generated without demosaicing.

Click to view attachment

The result is gamma 1

Click to view attachment

One can then use the gamma 2.2 curve and a contrast boost to get the final result. The background is ugly, but I used what was available to demonstrate the point.

Click to view attachment

An RGB file in WHAT color space? How did you decide how to map the scale of the RGB primaries from the Raw data? This is the meat and potatoes here. You've got a pile of numbers. What's the scale and how did YOU or your Raw splitting software decide how to define these primaries? That would give us a color space definition. Read what the color scientists and experts I've quoted have said about this assignment. How do YOU define the gamut by defining the scale of the primaries of what isn't a color file?

And I suspect this goes back to the DNG spec you keep mentioning. Where does it tell us the scale of the primaries so we can define a color space and gamut? As I posted by another author, someone, somewhere makes it up more or less.

At the time of demosicing, the primaries are defined, you have a color image. Richard F. Lyon explains it the best:the numbers are usually matrixed to somebody's standard RGB space, based on a set of primaries. The color scientists at Munsell said it too:In the color world, encoding is based on some explicit or implied display.

No, here is a color image from Raw WITH demosaicing!

http://en.wikipedia.org/wiki/Demosaicing

This is only a proof of concept demonstration with unmanaged colors. Your comments are irrelevant to the demonstration, but the colors could be scaled by performing a 3 by 3 matrix conversion from the camera space to CIE XYZ and then another transformation to one's preferred working space.

NO, my points are very relevant to the original post you made far too many posts ago:



You say the statement is debatable. We have a debate, one in which you haven't, as yet proven to me (and perhaps anyone else) that a Raw file has a color space. I say its essentially Grayscale, it has no color space. I stick to that opinion. You've quoted page 47 where it states "from camera color space" assuming that this is defined, I've mentioned several well qualified experts who say, its whatever you want to assign the scale of the primaries at encoding which I'll add is happening AT the state of demosaicing. You attempted to demonstrate this above, and so I again ask, what's the color space you made up? And how does this make it 'correct' and further prove that prior to this part of the process, the camera had a gamut?

Digidog, you are confusing the issue. I did not state that a digital camera has a gamut. A raw file is gray scale according to your nomenclature, and Bruce stated p. 119 of Real World PSCS2, "In Photoshop, files saved in the RGB mode typically uses a set of three 8 bit grayscale files..." [bold added for emphasis]. The grayscale files can form a color image because the three primary colors of the image are represented by the grayscale files. The raw file is also gray scale, but a color image can be formed from the red, blue and green elements of the Bayer array. In the latter case, all three primaries are encoded into one file rather than three separate files. If you can't see the analogy, I don't think you are very bright.

Fine



Where do you see him referring to Raw files in this context?



The difference, and Jeff clearly stated this, is that the Raw is a single channel Grayscale file. In fact, this all goes back to me saying, "Raw is Grayscale data" which I continue to state and stand by.

That a single channel Grayscale file is different from a three channel Grayscale file shouldn't even have to be discussed here. Its a big Duh!

The fact that a Raw Grayscale file was produced with color filters, or that you can somehow split this data into now, three Grayscale files to represent a color image (something we know is the case, we've discussed demosicing), doesn't change the facts as I and Jeff and others have presented them. Yes, a Raw, Grayscale file HAS the potential to be a color image. Another big Duh moment. That doesn't make the single channel Grayscale Raw file a color file, cause it's not. ITS GRAYSCALE, something I said almost in passing, and something you said was debatable thus bringing on this entire sh*&-storm.

Lets cut to the chase. If a Raw file isn't Grayscale, are you saying its a color file?

For the last time, what do you think of chapter 6 of the DNG specification?

You might also want to look at this thread on the Abobe Forums. Mr. Knoll put the camera color space in quotes, but he seems to be recognizing that it exists as per the DNG spec.

Yes he puts it in quotes for exactly the reasons I've explained (well others have explained far better) in the numerous quotes above. Thomas puts them in quotes because he's saying as far as I'm concerned, that he agrees with the concepts expressed by both Phil and Richard which I don't think you've read. That basically, the color space is whatever anyone wants to say it is based on how they want to encode the data. Otherwise, both the DNG spec AND Thomas's quote would be far from vague, which clearly they are.

I agree that a raw file is gray scale. How many times do I have to state that? (So are the channels of an RGB file. Bruce has confirmed this, but you don't seem to get it.) However, if you understand the Bayer grid, you can extract the color information from a gray scale raw file. It is a special type of gray scale file.

Of course an RGB file in ProPhotoRGB contains no color. It merely has the data necessary to construct color when viewed with the proper software. The raw file contains no color, but it also has the information necessary to produce color

Just once, after you said this:



In closing, here's the bottom line. We have a Raw file that I say is Grayscale. You read the DNG spec and say, based on what is said in the document, a Raw file is a color file because at some point, the Raw data is encoded into a color space. So, is a Raw file a color file or a Grayscale file? Not what it may become based on some processing, the damn file sitting on your flash card?

This seems (it is) all so simple.

I'll ignore your ill manners and comment at the top of this thread since I can feel your frustration at having to admit that you were barking up the wrong tree from the get go.

You have addressed only one part of my post. Again, what do you think of Chapter 6 of the DNG specification? Make what you will of Mr. Knoll's quotes. You can't read his mind. Since you do not respond to reason, this is my last reply to you on this issue.

And to go even deeper, its neither, its just a big pile of numbers.

I told you what I thought of chapter 6 of the DNG spec. As explained by Phil and Richard.

Andrew,

I am sorry that my frustration affected my manners. Please accept my apology. Also, thank you for taking the high road and not replying in kind. I have edited my offending post. Just as in the Adobe forum postings on the same topic, agreement was not reached, and we will have to agree to disagree.

Bill

As it relates to DNG what I think is throwing you is that DNG ASSIGNS a color space based upon Thomas' testing of the spectral response of the sensor at D65 and Standard Illuminate A. Then based upon the white point in the metadata an arbitrary white point is assigned. Assigned mind you...and this is only on a DNG whose sensor has been decoded by Thomas and whose spectral properties have been measured.

Until Thomas MEASURES the the response and ASSIGNS a color space, there is no color space. Note that a DNG is a PROCESSED file, no longer a native raw file...

We are also hung up on the term "demosaicing"...well, the reason I pointed out the Wikipedia article is that my understanding of the term demosaicing is based upon that: "A demosaicing algorithm is a digital image process used to interpolate a complete image from the partial raw data received from the color-filtered image sensor (via a color filter array or CFA) internal to many digital cameras in form of a matrix of colored pixels."

So, splitting a single channel grayscale file into 3 grayscale files and combining them into a single RGB file is a "demosaicing algorithm" even if the process of assigning the red, green and blue pixels is manually done. Something, somewhere is saying, "ok, these pixels represent the red pixels, etc."

Taking the original raw, grayscale file and looking at it in any viewer app that does NOT designate which pixel are which colors and interpret those grayscale pixels as colors, will show the raw image as a single channel, grayscale file. Right? So, something, somewhere needs to do something to interpret the color of the Bayer Array, and I call that process a "Demosaicing" consistent with the definition on Wikipedia.

If you or Pannopeeper have a problem with that definition, I suggest you sign up on Wikipedia and do some updating of the article in question.

LOL, that does it for me.

Do you dispute that DNG Converter processes the proprietary raw file into a DNG file? It's also what Camera Raw does when it opens a proprietary raw file...

Well, I thought that assigning a profile does not change the numbers in the file, but only determines how they are interpreted or what they represent. When Mr. Knoll measures the response of the camera at D65 he is determining the coefficients needed to perform a 3 by 3 matrix conversion from the "camera color space" to CIE XYZ or perhaps the PhotoProRGB chromaticity with a linear TRC. He is not creating a new space but merely describing the inherent properties of the sensor, i.e. the "native color space" of the camera . It is my thesis that those coefficients are implicit to the camera space. I don't know the details, but Mr. Knoll has stated that CIE XYZ has no white point. Perhaps you can help us in understanding the significance of this difference.

Furthermore, I don't think a white point is assigned at the time the DNG is created, but it can be interpolated from the D65 and Illuminate A profiles that are present in the DNG when needed. The AS SHOT white point is encoded in the raw file and can be used in this process.



Form the Adobe DNG Primer for Manufacturers it is apparent that part of the DNG consists of metadata that the converter fills in describing the characteristics of the camera that are necessary for raw conversion.

The camera's actual image data are then "stored in a linear non-white balanced color space, usually the native color space of the camera". The data can be in mosaic form or demosaiced, but the mosaiced form is preferred, since it represents the original data captured by the camera.

In the preferred case, the actual image data are the same as in the raw file. If the file is demosaiced, then it is heavily processed and it would not be possible to apply a better demosaicing algorithm that might be developed in the future. Note that Adobe again mentions the native color space of the camera, which is not in accordance with your strict definition of a color space. As I initially stated, whether a raw file has a color space is debatable. Depending on one's definition, one can make either argument. However, I am not contending that this "native color space" can be used in Photoshop as one would use a printer profile. You discuss the pros and cons of camera profiling in your book, and make a good case that its absence in ACR is not a handicap.



Strictly speaking the SPLIT_CFA command is demosaicing in the sense that it separates the red, blue and green components of the raw file into separate components. But it does no interpolation and does not fulfill the definition of demosaicing on Wikipedia. It does not attempt to determine the two primary colors that are absent in each Bayer array pixel. I can't speak for Panopeeper, but I have no quarrel with that definition.

Uh huh...but putting the 3 split channels into a single 3 channel file (which is what you were talking about) DOES constitute a "CFA interpolation or color reconstruction"...the fact it does no resampling doesn't mean it ain't a "demosaicing algorithm", it just means it's not doing any upsampling. You take a single channel file, split it into 3 channels then combine them into an RGB file. I call that "demosaicing".

Well, considering that camera sensors suffer from metameric failure at different spectral illuminates, (the whole reason Thomas tests the sensor at D65 AND Illuminate A), the spectral response of the camera's sensor is a sliding scale and thus would not have a single "color space" but potentially many "color spaces" depending on the illuminate.

You may call it demosaicing, but I don't think that the Wikipedia definition is met as there is no interpolation. Since, I have already conceded that by a strict definition, what I performed could be considered demosaicing, I don't know why you even brought up the subject other than to have the last word. Interpolation is making up values that did not exist. As you know, the Bayer array can capture only one color per pixel, and the other two colors are interpolated by best guess from surrounding pixels. My process did not invent any new data to "fill in the blanks" as Wikipedia states it, but merely looks at the same data in a different way.

Anyway, I don't see how this affects the argument that the raw file has no color space. If it has no space before the demosaicing, then it has no space after the demosaicing. What has changed that would make it a color space after demosaicing? The file may acquire characteristics to fulfill your definition of a color space later in the process.

It has no color space before because its not a color image, it has a color space after because someone has encoded the demoasiced data into a color space. That's exactly what encoding does.

http://www.color.org/ICC_white_paper_20_Di...ment_basics.pdf

You are obsessing on white point, which is part of the ICC profile, but CIE XYZ lacks a white point and is widely considered to be a color space. Look at Wikipedia. Is it a color space by your definition?

Yep...cause it has known and defined set of tristimulus values...a raw capture doesn't. The tristimulus values at D65 will be different than at Illuminate A (or presumed to be which is why Thomas tests cameras at both).

Your link does not work, so I can't comment. The demosaicing does not encode a color space, but merely separates out the color information from the Bayer array and fills in the missing information. As Jeff explains on pp 4-5 of his Camera Raw book, colorimetric interpretation, white balance, and tone mapping are separate steps and I think the encoding to which you refer takes place during these steps. As I stated, the demosaicing per se has no effect on whether the image has a color space or not. After the demosaicing, it is a color image by your definition, but apparently still lacks a color space.

Now we are expanding on the definition of a color space, and dropping the requirement of a white point. Hard to hit a moving target. BTW, illuminate is a verb. I think you mean illuminant.

Yep, it _IS_ hard to hit a moving target...and a sensor's response at D65 and Illuminant A means that a camera sensor DOESN'T have a fixed "color space".

Absolute color space: A color space in which colors are unambiguous, that is, where the interpretations of colors in the space are colorimetrically defined without reference to external factors.

Since the spectral response of a sensor depends on the illuminant (because of metameric failure) then a camera doesn't have an "Absolute color space". (there is another definition but I don't think it applies). Ref: Absolute color space

The links on the ICC web page are kind of hosed, but the article discusses the role of color space encoding. I'll email them to fix this link.



After which, the color has to be encoded into a color space. IF you'd only read the posts made by Phil, and could get to the ICC link, you'd see this is a somewhat arbitrary process that someone has to define.




They have to happen, at what step in the process, once we have a true color image that can have a color space may be questionable but at some point, the numbers have to have an associated scale placed on them. IT is at this point that these values have a color space. Prior to this, its Grayscale data (which you finally admit), it can't have a color space association.



No, at such a point, a color space is assigned if you want to use the term. Again, this is very clearly stated in the post I provided by Richard and Phil:





It again, is pretty simple. I don't know why you don't wish to accept this.

The posts by Rich and Phil were concerning the lack of a gamut for digital cameras. I'm not certain what I should have learned from their posts that affects the current discussion.

It was Panopepper who was confused on the issue of camera gamut. I was well aware of that issue, having made the same mistake some time previously in a different thread. It is easy to slip up, and even an expert like Peter Lange mentioned the gamut of a digital camera.

I think its time to give up here.

You can't define a color gamut without a color space. They clearly indicate encoding of the data after a color file is produced using what you are calling demosaicing.

The original set of debates started because YOU questioned that a Raw file was Grayscale. Then you accepted it was Grayscale (because it is). A Grayscale file doesn't have a color space right? You can't define the boundaries of the primaries which is all a color space is. OK, we've got a color image now, we used demosaicing. So what's the color space? You keep going back to an article that doesn't define this to point out you feel Raw is a color image but we all know now, as you admit its not. Once we DO have a color file, what's the color space? Phil AND Rich have explained this.

Why do we keep going around in circles here?

The acceptance of yours that a digital camera has no color gamut AND that a Raw file has no color (nor color space) indicates to me we're at least making some progress. Again, read what Phil and Richard have written.

Well, metameric failure occurs not only with printers, but also printers. The Epson 2000 was noted for prominent metameric failure. Yet people still made profiles for the printer.

Yes, I think so. You keep reiterating the same things and my responses also have become repetitious. Your ability to see another point of view is limited if it does not fit into your preconceived ideas, which are usually but not invariably correct.


First of all, we are not trying to define the color gamut of a camera, because it has none, as you have already pointed out. However, if you extended the analysis into the deep infra-red and ultra-violet a gamut would emerge.


Your memory is rather selective and defective here. I did not deny that a raw file was gray scale, but I also pointed out that an RGB file is also gray scale as per Bruce Fraser's writings. Please refer back to post #11 in this thread. I have not changed my opinion. The raw file is gray scale because it records only luminance, not color.

The channels of an RGB file are gray scale for the same reason. The difference, which you don't seem able to grasp, is that both of them contain color information. Once we know what the channels of the RGB file represent, we can generate color. By the same token, once we know what the gray scale pixels in the Bayer array represent, we can also generate color. Demosiacing is not necessary to generate the color. It was Jeff who went off on that tangent.

A monochrome file can have a color space if it contains color information that, when properly decoded, defines colors.

Furthermore, I merely stated at the outset that raw files lack a color space is debatable, which this thread certainly has proved. To paraphrase Bill Clinton, who said, "it depends on what the meaning of IS is", it depends on what the definition of a color space is. I have yet to see your definition of what constitutes a color space; lacking this, we can go nowhere.

Schewe has it spot on!
1. Noise comes from high ISO (sensor) long before the color space gets applied. A wider gamut simply stretchs it, but doesn't add to nor subtract from it. To cure noise, shoot at ISO 100 or maybe 200. At 400 or beyond, you are going to have to deal with the noise.

2. Sensors and RAW are LINEAR gamma, which is NOT how humans perceive things. That is why the linear data needs to be converted into a color space. Schewe pointed that out, but the significance is enormous.

3. If you choose a wide gamut, such as ProPhoto and edit in 16-bit, then there is "room" for gradations of color. If you don't EDIT a photo, color space doesn't mean much -- colors are what they are.

4. Lastly, RAW is 12 bit. If you edit in 8-bit, then you are throwing out some info (like rounding pennies to nearest dollar). If you are fussy about colors and detail (I am), then edit in 16-bit and print in high resolution. It makes a difference in skin color and details like hair. And if fine hair detail matters, then use a tripod so you aren't editing a 16-bit blur.