Help - Search - Members - Calendar
Full Version: Converting to DNG - do you lose quality?
Luminous Landscape Forum > Equipment & Techniques > Computers & Peripherals
Pages: 1, 2
The View
One of the basic truths in image editing is, that if you convert to a different color space, you lose quality.

If you convert to a different file format, you lose quality.

Do you think, that I lose quality when I convert my Canon 40D files to DNG?

And is there a good reason to do so, or shouldn't I just keep the files as they come?

Thanks!
ErikKaffehr
No, you don't loose quality
Yes, there are good reasons

- DNG contains the raw data coming from the camera it doesn't add a color space
- As the pixels are kept you will not loose quality
- You don't loose quality if you convert from one non lossy file format to another non lossy file format.
- It is actually possible to set up DNG so it will store the actual original raw-file in DNG.

The very good reason to save files in DNG that there are so many versions RAW-file formats, it is quite probable that it can be difficult to read todays RAW in perhaps five years. DNG is intended as a portable RAW-format. It is developed and maintained by Adobe but specifications are open AFAIK.

Are there any disadvantages with DNG? Sure some proprietary information is lost, if you don't use the option to save the original raw file in the image. I'm not really sure that lossless actually is lossless.

Best regards
Erik


QUOTE (The View @ Jul 12 2008, 07:09 AM)
One of the basic truths in image editing is, that if you convert to a different color space, you lose quality.

If you convert to a different file format, you lose quality.

Do you think, that I lose quality when I convert my Canon 40D files to DNG?

And is there a good reason to do so, or shouldn't I just keep the files as they come?

Thanks!
*
The View
QUOTE (ErikKaffehr @ Jul 11 2008, 10:33 PM)
No, you don't loose quality
*


Am I misreading this as a contradiction?

Philosophically speaking, there is no gain without a loss. You don't lose quality vs. there isn't actually any lossless.

QUOTE (ErikKaffehr @ Jul 11 2008, 10:33 PM)
Are there any disadvantages with DNG? Sure some proprietary information is lost, if you don't use the option to save the original raw file in the image. I'm not really sure that lossless actually is lossless.

*


This is actually a great message by you (if I don't misread it). In the digital era we strive for anything lossless. Lossless compression, non-destructive editing.

But it looks like this is all marketing speak.

I just compared my original 40D file, converted by the not very workflow-friendly Canon DPP with the superb workflow software Lightroom's conversion.

Looks like you're taking a hit in conversion quality vs. workflow quality.

And, regarding the superb workflow format DNG, maybe one pays for this with losing, as you said, some "proprietary information".

Proprietary information. This is actually wonderfully said. Isn't anything coming from a camera regarding image quality "proprietary information".

You either have a talent for philosophy or diplomacy, or both.
The View
Just erased the last import, and re-imported in original Canon RAW format CR2.

Result:

Much better.

Converting your RAW files to DNG ruins your images. Makes them lifeless, soggy, and you lose detail. The colors wash out.

For me DNG is history. Never again!
ErikKaffehr
Hi!

There is actually a lot of lossless compression. A very simple example is:

000000000 can be written 9x0. This is essentially what is called run length encoding. Takes 1/3 of the place (in this case).

Lossless encoding means that you can restore every bit in the image.

One issue is that RAW has linear encoding, so it does not use gamma. It is feasible to convert a 12 bit linear image to an 8 bit gamma encoded image without loosing any significant information, because of gamma encoded imaging is more efficient. BUT this would not be pixel for pixel equivalent with the original image. I doubt that any observer would be able to observe a difference. Such a conversion would be virtually lossless. You would not loose information but could not reconstruct the original image. Could you see the difference? Probaly not! Could you measure the difference? Absolutely! OK this may or may not be a diplomatic view.

Regarding proprietary information there is a lot of information embedded in the image, some of that information is standardized like EXIF and IPTC tags. Some tags are vendor specific. One example is the white balance information.

The issue is really that with each camera there is a new "raw" format. Will you be able to read your present "raw" files on your Mac OS X/7 or Windows 2012 computer?

Washed out colors may relate to color space. Lightroom uses ProPhotoRGB as a color space. If your Photoshop is incorrectly set up it may ignore the color profile. I don't thing DNG has anything to do with Color Profiles. It just holds pixel data and information tags.

This is not a question of diplomacy or philosophy, more about how things are. Many people of great experience and knowledge use Lightroom and DNG, like Michael Reichmann (the owner of this site) or Jeff Schewe, they both use Canon and they both know what they are doing. It is quite obvious that some people have issues with Lightroom. I don't know the reason for this. There was a gentleman on the MFDB forum who had a real bad experience.

If you write on this forum for advice you may get good or bad advice but I presume that all information is intended to be helpful.

Best regards
Erik


QUOTE (The View @ Jul 12 2008, 09:12 AM)
Am I misreading this as a contradiction?

Philosophically speaking, there is no gain without a loss. You don't lose quality vs. there isn't actually any lossless.
This is actually a great message by you (if I don't misread it). In the digital era we strive for anything lossless. Lossless compression, non-destructive editing.

But it looks like this is all marketing speak.

I just compared my original 40D file, converted by the not very workflow-friendly Canon DPP with the superb workflow software Lightroom's conversion.

Looks like you're taking a hit in conversion quality vs. workflow quality.

And, regarding the superb workflow format DNG, maybe one pays for this with losing, as you said, some "proprietary information".

Proprietary information. This is actually wonderfully said. Isn't anything coming from a camera regarding image quality "proprietary information".

You either have a talent for philosophy or diplomacy, or both.
*
chappers
Not a real scientific test, but I've converted one CR2 image into dng, and imported them both into Lightroom. I can't see any difference between the two. Even at 100%. Now it may be that they don't contain any really saturated colours, but the image has green foliage, green grass. Neutral grey vehicle body work and a deep blue flag.

Canon DPP does produce a different result to Lightroom with regards certain images. From what I remember of the topic DPP does seem to give a warmer and more saturated conversion than Lightroom does "as standard". But this may be simply down to the individual way both applications process the initial view.
Jonathan Wienke
QUOTE (The View @ Jul 12 2008, 11:38 AM)
Converting your RAW files to DNG ruins your images. Makes them lifeless, soggy, and you lose detail. The colors wash out.

For me DNG is history. Never again!


Bullshit. DNG files contain the exact same RAW data as the manufacturer's file format. If you convert an original RAW file and the DNG in ACR with the same settings, the converted images will be identical down to the last pixel value. You can verify this by pasting the versions on top of each other in PS and setting Difference blend mode.

If you got different results after converting to DNG, YOU screwed something up.
Panopeeper
QUOTE (The View @ Jul 12 2008, 01:38 AM)
Converting your RAW files to DNG ruins your images. Makes them lifeless, soggy, and you lose detail. The colors wash out

Nonsense. I am working (programming, coding/encoding) native raw files as well as DNG files. I can assure you, that there is no relevant difference in the data except in a very few muddy cases (certainly not with Canons).

For example the data encoding (what you call "compression") in DNG is the very same as the one adopted in Canon CR2 raw files. The main difference is, that the DNG converter has more time and more computing power than the camera, thus it can compress more effectively.

Nevertheless, I don't see any (really none) reason to convert the 40D CR2s in DNG except the sightly reduced size, which is not enough for me. Plus, if you want to keep the original raw data, the DNG will be much larger than the CR2, and anyway it is a brainsick idea to embed the original in the converted file.
DarkPenguin
QUOTE (The View @ Jul 12 2008, 04:38 AM)
Converting your RAW files to DNG ruins your images. Makes them lifeless, soggy, and you lose detail. The colors wash out.

For me DNG is history. Never again!
*

Pilot error. Unless you select convert to linear data the data in the dng is the same as the data in the cr2.

Did your coverter have settings saved for the cr2 version and not for the dng version?

As to your original question. It can be nice to have the full sized preview in the dng. It can also be nice to not have side car files to lose. It can also be helpful to convert to dng to get your files into a format that an older version of photoshop will support. (Which is funny to use a new version of the dng converter to allow an older version of photoshop to work.)

What you lose is the ability to use some software. Notably DPP. (I think Nikon's software only supports NEF.) It can be hard to tell if you care about this. DPP is a pain to work with but does have some support for correcting lens flaws. This was not a feature at the time I converted most of my files.
The View
No idea what happened.

I just noticed from my old Pentax files, that the DNG were 50% smaller than the PEF files. Maybe that's a clue? I can't see an item in the import dialogue that allows you to mess up images the way mine were.

I'll see if I can find what went wrong.
The View
QUOTE (Jonathan Wienke @ Jul 12 2008, 04:45 AM)
Bullshit.
*


Not exactly a terminus technicus, at least not in photography.
The View
QUOTE (ErikKaffehr @ Jul 12 2008, 02:06 AM)
Hi!

There is actually a lot of lossless compression. A very simple example is:

000000000 can be written 9x0. This is essentially what is called run length encoding. Takes 1/3 of the place (in this case).

Lossless encoding means that you can restore every bit in the image.

(...)

I don't thing DNG has anything to do with Color Profiles. It just holds pixel data and information tags.


Best regards
Erik
*


I see it can't be DNG's fault. Whatever it was, the difference was so striking...

And it can't be some kind of color space switch without my knowledge.

For the moment, I'll keep working with the original Canon RAW format until I know what happened.
Panopeeper
QUOTE (The View @ Jul 12 2008, 06:18 PM)
I just noticed from my old Pentax files, that the DNG were 50% smaller than the PEF files

This depends on which Pentax. For example the *ist PEFs are uncompressed, the K100 and K200 PEFs are compressed by "packing", while the K10's PEFs are encoded like Canon, Nikon, DNG. Accordingly, the proportion between the PEF and DNG sizes varies.
The View
QUOTE (Panopeeper @ Jul 12 2008, 06:37 PM)
This depends on which Pentax. For example the *ist PEFs are uncompressed, the K100 and K200 PEFs  are compressed by "packing", while the K10's PEFs are encoded like Canon, Nikon, DNG. Accordingly, the proportion between the PEF and DNG sizes varies.
*


So that can't be it then, either.

And there are no "conversion settings" to convert to DNG.
Panopeeper
QUOTE (The View @ Jul 12 2008, 06:38 PM)
there are no "conversion settings" to convert to DNG.

The conversion settings are under "Change Preferences" (there are not many).
The View
QUOTE (Panopeeper @ Jul 12 2008, 06:47 PM)
The conversion settings are under "Change Preferences" (there are not many).
*



CHANGE preferences? You mean the preferences pane with the Command comma shortcut in "lightroom" menu?

I actually only found export settings, color space for exporting to external editors, etc.
Nick Rains
QUOTE (The View @ Jul 12 2008, 07:12 AM)
Am I misreading this as a contradiction?

Philosophically speaking, there is no gain without a loss. You don't lose quality vs. there isn't actually any lossless.
This is actually a great message by you (if I don't misread it). In the digital era we strive for anything lossless. Lossless compression, non-destructive editing.

*


CR2 files are already compressed - why do you think they vary in size? If you don't believe me see for yourself - or do the maths ( 2 bytes per photo site x megapixel rating ie 12mp should be a 24Mb file, except it ain't) . You'll find that ones with lots of detail are bigger than ones with big smooth areas - just like jpegs.

DNG files simply store the exact same RAW data in a new 'box' with a nice jpeg preview based on the RAW adjustment defined in your RAW converter (ACR) plus a set of instructions comprising the settings used to adjust it. That's all - converting to DNG does not compress or alter anything in the RAW data.

What you are seeing is different conversions being applied to the same data - this is what you need to attend to.
Panopeeper
QUOTE (The View @ Jul 12 2008, 10:08 PM)
CHANGE preferences? You mean the preferences pane with the Command comma shortcut in "lightroom" menu?

Were you converting the CR2 in DNG via LR? SOrry, I have no idea, how that works.

I am using the stand-alone DNG converter, which has a "Change Preferences" button; that's what I was referring to.
The View
QUOTE (Panopeeper @ Jul 13 2008, 11:13 AM)
Were you converting the CR2 in DNG via LR? SOrry, I have no idea, how that works.

I am using the stand-alone DNG converter, which has a "Change Preferences" button; that's what I was referring to.
*


Which one do you use?

I converted to DNG using the LR setting "convert to DNG and import".
DarkPenguin
Adobe DNG Converter. You can get it bundled with ACR at the Adobe site. It is just a standalone executable.
Panopeeper
QUOTE (The View @ Jul 13 2008, 11:25 AM)
Which one do you use?

Adobe's stand-alone DNG converter is part of the ACR package. When downloading, you can chose this package and unpack the converter. "Stand alone" means in this context, that it is running w/o PS or LR. It created a DNG file, which can be processed by ACR in LR or PS.

This conversion is not influenced by any other setting, except the few options like "compression", "JPEG preview" (do *not* convert in linear).

After the conversion open both the CR2 and the DNG *together* in Photoshop, set all adjustments equal in ACR and see if there is any difference.
Jonathan Wienke
QUOTE (The View @ Jul 13 2008, 04:30 AM)
Not exactly a terminus technicus, at least not in photography.


It's a perfectly accurate description of the ignorance and misinformation contained in your original post.

DNG uses a lossless compression method that reduces file size without altering the RAW data contained in the file. Think of a ZIP file, but with a different compression algorithm optimized for image data. Therefore, the DNG file size is almost always smaller than the original, but that does not mean the RAW data inside has been altered in any way.
The View
QUOTE (Jonathan Wienke @ Jul 13 2008, 05:23 PM)
perfectly accurate
*


You could have handled it with courtesy and knowledge, and simply stated facts, like others did.

But I see you're not good with words and can't handle your temper. So be it.
ErikKaffehr
Just a comment.

I think that we can see from the conversation that the issue you have is probably not with DNG but with LR. Sorry for that! It seems that LR works better for some folks than for others.

Now, there is an idea behind LR and that idea is that you keep the RAW file untouched and render it on the fly for different needs. LR gives you a very large gamut RGB which is internally linear.

If you just want to use LR as raw-converter it is probably not the best tool, it may be better to use some of the top class raw-converters like Phase One, Iridient Developer or what tool you want to use and than have an industrial strength asset management system for your "tiffs".

If you wan't to have a "raw" based workflow I think that LR is one of the better solutions, the other one is Aperture.

Regarding the issue with DNG I think that there may be a couple of issues. We have seen that the "pixels themselves" are not affected by conversion to DNG. As I told you there are a lot of additional information called "tags". Some of these are standard and some are vendor specific.

Now, one of the features of DNG is that it can contain XMP information. Lightroom actually stores it's development settings in the DNG file, it can also put it into a "sidecar" file. So this may have caused some of your issues, not the DNG file on its own, but the instructions in it. Sorry for being diplomatic, but I just try to guess what is happening.

The development settings go in three places in Lightroom

1) The internal database
2) I think also the DNG files
3) Optionally the XMP-sidecar files

Best regards
Erik

QUOTE (The View @ Jul 13 2008, 04:36 AM)
I see it can't be DNG's fault. Whatever it was, the difference was so striking...

And it can't be some kind of color space switch without my knowledge.

For the moment, I'll keep working with the original Canon RAW format until I know what happened.
*
Panopeeper
QUOTE (ErikKaffehr @ Jul 13 2008, 09:20 PM)
LR gives you a very large gamut RGB which is internally linear

Do you mind explaining what this means? I don't know of any RGB, which is linear "internally" or "externally", nor do I see any CR option for generating a linear output.
ErikKaffehr
Hi!

As far as I understand LR is internally using a version of ProPhoto RGB but without a gamma curve. It's called "Melissa RGB".

Best regards
Erik

QUOTE (Panopeeper @ Jul 14 2008, 05:56 PM)
Do you mind explaining what this means? I don't know of any RGB, which is linear "internally" or "externally", nor do I see any CR option for generating a linear output.
*
Panopeeper
QUOTE (ErikKaffehr @ Jul 14 2008, 08:22 AM)
As far as I understand LR is internally using a version of ProPhoto RGB but without a gamma curve. It's called "Melissa RGB"

Ok, now I understand. ProPhoto is not linear, but LR converts an RGB image (which is never linear) into linear form and makes calculations on that; that is nothing special. The gamma-converted format is not suitable even for the simplest adjustments, like brightness or contrast. When writing the file, it gets gamma-converted again.
Jonathan Wienke
QUOTE (The View @ Jul 14 2008, 03:34 AM)
You could have handled it with courtesy and knowledge, and simply stated facts, like others did.


Others tried that approach, and you continued to insist on your point of view that converting to DNG caused quality loss and negatively impacted the conversion of images from the DNG compared to the original RAW. I restated the point in more forceful terms, and somewhere along the line you finally realized (I think) the error of your original position. If you don't know what you're talking about, asking questions is fine (it's how we all learn), but passing off your own ignorance as fact is likely to confuse and misinform others who read your posts. If you'd done even the most basic research on DNG and how it works you'd have realized that the issues you were encountering were the result of user error, and not a loss of quality due to converting the file to DNG.
Farkled
Nick Rains is correct. What you are seeing is the different interpretation (initial or otherwise) presented by each program. DPP tends to preprocess the RAW preview into the same form the JPG would have taken in camera. DNG is not so aggressive.

The primary reason to not convert to DNG is that you then lose DPP as a processor. It can be very valuable.
Panopeeper
QUOTE (Farkled @ Jul 17 2008, 05:05 PM)
DPP tends to preprocess the RAW preview into the same form the JPG would have taken in camera.  DNG is not so aggressive.

(LOL, another one making definitive statements instead of asking about the subject)

DPP is a program, while DNG is an image data format.
Farkled
QUOTE (Panopeeper @ Jul 17 2008, 06:35 PM)
(LOL, another one making definitive statements instead of asking about the subject)

DPP is a program, while DNG is an image data format.
*

I apologize. I wrote DNG when I meant the viewer in Bridge - which does not produce the same initial results as DPP when displaying CR2 files.
madmanchan
QUOTE (Panopeeper @ Jul 14 2008, 04:34 PM)
Ok, now I understand. ProPhoto is not linear, but LR converts an RGB image (which is never linear) into linear form and makes calculations on that; that is nothing special. The gamma-converted format is not suitable even for the simplest adjustments, like brightness or contrast. When writing the file, it gets gamma-converted again.
*


Just to clarify: CR/LR converts the linear raw data to a RGB color space which has ProPhoto primaries, but linear gamma (i.e., gamma = 1.0). We sometimes refer to this informally has "ProPhoto linear." Subsequent image processing is done using this color space, the idea being that (1) the color space is very large, so it's unlikely that users will clip captured colors as a result of image processing, and (2) the pixel values are still linear, which is needed (or at least beneficial) for certain types of image processing.

Once the raw image processing is done, the ProPhoto linear data is then converted to the output space chosen by the user (e.g., in CR's workflow options or LR's export dialog). For display to the screen, of course the image data is converted via the monitor profile.

Some more details are in the DNG 1.2 spec for those who are interested.
budjames
For about 6 months, I've been archiving my Canon RAW files (40D, 1Ds MkII, 1Ds MkIII) and converting to DNG upon ingestion into Lightroom v1.4.

I cannot tell any difference between ACR and LR conversions using the original RAW file or DNG created by LR. I've made a number of prints in varying sizes from the DNG files and they look great! Any of my images requiring pixel mashing is done in PS CS3 with final sharpening done using PhotoKit Sharpener. The convenience of not having sidecar files lose is a nice plus to using DNG.

The only reason that I see to use RAW as the working format is if you use DXO Optics RAW conversion as this program does not read DNG. I've found that LensFixCL plug-in for PS CS3 provides the lens correction capability to PS which is all I need to fix the distortion of Canon wide angle zooms.

When you look at the great image masters like Jeff Schewe and Michael Reichmann using DNG, I think that they have enough experience to better judge the benefits of DNG vs original RAW conversions.

Cheers.
Bud James
North Wales, PA
Schewe
QUOTE (budjames @ Jul 20 2008, 03:43 AM)
When you look at the great image masters like Jeff Schewe and Michael Reichmann using DNG, I think that they have enough experience to better judge the benefits of DNG vs original RAW conversions.
*



If you are talking about Camera Raw or Lightroom's ability to render either the original raws of DNGs, there is ZERO differences...in point of fact, Camera Raw's first steps when opening a raw file is to convert that data into DNG data (not a DNG file, just DNG data) in order to process it. I'm pretty sure Lightroom does the same thing.

So, from the standpoint of CR/LR, there CAN'T be any difference in rendering because the rendering is being based on the exact same data.

Look, there's a lot of misconception and myth regarding raw image data. Some people think the only Nikon and Canon can possible know what the raw files' secret data means...and to an extent it's true that the undocumented data does produce problems for 3rd parties. But if you look at the experience and knowledge Adobe has obtained regarding raw files, I think that with over 170 raw file formats under their belt, Thomas Knoll and the other Camera Raw engineers pretty much know exactly what's in those raw files.

There are some real advantages regarding DNG...the raw data & xmp metadata is contained in a single file. DNG has completely lossless compression. DNG is fully documented and for long term conservation and preservation, that's critical. DNG, as a raw file format, has enormous 3rd party support which can not be said for all the 170+ raw file formats which can be turned into DNG. Adobe is also in talks with the ISO to allow the ISO to take over stewardship of DNG. Adobe has already allowed the ISO to take over portions of TIFF for the development of TIFF-EP (Tiff for electronic photography) and they have a lot of experience in standardized formats such as PDF and initiatives such as XMP.

On the contrary, the camera companies (particularly Nikon and Canon) continue to keep their formats proprietary and undocumented and "secret" even if their formats are based on TIFF-EP and NEF & CR2 file formats are so close to DNG that a camera firmware update could prolly allow current cameras to write DNG in addition to their own formats.

There is nothing in Nikon and Canon's desire to keep their formats proprietary that is in the user's best interest...so, as a photographer I can't see any reason to support Nikon and Canon refusing to accept a standard...and any photographer who does support Nikon and Canon either don't understand the issues or have their own agenda in play.

There is one area where DNG is not a benefit with Lightroom and that is writing XMP metadata into the DNG files and then backing up. If you do choose to write the XMP metadata into the DNG files, their modification dates will be changes (because of the XMP metadata update) and any backup scheme that uses modification date to do incremental backups will see those files as updates cause all of the DNG file to be copied on backup. If you do import the raw files not as DNGs, the only way to save out the metadata is in XMP sidecar files–which compared to the image data is tiny.

But that's the only downside of using DNG if you are using Camera Raw/Lightroom...
budjames
QUOTE (Schewe @ Jul 22 2008, 02:12 PM)
If you are talking about Camera Raw or Lightroom's ability to render either the original raws of DNGs, there is ZERO differences...in point of fact, Camera Raw's first steps when opening a raw file is to convert that data into DNG data (not a DNG file, just DNG data) in order to process it. I'm pretty sure Lightroom does the same thing.

So, from the standpoint of CR/LR, there CAN'T be any difference in rendering because the rendering is being based on the exact same data.

Look, there's a lot of misconception and myth regarding raw image data. Some people think the only Nikon and Canon can possible know what the raw files' secret data means...and to an extent it's true that the undocumented data does produce problems for 3rd parties. But if you look at the experience and knowledge Adobe has obtained regarding raw files, I think that with over 170 raw file formats under their belt, Thomas Knoll and the other Camera Raw engineers pretty much know exactly what's in those raw files.

There are some real advantages regarding DNG...the raw data & xmp metadata is contained in a single file. DNG has completely lossless compression. DNG is fully documented and for long term conservation and preservation, that's critical. DNG, as a raw file format, has enormous 3rd party support which can not be said for all the 170+ raw file formats which can be turned into DNG. Adobe is also in talks with the ISO to allow the ISO to take over stewardship of DNG. Adobe has already allowed the ISO to take over portions of TIFF for the development of TIFF-EP (Tiff for electronic photography) and they have a lot of experience in standardized formats such as PDF and initiatives such as XMP.

On the contrary, the camera companies (particularly Nikon and Canon) continue to keep their formats proprietary and undocumented and "secret" even if their formats are based on TIFF-EP and NEF & CR2 file formats are so close to DNG that a camera firmware update could prolly allow current cameras to write DNG in addition to their own formats.

There is nothing in Nikon and Canon's desire to keep their formats proprietary that is in the user's best interest...so, as a photographer I can't see any reason to support Nikon and Canon refusing to accept a standard...and any photographer who does support Nikon and Canon either don't understand the issues or have their own agenda in play.

There is one area where DNG is not a benefit with Lightroom and that is writing XMP metadata into the DNG files and then backing up. If you do choose to write the XMP metadata into the DNG files, their modification dates will be changes (because of the XMP metadata update) and any backup scheme that uses modification date to do incremental backups will see those files as updates cause all of the DNG file to be copied on backup. If you do import the raw files not as DNGs, the only way to save out the metadata is in XMP sidecar files–which compared to the image data is tiny.

But that's the only downside of using DNG if you are using Camera Raw/Lightroom...
*


Jeff, as always, thanks for your detailed explanation. I agree with you since I'm a follower (and purchaser of your products and tutorials). I learned a lot from the tutorials that you and Michael have produced, so keep them coming.

As a photo hobbyist, although a serious one, the workflow offered by using LR and PS CS3 instead of Canon's DPP serves me well as to keep me from having to deal with unfriendly software that does not have such a great community of sharing pros like yourself to support and encourage the learning process like Adobe products.

My prints and images look great so I quite happy with the results and the virtual network of similarly minded individuals on forums like this.

Cheers.
Bud James
North Wales, PA
Panopeeper
Before we burst into tears, because not all files have been converted in DNG, a few things have to be explained.

Note, that the issue is NOT if converting native raw files in DNG causes loss of raw data. Such claims came only from amateurs.

To the evaluation of "advantages" coming with DNG:

1. Embedded XMP tags.

Well, you may clasp for this, or you may whine about it. It may seem to be an advantage for some, it is a show stopper for others. I deem it as an idiotic idea to store the adjustment parameters (secondary, changable information) inside the original, primary information, which ought to be preserved absolutely unchanged.

2. Lossless compression.

I don't know if I am supposed to laugh or to whine when reading this rubbish. The raw data is lossless, if is has been created losslessly. However, the lossy raw data does not become lossless by virtue of converting it in DNG.

3. "Enormous support".

It is totally irrelevant for any user, how many cameras are supported by the software (s)he is using.

4. Some (in fact, all important) manufacturers are sticking to their proprietory format.

I have all understanding for them. The DNG format does not allow for any new features, it does not allow for honouring in-camera settings (contrast, saturation, style, color space selection, etc.), and most importantly, it does not allow for the best possible color representation.

This is a very difficult problem. On one hand, the DNG specification can not account for future features. On the other hand, manufacturers don't want to reveal their plans, nor do they want to delay their release, until the DNG specification gets updated.

For example the proprietory raw converters honour picture styles and dust delete data. If someone goes DNG, one can forget about that.

5. DNG is a standard.

Well, it is a standard of Adobe. I hope, that before it will be declared an official standard, some software architecs will be asked about it, because this did not happen until now.

Let's keep an important fact in eyes: camera owners are using DNG due to Lightroom, Bridge and mainly for the features of ACR, not for DNG. If Canon came out with a raw converter equipped comparable to ACR, photogs would make a run for it and forget about ACR.

(Please note: I am a proponent of a raw standard, something like DNG, but I am not blind.)
budjames
QUOTE (Panopeeper @ Jul 22 2008, 07:29 PM)
Before we burst into tears, because not all files have been converted in DNG, a few things have to be explained.

Note, that the issue is NOT if converting native raw files in DNG causes loss of raw data. Such claims came only from amateurs.

To the evaluation of "advantages" coming with DNG:

1. Embedded XMP tags.

Well, you may clasp for this, or you may whine about it. It may seem to be an advantage for some, it is a show stopper for others. I deem it as an idiotic idea to store the adjustment parameters (secondary, changable information) inside the original, primary information, which ought to be preserved absolutely unchanged.

2. Lossless compression.

I don't know if I am supposed to laugh or to whine when reading this rubbish. The raw data is lossless, if is has been created losslessly. However, the lossy raw data does not become lossless by virtue of converting it in DNG.

3. "Enormous support".

It is totally irrelevant for any user, how many cameras are supported by the software (s)he is using.

4. Some (in fact, all important) manufacturers are sticking to their proprietory format.

I have all understanding for them. The DNG format does not allow for any new features, it does not allow for honouring in-camera settings (contrast, saturation, style, color space selection, etc.), and most importantly, it does not allow for the best possible color representation.

This is a very difficult problem. On one hand, the DNG specification can not account for future features. On the other hand, manufacturers don't want to reveal their plans, nor do they want to delay their release, until the DNG specification gets updated.

For example the proprietory raw converters honour picture styles and dust delete data. If someone goes DNG, one can forget about that.

5. DNG is a standard.

Well, it is a standard of Adobe. I hope, that before it will be declared an official standard, some software architecs will be asked about it, because this did not happen until now.

Let's keep an important fact in eyes: camera owners are using DNG due to Lightroom, Bridge and mainly for the features of ACR, not for DNG. If Canon came out with a raw converter equipped comparable to ACR, photogs would make a run for it and forget about ACR.

(Please note: I am a proponent of a raw standard, something like DNG, but I am not blind.)
*


This is not a political debate. Do what you want for your purposes. By making it easy, consistent and repeatable, Adobe and DNG will yield better results in the long-term. I'll stick to DNG, you do as you please.

Bud
ErikKaffehr
Hi,

Can you explain "It does not allow for the best possible color representation.", please.

Best regards
Erik


QUOTE (Panopeeper @ Jul 23 2008, 02:29 AM)
I have all understanding for them. The DNG format does not allow for any new features, it does not allow for honouring in-camera settings (contrast, saturation, style, color space selection, etc.), and most importantly, .


*
Panopeeper
QUOTE (ErikKaffehr @ Jul 22 2008, 08:21 PM)
Can you explain "It does not allow for the best possible color representation.", please

The DNG specification prescribes a certain method of converting the colors from the camera's color space in RGB (you can see often references to "color conversion matrixes"). The result of this conversion using the default matrixes is far from being optimal. The calibration process shows the weakness: only three or four colors can be included.

Other programs adopt different approaches of color conversion. The manufacturers' own converters can reproduce the colors better than ACR (there is no real discussion about it, only a few fanatics would dispute this). The difference may not be relevant for some (like myself), but it is the show stopper for others.

DNG version 1.2 has been published a few weeks ago; many of the changes are the implicite acknowledgment of this problem. It has to be shown yet, if the problems are solvable with the extentions.
ErikKaffehr
Thanks!

QUOTE (Panopeeper @ Jul 23 2008, 07:08 PM)
The DNG specification prescribes a certain method of converting the colors from the camera's color space in RGB (you can see often references to "color conversion matrixes"). The result of this conversion using the default matrixes is far from being optimal. The calibration process shows the weakness: only three or four colors can be included.

Other programs adopt different approaches of color conversion. The manufacturers' own converters can reproduce the colors better than ACR (there is no real discussion about it, only a few fanatics would dispute this). The difference may not be relevant for some (like myself), but it is the show stopper for others.

DNG version 1.2 has been published a few weeks ago; many of the changes are the implicite acknowledgment of this problem. It has to be shown yet, if the problems are solvable with the extentions.
*
The View
QUOTE (Panopeeper @ Jul 23 2008, 09:08 AM)
The DNG specification prescribes a certain method of converting the colors from the camera's color space in RGB (you can see often references to "color conversion matrixes"). The result of this conversion using the default matrixes is far from being optimal. The calibration process shows the weakness: only three or four colors can be included.

Other programs adopt different approaches of color conversion. The manufacturers' own converters can reproduce the colors better than ACR (there is no real discussion about it, only a few fanatics would dispute this). The difference may not be relevant for some (like myself), but it is the show stopper for others.

DNG version 1.2 has been published a few weeks ago; many of the changes are the implicite acknowledgment of this problem. It has to be shown yet, if the problems are solvable with the extentions.
*


Sounds very interesting. So, conversion to DNG is lossless, but... you may lose colors?
The View
I actually don't need DNG.

I'm perfectly satisfied with Canon's proprietary CR2 format.

There aren't actually THAT many RAW formats out there.

Don't we have a different lens mount for every camera system, too?

On the other hand, it seems to be critical to try to invent an artificial standard.

Remember the M42 lens mount?

I feel DNG limits me, as many software applications don't accept it. Lossless, or not, this seems a loss to me.
Panopeeper
QUOTE (The View @ Jul 23 2008, 09:46 PM)
So, conversion to DNG is lossless, but... you may lose colors?

Well, this term is correct if you make a direct comparison and find, that DPP creates more fidel colors, than ACR.

However, one has to consider, that the camera can be profiled in ACR. Some users have been doing that and reported better results, but allegedly still not as good as DPP. Apologists of the ACR approach counter, that the profiling needs to be done better, but that is ridiculous, as no-one came up with a perfect profile.

AFAIK it is not proven, that it is possible at all to adjust the ACR conversion with the result equalling the rendering of a camera-specific conversion (you can read sometimes about LUT or look-up table, meaning a very practical, sensor specific conversion instead of the generic approach).

The new DNG spec provides for additional profiles (these profiles have nothing to do with the profiling available on the user interface of ACR). However, these are still based on the basic approach of primary conversion by the matrixes; I don't know, how much this will help (I am not saying, that this is not an adequate solution; I don't know).

Adobe resisted to calls for allowing ICC profiles in the raw conversion; I don't know their reason.
madmanchan
My understanding is that DNG support in non-Adobe apps is on its way, but will take some time to get to market.

There are some development concerns that we (i.e., members of the CR team at Adobe) have that end up trickling down to hurting users, in our opinion. Cameras with new formats take longer to support, which means a longer wait between the time a photographer gets a new camera and when the software will support it. Sometimes the new format is easy to add support for and can be done within a few days of work, thereby meaning the software is ready by the time the new model hits the streets and ends up in users' hands. Other times it takes weeks, or even months. A couple of new models that Michael covered recently on this site fall have new formats, unfortunately. They'll take extra time to support. With a bit homework you can probably figure which ones they are.

I also believe this indirectly hurts users because we get a lot of feature requests from users. Time spent deciphering new formats just to support a camera means less time spent working on improvements.

In general camera formats aren't very interesting. Photography, on the other hand, is very interesting. We'd rather spend time building better tools for developing photographs rather than figuring out what the 1's and 0's in the file mean. Ick.

BTW, Gabor, the DNG 1.2 spec allows a camera color profile to use 1 or more lookup tables, in addition to matrices. It is certainly true that none of the current profiles (built prior to DNG 1.2 development) contain any such lookup tables; the current profiles only use matrices to map camera 3-color or 4-color data to XYZ.
Schewe
QUOTE (The View @ Jul 23 2008, 11:49 PM)
There aren't actually THAT many RAW formats out there.

Don't we have a different lens mount for every camera system, too?

On the other hand, it seems to be critical to try to invent an artificial standard.
*



So, 170+ file formats isn't a lot of different file formats? That's what DNG/Camera Raw supports. And 170+ would be a problem if you were a software engineer that had to keep track of each and every new camera model's file format. Yes, there's NEF and CR2 (Nikon and Canon) but each and EVERY camera model produces a DIFFERENT file format, even if the file extension is NEF or CR2.

Take Canon for example...they've produces digital cameras whose file format went from CRW to TIF to CR2...that's three changes in base level file formats from one company in less that 8 years. Take Nikon...in their marketing stupidity they named digital files from their cameras NEF. What do they name files from their scanners? NEF...are they the same? Nope...not even close.

Until and unless the digital camera companies do finally arrive at a standardized file format, the long term preservation and conservation of today's digital capture is at severe risk. The fact that YOU don't see it is actually part of the problem, not the solution. And DNG is not an "artificial" solution...it's a well designed and implemented solution by a company with a track record for producing standards...something neither Nikon nor Canon have much experience with.

So, until you education yourself a bit more about the issues involved, it would be useful to quit making broad and sweeping and technically incorrect statements. Try doing some reading...Digital Preservation. In fact Congress has spent nearly 100 million dollars looking into the issue, see: The Library of Congress. You'll find that undocumented and proprietary file formats are the WORST thing for the industry.

And to have somebody like you pass off the efforts as a "artificial standard" is simply not useful (and ignorant).
Panopeeper
QUOTE (madmanchan @ Jul 24 2008, 09:06 AM)
the DNG 1.2 spec allows a camera color profile to use 1 or more lookup tables, in addition to matrices

I did only a cursory reading of this subject yesterday, but my understanding is, that the hue-saturation mapping is an addition to the primary color space conversion, which remains unchanged (based on the conversion matrixes).

I can not judge, if this addition is enough to achieve an adequate conversion. At first sight, one could say this allows for any transformation; however, that is not so. The first, matrix based transformation leads to invalid results for many source co-ordinates, i.e. many colors can not be converted in XYZ, thus the following hue-saturation conversion is helpless. While this may not be totally avoidable by any method, I don't see it as given, that a sensor-specific conversion instead of the conversion by matrix would not yield a result preferrable to some customers.
The View
QUOTE (Panopeeper @ Jul 24 2008, 10:35 AM)
I did only a cursory reading of this subject yesterday, but my understanding is, that the hue-saturation mapping is an addition to the primary color space conversion, which remains unchanged (based on the conversion matrixes).

I can not judge, if this addition is enough to achieve an adequate conversion. At first sight, one could say this allows for any transformation; however, that is not so. The first, matrix based transformation leads to invalid results for many source co-ordinates, i.e. many colors can not be converted in XYZ, thus the following hue-saturation conversion is helpless. While this may not be totally avoidable by any method, I don't see it as given, that a sensor-specific conversion instead of the conversion by matrix would not yield a result preferrable to some customers.
*


Thank you for your view that is not biased by relations to any particular software company.

Of course, everybody who stands up for his point of view, will have to take a certain amount of abuse of people who would love to ignore how opinions are formed and knowledge is gained: by discussion and competition out there in the real world.

But still, this the price of having a backbone and the ability to judge and draw conclusions is worth it. After all, only corpses drift with the flow.

So, Panopeeper, I really appreciate your standing up here and allowing readers to gain knowledge. People like you enrich this forum as a way to get knowledge, that is possible drowned out by the chorus otherwise.

Generally, the quest for maximum quality and practicability should never be underestimated. The age of digital photography is exiting its phase of infancy, and a solution will be found, one way or other.
The View
It is an honorable attempt for any company to look for a general standard, a kind of lingua franca of RAW conversion.

At the same time, while the attempt is good, it would be hubris for such a company and anybody who works for it, to proclaim such a standard, and pursue those who don't agree, as infideles.

Especially when that certain standard is not that future proof - and that's the way it looks if you attempt to read Panopeepers comment instead of just browsing to discard.

If Canon, Nikon, and others don't take part in DNG this has surely reasons that may be economical, "political" (market power related), or technical (does not give most possibilities, like color accuracy and the ability to store proprietary information).
The View
There have been countless attempts for unity of any kind of language in any field.

Not only in M42 lens mounts...

Think of Esperanto. An artificial language, put together from English, Spanish, French, and other European languages. It provided a simple grammatical structure, simple declinations of the verbs, no "weird" (historically grown) rules, and a very practical, easy to memorize vocabulary.

And it was a failure. Esperanto was an honorable attempt, but failed, because it was an artificial standard, that did not include what people need in a language, and what goes beyond communicating information.

Technical standards can and have repeatedly hampered technological development, and to claim otherwise is actually more ignorant than the insolence of ignoring Adobe's supremacy in the development of software standards.

If someone invented a standard camera companies could customize to their needs, it would stand a chance, and it may happen.

Any other standard, that does not find support, is per definitionem, artificial. Because it only acts like a standard, but isn't. Standards become standards through widespread acceptance, not by definition.
DarkPenguin
I don't know that what panopeeper said was a direct problem with DNG. I think it is fallout from the fact that a lot of camera makers software doesn't read DNG.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2008 Invision Power Services, Inc.