Understanding RAW Files

By Michael Reichmann originally posted March 2, 2011

There is probably more misinformation, disinformation, and lack of information regarding what raw files are all about than on any other digital imaging topic. There are also many reasons why one should shoot in raw mode, but also quite a few reasons why many people don’t. Let’s have a look at both.

When a digital camera makes an exposure to the imaging chip (whether it’s CCD or CMOS) records the amount of light that has hit each pixel, or photo site. This is recorded as a voltage level. The camera’s analog to digital circuitry now changes this analog voltage signal into a digital representation. Depending on the camera’s circuitry either 12 or 14 bits of data are recorded. Incidentally, if the camera records 12 bits of data then each pixel can handle 4,096 brightness levels (2^12), and if 14 bit then it can record 16,384 different brightness levels (2^14). (To my knowledge no current imaging chip records a true 16 bits worth of data).

Of course, what happens after you’ve taken the photograph depends on whether you have the camera set to save images to the memory card as raw files or JPGs.

If you’ve saved the file in raw mode when it is subsequently loaded into a raw conversion program and then saved to a TIFF or PSD format file it can be exported in 16-bit mode. The 12 or 14 bits recorded by the camera are then spread over the full 16-bit workspace. If you’ve saved the file in-camera as a JPG then it is converted by the camera’s software to 8-bit mode and you will only ever have 256 brightness levels to work with.

Spanish Geometry — Seville, Spain. April, 2004

Minolta A2 at ISO 64. 150mm Equiv.

Saving the Raw file

If you are saving raw files the camera creates a header file which contains all of the camera settings, including (depending on the camera) sharpening level, contrast and saturation settings, colour temperature / white balance, and so on. The image is not changed by these settings, they are simply tagged onto the raw image data.

This raw image data, what the imaging chip recorded along with the so-called meta-data (the camera settings and other technical information) is now saved to the card. Some cameras compress these files, others don’t. In any event if they are compressed it is done losslessly so that there is no deterioration of the file due to compression artifacts. (Some companies, Nikon and Kodak specifically, use a slightly lossy compression algorithm when saving raw files).

Saving the JPG File

If you have elected to have the camera save the file as a JPG, here’s what happens…

— By itself no sensor can record colour. In all imaging chips except that from Foveon (which uses three separate layers of silicon, one for each primary colour) all imaging chips use what is called a Bayer Matrix or colour Filter Array in order to record colour. The way this is done is to place red, blue and green filters over each pixel. Half of the pixels are filtered green and the remainder are either red or blue. Through a very complex algorithm the values recorded by each pixel are compared with its neighbors, and full colour information is derived. And, just to complicate things, the Sony F828 uses a four colour array pattern with Emerald Green as the fourth colour.

— Imaging chips are linear devices. That means that as twice as much light hits one pixel than another it will produce twice the voltage. But F stops and the way brightness values are rendered is logarithmic. This means that an uncorrected image appears very dark, and therefore a tone curve needs to be applied to the data to recreate a natural looking scene.

— Digital images that have not had sharpening applied look flat, low contrast and therefore unsharp. People want their out-of-camera JPGs to look good, and so USM (Unsharp Masking) is applied to the file by the camera. USM finds the edges between light and dark areas and enhances the contrast. In doing so halos are produced around the edges. If sharpening is set too high these halos will be visible in a final print. If they are set too low then sharpening may be insufficient. Cameras usually have between 1 and 3 different sharpening settings that the user can choose.

— Contrast can be set on some cameras.

— Colour saturation can be set on some cameras, with usually two or three setting levels.

— The camera converts the file from 12 or 14 bit mode to 8 bit mode. In other words, the 4,096 to 16,384 brightness levels recorded by each pixel are reduced to 256 brightness levels.

— Compression is the final setting for a JPG file. JPG by definition is a lossy format. That means that to make smaller file sizes (which is the whole point of the JPG format) some data is thrown away. If compression is set to a low level (say 2:1) then there is very little information lost and it can be almost impossible to see any data loss. High compression will increasingly cause artifacts to become visible. Most cameras offer at least two or three compression levels.

Brickworks Ruins — Toronto, May, 2004

Panasonic LC1 @ ISO 100. 7mm (28mm Equiv)

Pros and Cons

So now you see the difference. A raw file is essentially the data that the camera’s chip recorded along with some additional information tagged on. A JPG file is one that has had the camera apply linear conversion, matrix conversion, white balance, contrast, and saturation, and then has had some level of potentially destructive compression applied.

If it sounds like I’m editorializing though my emphasis, it’s because I am. But not by that much. The above is what actually happens in each case. Now, before we go further let me add that a JPG file directly from the camera can in many cases produce very high-quality prints. There are in fact numerous valid reasons why one would want to shoot JPG files.

Reasons to Shoot JPG

Files are smaller and therefore more of them fit on a card.

For many applications, image quality is more than sufficient (family snapshots, news images).

Small files are more easily transmitted wirelessly and online. This is important to newspaper photographers.

Many photographers don’t have the time or inclination to post-process their files.

Many cameras (especially digicams) cannot shoot quickly when working in raw mode. Some lower-end models can’t record raw files at all.

Reasons to Shoot Raw

— A raw file is comparable to the latent image contained in an exposed but undeveloped piece of film. It holds exactly what the imaging chip recorded. Nothing more. Nothing less. This means that the photographer is able to extract the maximum possible image quality, whether now or in the future. A good analogy with the traditional world of film is that you have the opportunity to use a different type of developer or development time at any point in the future if one comes along that you think might do a better job of processing the image.

— Raw files have not had while balance set. They are tagged with whatever the camera’s setting was, (either that which was manually set or via auto-white-balance), but the actual data has not been changed. This allows one to set any colour temperature and white balance one wishes after the fact with no image degradation. It should be understood that once the file has been converted from the linear space and has had a gamma curve applied (such as in a JPG) white balance can no longer be properly done.

— File linearization and colour filter array (Bayer) conversion is done on a computer with a fast and powerful microprocessor. This allows much more sophisticated algorithms to be used than those done in a camera with its slower and less powerful processor and with less space for complex conversion programs.

— The raw file is tagged with contrast and saturation information as set in the camera by the user, but the actual image data has not been changed. The user is free to set these based on a per-image evaluation rather than use one or two generalized settings for all images taken.

— Possibly the biggest advantage of shooting raw is that one has a 16 bit image (post raw conversion) to work with. This means that the file has 65,536 levels to work with. This is opposed to a JPG file’s 8 bit space with just 256 brightness levels available. This is important when editing an image, particularly if one is trying to open up shadows or alter brightness in any significant way.

Figures #1 and #2 below shows why. Assuming for this example a 5 stop dynamic range, you can see how much data is found in each of the brightness levels in the image. In other words with a 12 bit file the two darkest levels of the file combined have some 384 brightness levels to work with.

An 8 bit JPG file on the other hand has considerably less. Both the sRGB and Adobe RGB colour spaces use a gamma 2.2 encoding. Gamma encoding reallocates encoding levels from the upper f-stops into the lower f-stops to compensate for the human eye’s greater sensitivity to absolute changes in the darker tone range. Therefore an 8 bit JPG file has just 47 brightness levels available in the bottom two stops. (The remaining levels out of 256 are for the f-stops beyond the 5 in this example).

A 12 Bit RAW file

Within the first F/Stop, which contains the Brightest Tones  2048 levels available
Within the second F/Stop, which contains Bright Tones         1024 levels available
Within the third F/Stop, which contains the Mid-Tones          512 levels available
Within the fourth F/Stop, which contains Dark Tones              256 levels available
Within the fifth F/Stop, which contains the Darkest Tones      128 levels available

A 12 Bit raw File

Within the first F/Stop, which contains the Brightest Tones	2048 levels available
Within the second F/Stop, which contains Bright Tones	1024 levels available
Within the third F/Stop, which contains the Mid-Tones	512 levels available
Within the fourth F/Stop, which contains Dark Tones	256 levels available
Within the fifth F/Stop, which contains the Darkest Tones	128 levels available

Figure #1

An 8 Bit JPG File

Within the first F/Stop, which contains the Brightest Tones	69 levels available
Within the second F/Stop, which contains Bright Tones	50 levels available
Within the third F/Stop, which contains the Mid-Tones	37 levels available
Within the fourth F/Stop, which contains Dark Tones	27 levels available
Within the fifth F/Stop, which contains the Darkest Tones	20 levels available

Figure #2

Now imagine that you want to make a modest adjustment to the file in Photoshop or any other editing program. Which would you rather have to work with, 47 levels or 384 levels? Clearly the 8 bit file will show posterization, which is the effect that one sees when instead of smooth transitions between brightness levels you see abrupt jumps.

— Because a raw file has not been processed in any way, if new and improved methods of linearizing files, applying colour filter array decoding, or other image processing advances are made, you can return to your archived raw files and work on them afresh. A JPG file, on the other hand, is fully baked.

Raw Converters

No two manufacturers encode their raw files in the same way. Indeed many camera makers change their raw format between camera models. (We can only hope that one day the industry well agree upon a common format). Every manufacturer provides software for decoding and processing their camera’s raw files. In most cases, this software is free and provided with the camera purchase. In other cases, this is an extra cost program to be purchased separately.

At the risk of editorializing — almost universally these programs range from poor to simply acceptable, though a couple such as high-end versions from Kodak and Nikon are quite good. Most are frequently very slow, have poor user interfaces and sometimes limited capabilities. Much better are third-party offerings. There are too many to mention here, so I’ll just mention the two most popular and arguably the two best, the Camera RAW converter that is now built into Photoshop CS (it used to be available as a plug-in for Photoshop 7), and Capture One from Phase One.

Both programs are full-featured and have an excellent workflow. Capture One supports most major DSLRs while Photoshop CS now supports virtually every DSLR and digicam on the market, with updates for new models appearing several times a year as new cameras are released.

The major differences between these programs is that you currently need the full Photoshop CS program to use Camera Raw, while Capture One is a stand-alone program that can export converted raw files to subsequently be used with any image processing program. Capture One is also available in different versions for different camera types, with the version for low-end DSLRs costing the least, and that for high-end Pro models the most.

In Summary

Something to consider is that every digital camera is indeed always shooting in raw mode. But, if we choose to save the file as a JPG we are committing to the raw conversion program that is built into the camera. If we allow the file to be saved in raw format though we have the opportunity to do the conversion on a more sophisticated platform, and to do so again and again if there’s any benefit to this in future. In other words, the decision is — do you want to do the raw file conversion now in the camera, or later on your computer?

With a JPG file, you are largely committing yourself at the time of exposure to several of the most important aspects of image quality, namely white balance, overall contrast, colour saturation and the like. With a raw file, you are free to make decisions about these settings at your leisure.

Because JPG files require little or no additional processing when adjustments are made in post-processing, care needs to be taken to keep these within a limited range, or processing artifacts will be seen. For some photographers, the ease and speed of use are a benefit, for others not. Certainly, anyone looking for the best possible image quality will want to shoot in raw mode whenever possible.

Some cameras can save both raw and JPG files simultaneously, and for many photographers, this is an ideal solution. It provides a ready-to-use image for many applications, while a raw file is available for later and more comprehensive processing. The only downside to this double format is the extra space that it takes on memory cards.

Some people complain that raw files are too large and that they take up too much space. With a memory card, hard disk, and DVD-R disks at all-time low prices, the cost for storage is relatively small. It does require good record keeping though to keep track of all of these files, and good back-up and archiving procedures are also required, but that’s a subject for another article.

Rediscover, is just that. A rediscovering of some of the best-archived articles this site has to offer.

Luminous-Landscape is a large website with over 5700 articles, reviews, tutorials, and videos.  As I was going through old articles I saw in many cases how timely they were.  While new advances in technology, new camera models, and new software have appeared the one thing that isn’t new is the basics.

We have gone back to the archives and we are editing and reformatting some of the best articles from the past.  Most of these articles are by Michael Reichmann, founder of this site.  Others are by guest contributors.  All of them will remind us of how important it is to understand the terms and digital basics that will be rediscovered here.

Please keep in mind that these articles may reference old pages, products, and cameras.  While newer models have been introduced the content of the topic hasn’t changed much or at all.  We decided to keep as much of the original content intact.  There may every now and then be a link that is bad, let us know and we will fix them. 

I’d like to thank Michael Tapes of Phase One and Thomas Knoll of Adobe for their feedback and contributions toward this article. If there are any novel insights to be found here it is likely as a result of their feedback. If there are any factual errors they are likely mine. – Michael Reichmann

Michael Reichmann
March 2011 – Redited and posted May 2018

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.