Samsung has just released a 108 MP phone sensor – a wild headline number that may mean little in practice. Is 108 MP in a phone valuable? Is it anything like 100+ MP in a larger camera? 50 MP? Even a good 24 MP APS-C or full-frame sensor? Or is it just sheer numerical hype?
We won’t really know until we see raw files not just from this sensor, but from the sensor, lens and processing hardware that are going to be sold together in some new Xiaomi phone. There are several reasons to think that it’s extremely unlikely to be a GFX 100 competitor, and somewhat unlikely to compete with 24 MP APS-C or Micro 4/3. Much depends on how high quality the sensor is, and on what kind of a lens it’s paired with. My best estimate is that it will be used in a slightly unwieldy phone with a substantial camera bump that has image quality in the range of a Sony RX100 or other quality compact camera with a 1” sensor, although likely with different strengths and weaknesses.
Is there any point to more quality than that in a phone, anyway? Is there any point to even that level of quality in a phone? What if it compromises the other uses of the phone by making it heavier, bulkier, harder to hold, or by reducing battery life? The most important purpose of a phone camera is for rapid image sharing online, and the platforms limit resolution, and to a lesser extent other aspects of image quality as well (dynamic range is limited by aggressive JPEG compression, and color quality by sRGB). How many Luminous Landscape readers have ever made a print from their phone cameras? How many have made than five prints greater than 5×7” from a phone? Do you often display phone images very large on a 4k or greater screen? If you don’t do these things with your phone, an increase in your phone camera’s image quality won’t help you. An increase in your main camera’s connectivity might, since it would allow quick sharing of images that are also destined for other uses…
There have been snapshot cameras at least since the original Kodak Brownie, and they have always had an important role in documenting our daily lives – a role they play best when they’re tiny and always with us. Anything that makes a snapshot camera bigger and more unwieldy makes it less useful for its purpose. Now that our snapshot cameras live in our phones, we also need to consider what size and shape of object makes a useful telephone, messaging device and pocket computer. Anything that makes a device less useful for those purposes is a detriment.
To use a phone camera for images other than snapshots, there are a number of features it lacks, even beyond its image quality. Unless you are Henri Cartier-Bresson, you might prefer some choice in lenses, whether it be a zoom or two or more than one prime. A viewfinder is at the very least a nice thing to have, and many would argue a necessity, for holding a camera steady and composing a thoughtful image. Is a touchscreen a substitute for every other control on your camera? I personally find my iPhone a very unpleasant camera to use because it depends exclusively on its touchscreen – and I often forget that my Z7 even has a touchscreen until I accidentally alter something with my nose.
Adding the controls and features to a phone to make it a more versatile camera would make it a less good phone. Its role as a phone/portable media device depends very heavily on its form factor – a non-slim phone wouldn’t fit so well in all the pockets, purses and bags we shove them in. It also depends on battery life, and a big camera will both suck up space that could go to the battery and use considerably more power. Even a shutter button you’d want to use would add either thickness or bezel to the phone (both of which phone manufacturers are trying to eliminate), and a physical aperture/shutter speed/exposure compensation dial would add quite a bit. Trying to cram a decent viewfinder, a grip, or both in would add considerably to every dimension.
Six cameras, each with a different sensor size and purpose. iPhone 6s+ (1/2.5”) (front), then (left to right) Sony RX100 mk IV (1”), Panasonic LX100 (4/3”), Fujifilm X-T2 (APS-C), Nikon Z7 (full-frame), Fujifilm GFX100 (medium format) .
The image above shows six cameras, with sensor size increasing as you move from the iPhone in front to the GFX 100 at right. Apart from the iPhone, all of them have zoom lenses that range from somewhat wide to slightly long (the GFX 100 is wide to normal). With each increase in sensor size comes an increase in camera size. As you accept a bigger, heavier camera, the pixel count trends upward (although the LX100 actually has fewer pixels than the smaller-sensor RX100). In addition to pixels, though, controllability increases, dynamic range increases, color quality improves and versatility increases – the larger-sensor cameras have interchangeable lenses and accept other accessories. Not every factor increases at every size increase – the X-T2, Z7 and GFX100 all have very good full controls, for example – by the time you reach the size and weight of an X-T2, there aren’t many more controls to add.
There are different sizes of cameras for different purposes – phones as snapshot cameras for capturing moments in life are at one extreme – compact and attached to something else that most of us carry, but with limited image quality and control. People who have kids often like to have a good pocket camera like a RX100 (of any version) around – it’s still small and light enough not to think about while chasing the kids, but it makes small prints and little photo books look a lot better than a phone. I personally prefer a top-end mirrorless camera with excellent, but compact lenses (I presently own a Nikon Z7) – because I am a landscape photographer who likes to print big, but I also tend to find my best images deep in the backcountry, so a camera I can carry for miles is critical, and so is one that is rugged enough to survive long backpacking trips.
I’m a photographer (with a background in science, but it’s ecology, not optical physics or semiconductors), not a physicist or engineer, so I may have made a mistake or two in how sensors and lenses work. There are barriers in the realm between physics and engineering to getting a lot more quality out of small sensors.
On the most basic level (and ignoring color for the moment), a digital camera sensor works like a rectangular array of baskets (pixels), while the incoming light is a stream of basketballs (photons), each of which will land in one of the baskets. The lens focuses the stream of balls so they form an image, based on how many balls land in each basket, while the electronics count balls in each basket. The more balls in a given basket, the brighter the pixel represented by that basket is.
The more baskets you have, the more pixels form the image – and the sharper the image can be. To get more baskets, you can either spread the baskets out over a larger area or make each basket smaller over the same area. Moving from a phone to a small-sensor compact to Micro4/3 to APS-C to full-frame to medium format increases the number of baskets by increasing the sensor area, while moving from a 12 MP D700 to a 24 MP D750 to a 46 MP D850 (all of which use the same size sensor) increases the number of baskets by making each of them smaller.
Twice as many balls in a basket always means that the pixel is twice as bright (a 1 stop difference) – whether it is that 16 balls is twice as bright as 8 balls, or that 1024 balls is twice as bright as 512 balls. This means that the theoretical dynamic range of a sensor is the number of doublings it can record (if each pixel can hold 256 balls, it has 8 bits of dynamic range, since 256 is 2x2x2x2x2x2x2x2, generally written 2^8). For a sensor to record 16 bits, it would need to hold 65,536 (2^16) balls per pixel.
There are a small number of random balls, not related to the image, bouncing around for various reasons – they’re called noise. If one lands in a basket that already has 10,000 balls in it (a bright highlight area of the image), it doesn’t matter, since the difference between 10,000 balls and 10,001 balls is imperceptible (1/100 of 1%). If, however, one lands in a basket with four other balls (a deep shadow), it makes a difference, since the difference between four and five balls is 25%.
One problem with making the baskets smaller is that a smaller basket holds fewer balls. For ease of calculation, each basket in a really good DSLR sensor can hold ~32678 balls (this number is called the full well capacity). A sensor with really tiny pixels (the densest phone sensors, including the new Samsung) might have each basket 1/32nd the size of a typical DSLR sensor, and each one only holds 1024 balls. If there was no noise, the DSLR sensor could record 15 doublings of the number of balls, so 15 stops of dynamic range. The phone sensor would only have 10 stops of dynamic range.
Due to various sources of noise, we can’t count the last 8 balls – they could be real signal, they could be noise, but we aren’t sure enough to say. We lose the bottom 3 stops, and the DSLR sensor still records 12 stops of usable dynamic range, but the phone sensor only has 7 stops. These numbers aren’t quite real, since I used exact powers of 2 to make the calculations come out clean – but they’re in the right range (and the magnitude of the difference is about right).
Both determinants of dynamic range, full well capacity and noise, improve gradually over time. DSLRs have gone from approximately 6 1⁄2 or 7 stops of photographically useful dynamic range on the Canon D30 and Nikon D1 in 2000 to around 11 or 12 stops today. At the same time, the pixels have gotten much smaller and there are many more of them, because the sensors have gotten a great deal better. At the same time, we have gone from “interesting” colors (to put it mildly when I look at a file from a very old camera) to today’s color depths and qualities that offer nearly unlimited latitude to choose what you want the print to look like in post-processing.
What if we made the baskets on the phone sensor 32 times taller, so they’d still hold 32,768 balls and give us an engineering dynamic range of 15 stops, with 12 of those really usable in a photo, even though their area is 32x smaller than the baskets on the DSLR sensor? First of all, that’s hard – building a sensor that can do that would be a major breakthrough (and DSLRs would quickly start using sensors like it, regaining their advantage). Speaking very roughly, a 108 MP phone sensor with 12 stops of dynamic range and color like we’re used to would be about as big a step from a modern full-frame sensor as a modern full-frame sensor is from the very first large CMOS image sensor to find its way into a consumer camera, in the Canon EOS-D30.
The new Samsung sensor almost certainly doesn’t give us 20 years of progress in one sensor (which would probably have to be an entirely new technology). It appears from the press release to be relatively conventional apart from its tiny pixels and larger than usual (for a phone) area.
Even if it were a breakthrough sensor that somehow had five stops more dynamic range than it “should” using current technology, there’s still a problem. Modern sensors are more than 90% efficient at capturing and counting photons that reach them. If you reduce the area of each pixel to 1/32nd of what it has been, while trying to capture the same number of photons, the sensor requires 32 times as much light per square millimeter, since it needs the same amount of light per pixel, but each pixel is 1/32nd the size.
If a good modern sensor might have a native ISO of 64, the phone sensor would have to have a native ISO of 2. ISO 2 means either extremely fast lenses or very long exposures. Even in absolutely full sun, according to the Sunny f16 rule, the required exposure would be 1⁄2 second at f16. Maybe the equivalent 1/60 second at f2.8 is more palatable – but remember that this is in full sun. If we’re in enough shade that the ISO 64 exposure is 1/60 second at a reasonable f4, the phone sensor requires 1/60 second at f0.7.
if you want to cram more photons into a smaller space, you need either a brighter lens or a slower shutter speed. If you raise the ISO, you’re capturing fewer photons and losing dynamic range (so simply setting the ISO 2 phone sensor back to ISO 64 puts you right back to 7 stops of dynamic range – each pixel might hold 32,768 photons, but it only has time to capture 1024).
There’s another, related problem with a sensor with a lot of very small pixels. As we’ve seen above, it will either have limited dynamic range or a ridiculously low native ISO. Since the baskets are very close together, it will also require an extremely sharp lens. A high-resolution DSLR or mirrorless camera like my Nikon Z7 has about 230 pixels per millimeter, meaning that the finest detail it can capture is determined by its lens’ performance at 115 line pairs/mm (it takes two pixels to show contrast, so the resolution of a lens is expressed in line pairs/mm, which is half the number of pixels/mm). Good to excellent modern lenses show some contrast at 100+ lp/mm, so there’s something being captured at that resolution.
Samsung’s new cell phone sensor, on the other hand, has a resolution of over 1000 pixels/mm, requiring a lens that can resolve in excess of 500 lp/mm to take full advantage of its resolution. Few if any lenses resolve much contrast at 500 lp/mm (there’s a reason why lens resolution charts only go up to 50 lp/mm). If there are lenses with resolving power like that, they are probably special-purpose devices found on spy satellites or inside optical lithography devices that manufacture microchips.
In addition to the lens needing to be extremely sharp, there is also diffraction to deal with. The published specifications in Samsung’s press release show a pixel size of 0.8μ. Using the diffraction calculator at Cambridge in Colour, a sensor with 0.8μ pixels begins to show diffraction between f1.4 and f1.8, meaning that even a perfect lens would also have to be fast and perfect wide open.
If such a lens can be built, what would it look like? Would it fit in a phone? Samsung isn’t publishing the exact physical size of the new sensor, but there is enough information out there to figure out that it is somewhere in the vicinity of 7x10mm or 8×11 mm. That is quite a bit larger than a standard cell phone sensor (around 5x7mm with some variation), while substantially smaller than the “1 inch” sensor used in the popular Sony RX 100 series compacts. Conveniently, it’s about the size of a standard 16mm movie film frame, or of the sensors used in the Pentax Q digital cameras, which featured interchangeable lenses (especially the later models, which used a slightly larger sensor), so we have some idea of what the lenses might look like.
A standard lens for that format would be somewhere in the vicinity of 12 to 14mm, depending on the exact size of the sensor. Cell phone manufacturers tend to prefer a somewhat wide lens, so something in the range of 8 to 10 mm is probably most likely. Both a Pentax Q lens and some high-end Carl Zeiss cinema lenses exist in the right focal length range, covering similar formats.
An Ultra 16 prime – at 1 kg and $15000 or so, not a terribly practical phone lens
The Carl Zeiss Ultra 16 primes meant for 16mm movie film (and similarly sized digital cinema sensors) will perform well on the very high resolution sensor if anything will. They are as sharp as any commercially available lens – Carl Zeiss technicians sometimes call them “the best lenses we’ve ever made”. They are fast enough to avoid diffraction even on a 108 MP sensor at T1.3 (T-stop is how much light actually gets through the lens, and is always slightly slower than f-stop, which is calculated from the size of the aperture) Unfortunately, their size, weight and price are intimidating. The 8mm and 9.5mm Ultra 16 lenses are around 75mm long and 80mm in diameter, weigh nearly a kilogram each, and cost over $15,000 apiece.
Much of the size, weight and cost of the Ultra 16 lenses is in their mechanical housing. They are also expensive because they are largely handmade in small quantities. Even If 4/5 of the cost and weight of the Zeiss lenses is due to their mechanical construction, a comparable mass-produced lens in a lightweight housing might still weigh something like 200 grams and cost $3000.
An 8.5mm Pentax Q lens – still not small enough for a phone, but closer.
Moving towards something more reasonable in size and cost there is the Pentax Q “Standard Prime 8.5mm f1.9”, which weighs 36 grams and costs $200. It is 23mm long and the optics are approximately 20mm in diameter – although it takes 40.5mm filters, there is a significant plastic barrel surrounding the actual optics to provide a grip. Even the little Pentax lens would create a very significant camera bump on a phone – an iPhone X is only 7.7mm thick, meaning that the camera bump has to be three times the thickness of the phone. The lens would account for almost 1⁄4 of the weight of the phone (an iPhone X weighs 174 grams). Unfortunately, the little Pentax is not enough lens to take full advantage of the sensor, either. It’s about 2/3 of a stop too slow to avoid diffraction on a 108 MP sensor. It’s also very well matched to the 12 MP resolution of the Q system, with little indication that it has a lot of performance to spare for a much higher resolution sensor.
If the Zeiss lenses suggest that a lens to match the sensor might weigh in the hundreds of grams and cost thousands of dollars – obviously prohibitive for a phone – the next question might be “how much would the Pentax lens grow in size, weight and cost if it needed to perform up to those standards”? Looking at the difference between a modern 50mm f1.8 lens of reasonable performance and a very high-performing 50mm f1.4 gives some idea of what the step up in aperture and performance might cost in size, weight and expense.
A small, light and inexpensive Sony 50mm lens
And a much larger, extremely sharp Sony 50mm lens.
Sony has two convenient lenses; the inexpensive 50mm f1.8 FE and the extremely sharp 50mm f1.4 FE Zeiss. The ratio in physical length is nearly 2:1, while the ratio in both cost and weight is roughly 5:1. If we apply those ratios to the Pentax lens, a high-performance 8.5mm f1.4 might protrude from the phone by about 40mm, weigh around 200 grams (more than the phone) and cost in the vicinity of $1000. Even that is hardly an appealing phone lens!
If a phone actually had a lens like that paired with the new Samsung sensor, it could produce an image of extraordinary resolution, but would be limited in other ways. It would probably still have around 7 stops of dynamic range, since that is affected by the full well capacity and noise of the sensor, not by the lens. It is likely that the color depth and saturation would be poor – again, small-pixel sensors tend to have color problems. We won’t know until we see files, but a 108 MP small sensor is unlikely to be a photographically useful tool under most circumstances.
Far more likely than actually using the sensor at full resolution (other than in a “marketing stunt mode”), the Samsung sensor is designed to have its pixels binned to 27 MP. The binned sensor is still a challenge for lenses, but not nearly impossible. It now needs a lens that resolves some contrast at around 250 lp/mm (still tricky, but possible for an excellent small-format lens), and it is diffraction limited around f2.8. It should gain two stops of ISO and
dynamic range. Color and saturation could improve enough that they are in the category of “up in the air – maybe pretty decent”, instead of “almost certainly awful”.
A Sony RX100 mk VII – the latest generation of a successful line of compact cameras.
A 20+ MP camera with 9 stops of dynamic range and decent color will perform something like a competent 1” sensor compact. The resolution will be a little better, perhaps noticeably so, while the color might not be as good. It will have its own quirks, but “something like a RX100” is a reasonable range of performance expectation.
A lens around the size and weight of the Pentax, especially with the benefit of a fully modern design, and without the design pressures of interchangeability, might serve this camera. The question is whether anybody wants a phone like that? The camera bump will still be several times thicker than the phone itself, the lens will add enough weight that it is significantly heavier than any other phone in the same size class, and the camera system will add at least $200 to the phone’s selling price. Are these drawbacks worth it for a RX100 class camera with a fixed prime lens? Every previous time phone manufacturers have tried something similar, it didn’t sell. Panasonic once offered a Lumix-branded phone with a 1” sensor and a Leica-branded lens, while Motorola offered a Hasselblad-branded “Moto mod”
that attached to the back of several of their phones. Nokia played around with oversize, high- resolution sensors several years ago. None of them went anywhere – probably because they were decent cameras, but lousy phones. All of them were big, thick, expensive phones with relatively small screens running old versions of Android (or in the case of Nokia, Windows Phone) that were hard to upgrade because the camera software made new versions tricky. Expensive phones tend to be very slim and elegant, and to run iOS (or sometimes the latest version of Android, but Apple outsells all other expensive phones combined – Android’s huge market share is mainly due to inexpensive phones).
Fundamentally, the purpose of a phone camera is to take snapshots to share on social media at very low resolution. Instagram’s maximum resolution is barely over 1 MP, and no other major social network supports anything over 2 MP. Very few web pages use images much over 1 MP, and the few that do are photography-oriented sites that make provision for high- resolution displays. E-mail varies by provider, and it allows higher resolution than any of the social networks, but still generally under 6 MP. The only common ways to send really high resolution images over the web involve downloads or FTP. Why make the sacrifices in cost and convenience to get high quality 20+ MP images when they will simply get cut right back down to 2 MP or less with high compression and limited color? 8-12 MP cameras that can use lenses with only a couple of mm of camera bump are perfectly adequate for 2MP sharing, and they preserve the size and shape of the phone.
Dan Wells
August 2019
Author
Elevate Your Vision
You may also like
