Nikon S4 Review – Test Shots

“Sunlit” Portrait:
(This is my new “Outdoor” Portrait test – read more about it here.)

The lighting in this shot is deliberately awful, about what you’d expect from noontime sunshine here in the Atlanta, GA area. (In fact, the color balance has been chosen to pretty well match the hazy sunshine here in mid-August.)

The reason for the harsh lighting is to provide a real “torture test” of how cameras handle conditions of extreme contrast; and in particular, how well they do holding onto highlight detail.

Look for:

• Overall color: Matching summer sunlight here in the South, the lighting in this scene is a bit more yellow-tinged than that in many parts of the country, or in the fall or winter. – So there may be an overall warm cast to the color. That said though, there’s a fair range of color represented in the bouquet, presenting a tough challenge for the cameras. For some reason, the blue flowers seem particularly hard to handle, with many cameras rendering them as purple. (In real life, they’re a light shade of navy blue, with just a bit of purple in them.)
• Skin tones: The overall slight warm cast will tend to leave Marti’s skin tones a bit on the warm side as well. Nonetheless, look to see if her skin seems overly pink or if they have a too-bright tinge of yellow: Some cameras oversaturate skin tones (make their color too intense), leading to an almost sunburned look. A little oversaturation can make for a more “healthy-looking” complexion, but it doesn’t take much variation for skin tones to look unnatural.
• Highlight detail: When Marti’s skin tones are at a more or less normal level of brightness, how much detail can you see in her shirt? Does it blow out entirely to white, or can you still see the creases and folds in the fabric?
• Overall contrast: Most consumer digital cameras produce bright, contrasty images, because that’s what most consumers like. Unfortunately, under bright sunlit conditions, many such cameras produce images with little or no highlight detail, and dark, plugged-up looking shadows.
• Shadow detail: The area under the flower bouquet is in quite deep shadow. Does the camera in question retain good detail here, with low image noise? To see, you may need to download the image and play with it in Photoshop(tm) or another imaging program. Brighten the image, and see how far detail extends into the shadows. Photo printers are generally much better at showing shadow detail than are CRTs, so you’ll want a camera that preserves good detail here. The ability to boost brightness without encountering too much image noise is important if you ever have to “rescue” an underexposed image on the computer.
• Detail in areas of subtle contrast: Most digital cameras employ some sort of noise-suppression to remove electronic noise from their images. Noise suppression is a good thing, but only if it’s not overdone. Too much noise suppression will “flatten out” subtle detail in areas of reduced contrast. You can often see this in hair, where the individual strands become blurred, and the image takes on an almost watercolor effect. Look at the detail in Marti’s hair, and compare how it looks with different cameras in the Comparometer.

To view the entire exposure series from zero to +1.3 EV, see files CPS4OUTAP0.HTM through /cameras/nikon-s4-review/image/718358?section=samples on the thumbnail index page.

The original intent of this shot was to illustrate why you really want a zoom lens if you intend to take close-up shots of people. Fixed focal length lenses tend toward the wide angle end of the spectrum, meaning that you have to get very close to your subjects to fill the frame with them. This results in very unnatural, chipmunk-like face shot, with exaggerated noses and rounded cheeks.

These days, most digital cameras come equipped with zoom lenses, so you won’t have to worry about the “Chipmunk look” in your portrait shots. (Just zoom the lens out to its telephoto position when you’re taking close-in shots like this.)

To view the entire exposure series from zero to +1.0 EV, see files CPS4OUTFACAP0.HTM through CPS4OUTFACAP3.HTM on the thumbnail index page.

Normal Flash +1.0 EV

Slow-Sync Flash +0.7 EV

This shot duplicates indoor shooting conditions in most US homes, with fairly bright incandescent room lighting. The challenge here is for the camera’s flash to blend naturally with the room lighting, and produce good, neutral color overall. – Some cameras will be overly affected by the room lighting, even with their flash enabled, and the result will be a strong orange cast. Another common failing is for the highlights from the flash to take on an unnatural bluish cast.

Finally, exposure is important here, and frequently a tough challenge for the cameras. Marti’s white shirt is central in the scene, reflecting a lot of the light from the flash right back at the camera. As a result, most cameras underexpose this shot, and require some positive exposure compensation to produce a good result. – And that’s an important consideration in itself: Does the camera even permit adjustment of its flash exposures? Many do not. These photos are a tough exposure challenge, if they come out OK, the camera in question can probably be coaxed into delivering a good flash exposure of any subject within its range.

Note too, that the normal flash shot (as opposed to the slow sync one, if the camera offers that feature) will be sharply rendered, any subject or camera movement frozen by the quick pop of the flash. That makes this shot a good one to look for the effect of over-aggressive noise suppression in Marti’s hair.

To view the entire exposure series from zero to +1.7 EV, see files CPSZ4INFP0.HTM through CPS4INFP5.HTM on the thumbnail index page.

To view the exposure series from zero to +1.3 EV in the Slow-Sync flash mode, see files CPS4INFSP0.HTM through CPS4INFSP4.HTM on the thumbnail index page.

Auto White Balance

Incandescent White Balance

Manual White Balance

The incandescent lighting used in most US homes actually has a very strong yellow color to it. Our eyes have an amazing ability to ignore color casts like this, something digital cameras struggle to emulate.

The incandescent lighting used for this shot is thus not only very common here in the US, but also very difficult for most digital cameras to deal with. While we probably want a little yellow color to remain in the image (to convey some of the mood of the original scene), too much will look unnatural and distort colors.

Most cameras’ auto white balance systems have a great deal of difficulty with this shot, but many incandescent white balance settings struggle as well. (It seems that many cameras’ incandescent settings are actually calibrated to the tungsten lighting used in professional studio systems, which isn’t nearly as warm-toned as typical household lighting.)

If you intend to do much shooting indoors after dark, pay careful attention to this test, as cameras vary widely in this regard.

To view the entire exposure series from zero to +1.7 EV, see files CPS4INMP0.HTM through CPS4INMP5.HTM on the thumbnail index page.

ISO Series:

“ISO equivalent” refers to a camera’s light sensitivity. ISO 200 represents twice the sensitivity of ISO 100, meaning that you can use a shutter speed that’s twice as fast. Higher ISO settings are often required to get any picture at all when shooting after dark, but even in full daylight, using a higher ISO can help you freeze fast action. The problem is, increasing a digital camera’s ISO also increases image noise. In practical terms, this means that higher-ISO images often can’t be used to produce prints as large as lower-ISO ones. The tricky thing here is that high-ISO images often look much different when printed at various sizes than they do when viewed on-screen. In particular, for any level of image noise, you’ll often find that while noise is quite evident at larger print sizes, as you reduce the size of the prints, there will come a point where it suddenly ceases to be an issue. We routinely print high-ISO photos from the cameras we test on our studio printer (currently a Canon i9900) at a range of sizes, and report our findings. If you’re interested in investigating the effect of image noise for yourself, don’t judge cameras’ performance by how their images look on your CRT, viewed pixel-for-pixel. Rather, download the test shots linked in the table below and output them on your own printer, so you can see how prints of various sizes will actually look.

One additional note about this particular test series though: Because they’re shot under household incandescent lighting, the camera has to boost its blue-channel signal quite a bit to get back to a neutral color balance. Since the blue channel is generally the one with the most noise, this makes this shot a real acid test of noise performance. Noise levels in high-ISO shots taken under daylight conditions usually won’t show as much noise. (See the “Far Field” test for examples of high ISO shots captured in daylight.)

Auto White Balance

Daylight White Balance

Manual White Balance

Like several of our tests, these images are actually photos of a high-resolution poster, shot under studio lighting. As of this writing, the poster in question was produced from a high-resolution scan of a 4×5-inch transparency, scanned to a resolution of about 150 megapixels. Even at that, it’s starting to show its age, as the combination of camera lens, film emulsion and scanner resolution mean that many of the current 5+ megapixel cameras come close to capturing all the detail that’s present. (We have a new poster under production, produced from 45 separate digital images, “stitched” into a single 400+ megapixel image, that should comfortably outpace the resolution of portable digital cameras for years to come. – Expect to start seeing shots of this new poster sometime in Fall, 2005.)

Why did we choose to shoot a picture of a picture? The idea was to show a typical subject (a house and surrounding foliage) in a way that would be absolutely consistent from camera to camera. Any outdoor subject is going to vary considerably from day to day, as the lighting changes with the weather, atmospheric conditions, and season. Shooting a poster lets us compare images from cameras shot weeks, months, or even years apart, with the sure knowledge that nothing has changed from one shot to another.

Things to look for here are fine detail, as seen in the foliage and tree limbs against the sky, sharpness in the corners, and the preservation of subtle detail in the shaded brick patterns. – Many cameras with overactive noise suppression severely blur the brick patterns that are in shadow.

While the House poster in the shot above provides absolute repeatability from test to test, it doesn’t offer the range of brightness (dynamic range) that the original scene had, nor does it contain the nearly infinite range of fine detail found in nature. For these reasons, we still shoot the original house, even though the vagaries of nature mean that no two shots will ever be directly comparable. (In fact, over the eight or so years since we first shot this subject, the trees in front of the house have now grown so large that they obscure much of the subject. – We’re unfortunately going to have to switch to a different subject in the near future.)

Things to look for here are how well the camera handles the range of light levels from very bright to quite dark, and how well it renders the very fine detail visible in various parts of the image.

Note though, that because this is shot outdoors, the character of the light is unavoidably going to change quite a bit, depending on the atmospheric humidity and the time of year. – You thus shouldn’t rely on it for absolute comparisons between cameras, since it’s unlikely that conditions will be identical from one test to the next.

For this subject, we shoot examples of pretty much anything the camera is capable of, with a resolution series to show how well the camera “downsamples” its images to make lower-resolution copies, an ISO series, color effects, contrast and saturation adjustments if they’re available, etc.

Resolution Series:

ISO Series:

Color Effects Series:

Color Effects Series
Vivid Color	Cyanotype	Sepia	Black & White

Lens Zoom Range

Simply reading “4x zoom range” doesn’t do a lot to help you visualize what that means. It also says nothing regarding just how wide the wide-angle end of that range is. To give you an idea of exactly what each camera’s zoom lens does, we shoot this series of images, showing results at maximum wide angle, maximum telephoto, and telephoto with “digital zoom” enabled. (Note of course though, that so-called “digital zoom” just crops out and enlarges the central pixels of the image, achieving increased size at the cost of reduced resolution.)

	Wide Angle
	10x Telephoto
	Digital Telephoto

Auto White Balance

Daylight White Balance

Manual White Balance

This is another poster, shot in the studio under controlled lighting. It’s intended to give an idea of how well various cameras render a range of skin tones, but it often winds up as more of a referendum on the camera’s white balance system. The background of the image has a lot of blue in it, which frequently tricks white balance systems into overcompensating toward warmer hues. The poster is also rather low resolution, having been made from a 20 megabyte digital file. As such, it can’t be relied upon to reveal much about detail rendition of modern, high-megapixel cameras.

Standard Macro Shot

Macro with Flash

Many are interested in close-up or “macro” photography. This test shows the best results we could obtain using each camera’s macro mode. What to look for:

• Minimum Macro Area: What’s the smallest area that the camera can photograph? – We calculate this and report on it in the Test Results section of each review.
• Softness in corners?: The images from most digital cameras get fairly soft when shooting in macro mode. Some are better than others though, you can use the texture of the paper fibers or the details in the dollar bill to compare corner sharpness between models.
• Flash performance in macro mode?: Macro mode flash performance varies widely between cameras. Some can’t throttle down the flash enough, others throttle down enough, but have very uneven lighting, while others have lenses that project into the path of the flash, casting strong shadows. The Macro with Flash shot here will show you what to expect.

Auto White Balance

Daylight White Balance

Manual White Balance

This subject is a good all-around laboratory test target, useful for the more analytically-minded in making objective comparisons between cameras. Things to look for:

• White balance: How well do the various white balance modes render the neutrally-colored subject?
• Color Accuracy: The “MacBeth” color chart is a widely available standard color target. The color values can be used to make very precise measurements of color accuracy. (We use this target internally at IR in conjunction with the Imatest image-analysis program to get a sense of how cameras handle color in various parts of the spectrum.)
• Shadow detail and noise: The charcoal briquettes in the black box at the bottom of the box are an excellent test of shadow detail.
• Highlight detail: The white-on-white of the gauze at lower right is a good test for highlight detail.
• Specular reflections: The shiny pot lid at top right reflects the studio lights right back into the lens. Cameras these days handle light overloads like this pretty well, but this test subject still reveals problems with “charge bleeding” in some cameras.

The results in the tests below mirror those seen above in other test shots. The test series are repeated here, for the benefit of our more quantitatively-oriented readers.

ISO Series:

Color Effects Series:

Color Effects Series
Vivid Color	Cyanotype	Sepia	Black & White

Low-Light Tests

Low light photography is an area where there are really enormous differences between digital camera models. This test starts at a light level about equivalent to typical city street lighting at night (one foot-candle), and then progresses down from there, each successive test being at half the light level of the preceding one. You may also see the effect of poor low-light autofocus in some of these shots, although we use a different test setup to check autofocus performance more directly. (The results of which are reported on in the main Test Results section.) Things to look for here include:

• Exposure limit: What’s the darkest level a camera can handle at each ISO setting? If the leftmost images are reasonably bright, the camera should do fine with typical city night scenes. If you’re shooting in the more dimly-lit suburbs, you’ll need a camera capable of producing good images one, two, or three steps to the right of that.
• Autofocus Limit: How dark can you shoot and still get well-focused pictures?
• White Balance: Does the camera’s white balance suffer at low light levels? (Many do.)
• Noise Levels: Look at the photos, print them on your own photo printer. How large a print can you make at acceptable quality levels, at various ISO settings and light levels?
• Detail loss to anti-noise processing?: Do details in the white gauze and even in the lettering on the test targets suffer at lower light levels?

(Note: If you’d like to use a light meter to check light levels for subjects you might be interested in shooting, a light level of one foot-candle corresponds to a normal exposure of two seconds at f/2.8 and ISO 100.)

Â	1 fc 11 lux	1/2 fc 5.5 lux	1/4 fc 2.7 lux	1/8 fc 1.3 lux	1/16 fc 0.67 lux
ISO 50	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5
ISO 100	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5
ISO 200	1.63 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5
ISO 400	1/1 sec f3.5	1.69 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5

Flash Range Test

Digital camera makers have gotten better with their flash-range ratings. In the early days, many cameras had rather “optimistic” flash range specs, to put it politely. These days, the manufacturers seem to be toeing the line. (No doubt at least in part because of tests like this.) Consistent with our philosophy of testing worst-case conditions, this test also involves some use of each camera’s zoom lens. Flash range is greater at wide angle focal lengths than at telephoto ones. If you’re shooting at the wide angle end of the lens’ range, you might get better flash range than what’s shown here. – But you’ll never get a nasty surprise if you let the test shots below be your guide to flash capability.

8 ft	9 ft	10 ft	11 ft	12 ft	13 ft	14 ft
1/60 sec f3.5 ISO 50	1/60 sec f3.5 ISO 50	1/60 sec f3.5 ISO 50	1/60 sec f3.5 ISO 50	1/204 sec f3.5 ISO 50	1/204 sec f3.5 ISO 50	1/204 sec f3.5 ISO 50

ISO-12233 (WG-18) Resolution Test

This is the standard ISO-12233 resolution target. It can be used with automated image-analysis tools like Imatest, but for our reviews, we generally just report the number of lines of resolution that we can see cleanly resolved in it.

The chart numbers correspond to hundreds of lines of resolution per picture height. – The reference to picture height is to provide a consistent measure of resolution, regardless of image aspect ratio. (Height to width ratio.)

In evaluating lines of resolution, it’s important to set as a threshold the point at which artifacts (jaggies or blurring of the lines) start to overcome the target details themselves. We tend to be a bit more conservative than some reviewers in the number of lines we’ll say a camera has: The point here isn’t to try to figure out the point at which you can just barely convince yourself that you’re still seeing the target lines, but rather to call a halt when the camera is no longer cleanly resolving the target detail. We also report the point at which the target lines entirely merge into oblivion, as the “extinction point.” This last point doesn’t relate to image resolution in any sense, but does correlate somewhat with perceived edge sharpness.

Things to look for on this target include: 

• Resolution: How many lines are resolved, both horizontally (across the vertically-oriented resolution wedge) and vertically (across the horizontally-oriented resolution wedge.)
• Where do artifacts start?: This is a more subtle measure. Somewhere well before the lines start to merge with each other, you’ll see subtle patterns of thickening and narrowing, artifacts caused by the camera’s sensor and image processing system. In an ideal world, these artifacts would be invisible. With most cameras, they are nearly so, but some models produce rather pronounced “jaggies.”
• Extinction point: As described above, the point at which the target lines fade away more or less completely. Generally correlates with perceived sharpness.
• Barrel/Pincushion distortion: Barrel distortion refers to a tendency for straight lines near the edges of the frame to bow outward in the middle. Pincushion distortion is just the opposite, lines near the edge bow inward in the middle. The amount of distortion is expressed as a percentage, dividing the amount of the bow by the length of the line over which you’re measuring it. Most lenses show significant amounts of barrel distortion at wide angle, on average about 0.8%. That doesn’t sound like a lot, but it’s very visible if you have straight lines near the edges of the frame. Pincushion distortion is usually seen at telephoto focal lengths, and varies widely between cameras, from none at all to as much as 0.6%.
• Chromatic Aberration: Chromatic aberration appears as a slight colored fringe around the edges of objects, usually just in the corners of the frame. If the coloration is intense, it can be very distracting in your images, particularly where dark objects are silhouetted against the sky. Obviously, the less of this the better.
• Corner Sharpness: Most cameras lose sharpness in the corners of the frame, at least somewhat, and particularly at larger apertures. Look at the resolution target elements in the corners to see how much the image softens there, and how far into the frame the softness persists.

Resolution Series, medium focal length

 

Resolution Test, Zoom Series

“Sunlit” Portrait:
(This is my new “Outdoor” Portrait test – read more about it here.)

The lighting in this shot is deliberately awful, about what you’d expect from noontime sunshine here in the Atlanta, GA area. (In fact, the color balance has been chosen to pretty well match the hazy sunshine here in mid-August.)

The reason for the harsh lighting is to provide a real “torture test” of how cameras handle conditions of extreme contrast; and in particular, how well they do holding onto highlight detail.

Look for:

• Overall color: Matching summer sunlight here in the South, the lighting in this scene is a bit more yellow-tinged than that in many parts of the country, or in the fall or winter. – So there may be an overall warm cast to the color. That said though, there’s a fair range of color represented in the bouquet, presenting a tough challenge for the cameras. For some reason, the blue flowers seem particularly hard to handle, with many cameras rendering them as purple. (In real life, they’re a light shade of navy blue, with just a bit of purple in them.)
• Skin tones: The overall slight warm cast will tend to leave Marti’s skin tones a bit on the warm side as well. Nonetheless, look to see if her skin seems overly pink or if they have a too-bright tinge of yellow: Some cameras oversaturate skin tones (make their color too intense), leading to an almost sunburned look. A little oversaturation can make for a more “healthy-looking” complexion, but it doesn’t take much variation for skin tones to look unnatural.
• Highlight detail: When Marti’s skin tones are at a more or less normal level of brightness, how much detail can you see in her shirt? Does it blow out entirely to white, or can you still see the creases and folds in the fabric?
• Overall contrast: Most consumer digital cameras produce bright, contrasty images, because that’s what most consumers like. Unfortunately, under bright sunlit conditions, many such cameras produce images with little or no highlight detail, and dark, plugged-up looking shadows.
• Shadow detail: The area under the flower bouquet is in quite deep shadow. Does the camera in question retain good detail here, with low image noise? To see, you may need to download the image and play with it in Photoshop(tm) or another imaging program. Brighten the image, and see how far detail extends into the shadows. Photo printers are generally much better at showing shadow detail than are CRTs, so you’ll want a camera that preserves good detail here. The ability to boost brightness without encountering too much image noise is important if you ever have to “rescue” an underexposed image on the computer.
• Detail in areas of subtle contrast: Most digital cameras employ some sort of noise-suppression to remove electronic noise from their images. Noise suppression is a good thing, but only if it’s not overdone. Too much noise suppression will “flatten out” subtle detail in areas of reduced contrast. You can often see this in hair, where the individual strands become blurred, and the image takes on an almost watercolor effect. Look at the detail in Marti’s hair, and compare how it looks with different cameras in the Comparometer.

To view the entire exposure series from zero to +1.3 EV, see files CPS4OUTAP0.HTM through /cameras/nikon-s4-review/image/718314?section=test on the thumbnail index page.

The original intent of this shot was to illustrate why you really want a zoom lens if you intend to take close-up shots of people. Fixed focal length lenses tend toward the wide angle end of the spectrum, meaning that you have to get very close to your subjects to fill the frame with them. This results in very unnatural, chipmunk-like face shot, with exaggerated noses and rounded cheeks.

These days, most digital cameras come equipped with zoom lenses, so you won’t have to worry about the “Chipmunk look” in your portrait shots. (Just zoom the lens out to its telephoto position when you’re taking close-in shots like this.)

To view the entire exposure series from zero to +1.0 EV, see files CPS4OUTFACAP0.HTM through CPS4OUTFACAP3.HTM on the thumbnail index page.

Normal Flash +1.0 EV

Slow-Sync Flash +0.7 EV

This shot duplicates indoor shooting conditions in most US homes, with fairly bright incandescent room lighting. The challenge here is for the camera’s flash to blend naturally with the room lighting, and produce good, neutral color overall. – Some cameras will be overly affected by the room lighting, even with their flash enabled, and the result will be a strong orange cast. Another common failing is for the highlights from the flash to take on an unnatural bluish cast.

Finally, exposure is important here, and frequently a tough challenge for the cameras. Marti’s white shirt is central in the scene, reflecting a lot of the light from the flash right back at the camera. As a result, most cameras underexpose this shot, and require some positive exposure compensation to produce a good result. – And that’s an important consideration in itself: Does the camera even permit adjustment of its flash exposures? Many do not. These photos are a tough exposure challenge, if they come out OK, the camera in question can probably be coaxed into delivering a good flash exposure of any subject within its range.

Note too, that the normal flash shot (as opposed to the slow sync one, if the camera offers that feature) will be sharply rendered, any subject or camera movement frozen by the quick pop of the flash. That makes this shot a good one to look for the effect of over-aggressive noise suppression in Marti’s hair.

To view the entire exposure series from zero to +1.7 EV, see files CPSZ4INFP0.HTM through CPS4INFP5.HTM on the thumbnail index page.

To view the exposure series from zero to +1.3 EV in the Slow-Sync flash mode, see files CPS4INFSP0.HTM through CPS4INFSP4.HTM on the thumbnail index page.

Auto White Balance

Incandescent White Balance

Manual White Balance

The incandescent lighting used in most US homes actually has a very strong yellow color to it. Our eyes have an amazing ability to ignore color casts like this, something digital cameras struggle to emulate.

The incandescent lighting used for this shot is thus not only very common here in the US, but also very difficult for most digital cameras to deal with. While we probably want a little yellow color to remain in the image (to convey some of the mood of the original scene), too much will look unnatural and distort colors.

Most cameras’ auto white balance systems have a great deal of difficulty with this shot, but many incandescent white balance settings struggle as well. (It seems that many cameras’ incandescent settings are actually calibrated to the tungsten lighting used in professional studio systems, which isn’t nearly as warm-toned as typical household lighting.)

If you intend to do much shooting indoors after dark, pay careful attention to this test, as cameras vary widely in this regard.

To view the entire exposure series from zero to +1.7 EV, see files CPS4INMP0.HTM through CPS4INMP5.HTM on the thumbnail index page.

ISO Series:

“ISO equivalent” refers to a camera’s light sensitivity. ISO 200 represents twice the sensitivity of ISO 100, meaning that you can use a shutter speed that’s twice as fast. Higher ISO settings are often required to get any picture at all when shooting after dark, but even in full daylight, using a higher ISO can help you freeze fast action. The problem is, increasing a digital camera’s ISO also increases image noise. In practical terms, this means that higher-ISO images often can’t be used to produce prints as large as lower-ISO ones. The tricky thing here is that high-ISO images often look much different when printed at various sizes than they do when viewed on-screen. In particular, for any level of image noise, you’ll often find that while noise is quite evident at larger print sizes, as you reduce the size of the prints, there will come a point where it suddenly ceases to be an issue. We routinely print high-ISO photos from the cameras we test on our studio printer (currently a Canon i9900) at a range of sizes, and report our findings. If you’re interested in investigating the effect of image noise for yourself, don’t judge cameras’ performance by how their images look on your CRT, viewed pixel-for-pixel. Rather, download the test shots linked in the table below and output them on your own printer, so you can see how prints of various sizes will actually look.

One additional note about this particular test series though: Because they’re shot under household incandescent lighting, the camera has to boost its blue-channel signal quite a bit to get back to a neutral color balance. Since the blue channel is generally the one with the most noise, this makes this shot a real acid test of noise performance. Noise levels in high-ISO shots taken under daylight conditions usually won’t show as much noise. (See the “Far Field” test for examples of high ISO shots captured in daylight.)

Auto White Balance

Daylight White Balance

Manual White Balance

Like several of our tests, these images are actually photos of a high-resolution poster, shot under studio lighting. As of this writing, the poster in question was produced from a high-resolution scan of a 4×5-inch transparency, scanned to a resolution of about 150 megapixels. Even at that, it’s starting to show its age, as the combination of camera lens, film emulsion and scanner resolution mean that many of the current 5+ megapixel cameras come close to capturing all the detail that’s present. (We have a new poster under production, produced from 45 separate digital images, “stitched” into a single 400+ megapixel image, that should comfortably outpace the resolution of portable digital cameras for years to come. – Expect to start seeing shots of this new poster sometime in Fall, 2005.)

Why did we choose to shoot a picture of a picture? The idea was to show a typical subject (a house and surrounding foliage) in a way that would be absolutely consistent from camera to camera. Any outdoor subject is going to vary considerably from day to day, as the lighting changes with the weather, atmospheric conditions, and season. Shooting a poster lets us compare images from cameras shot weeks, months, or even years apart, with the sure knowledge that nothing has changed from one shot to another.

Things to look for here are fine detail, as seen in the foliage and tree limbs against the sky, sharpness in the corners, and the preservation of subtle detail in the shaded brick patterns. – Many cameras with overactive noise suppression severely blur the brick patterns that are in shadow.

While the House poster in the shot above provides absolute repeatability from test to test, it doesn’t offer the range of brightness (dynamic range) that the original scene had, nor does it contain the nearly infinite range of fine detail found in nature. For these reasons, we still shoot the original house, even though the vagaries of nature mean that no two shots will ever be directly comparable. (In fact, over the eight or so years since we first shot this subject, the trees in front of the house have now grown so large that they obscure much of the subject. – We’re unfortunately going to have to switch to a different subject in the near future.)

Things to look for here are how well the camera handles the range of light levels from very bright to quite dark, and how well it renders the very fine detail visible in various parts of the image.

Note though, that because this is shot outdoors, the character of the light is unavoidably going to change quite a bit, depending on the atmospheric humidity and the time of year. – You thus shouldn’t rely on it for absolute comparisons between cameras, since it’s unlikely that conditions will be identical from one test to the next.

For this subject, we shoot examples of pretty much anything the camera is capable of, with a resolution series to show how well the camera “downsamples” its images to make lower-resolution copies, an ISO series, color effects, contrast and saturation adjustments if they’re available, etc.

Resolution Series:

ISO Series:

Color Effects Series:

Color Effects Series
Vivid Color	Cyanotype	Sepia	Black & White

Lens Zoom Range

Simply reading “4x zoom range” doesn’t do a lot to help you visualize what that means. It also says nothing regarding just how wide the wide-angle end of that range is. To give you an idea of exactly what each camera’s zoom lens does, we shoot this series of images, showing results at maximum wide angle, maximum telephoto, and telephoto with “digital zoom” enabled. (Note of course though, that so-called “digital zoom” just crops out and enlarges the central pixels of the image, achieving increased size at the cost of reduced resolution.)

	Wide Angle
	10x Telephoto
	Digital Telephoto

Auto White Balance

Daylight White Balance

Manual White Balance

This is another poster, shot in the studio under controlled lighting. It’s intended to give an idea of how well various cameras render a range of skin tones, but it often winds up as more of a referendum on the camera’s white balance system. The background of the image has a lot of blue in it, which frequently tricks white balance systems into overcompensating toward warmer hues. The poster is also rather low resolution, having been made from a 20 megabyte digital file. As such, it can’t be relied upon to reveal much about detail rendition of modern, high-megapixel cameras.

Standard Macro Shot

Macro with Flash

Many are interested in close-up or “macro” photography. This test shows the best results we could obtain using each camera’s macro mode. What to look for:

• Minimum Macro Area: What’s the smallest area that the camera can photograph? – We calculate this and report on it in the Test Results section of each review.
• Softness in corners?: The images from most digital cameras get fairly soft when shooting in macro mode. Some are better than others though, you can use the texture of the paper fibers or the details in the dollar bill to compare corner sharpness between models.
• Flash performance in macro mode?: Macro mode flash performance varies widely between cameras. Some can’t throttle down the flash enough, others throttle down enough, but have very uneven lighting, while others have lenses that project into the path of the flash, casting strong shadows. The Macro with Flash shot here will show you what to expect.

Auto White Balance

Daylight White Balance

Manual White Balance

This subject is a good all-around laboratory test target, useful for the more analytically-minded in making objective comparisons between cameras. Things to look for:

• White balance: How well do the various white balance modes render the neutrally-colored subject?
• Color Accuracy: The “MacBeth” color chart is a widely available standard color target. The color values can be used to make very precise measurements of color accuracy. (We use this target internally at IR in conjunction with the Imatest image-analysis program to get a sense of how cameras handle color in various parts of the spectrum.)
• Shadow detail and noise: The charcoal briquettes in the black box at the bottom of the box are an excellent test of shadow detail.
• Highlight detail: The white-on-white of the gauze at lower right is a good test for highlight detail.
• Specular reflections: The shiny pot lid at top right reflects the studio lights right back into the lens. Cameras these days handle light overloads like this pretty well, but this test subject still reveals problems with “charge bleeding” in some cameras.

The results in the tests below mirror those seen above in other test shots. The test series are repeated here, for the benefit of our more quantitatively-oriented readers.

ISO Series:

Color Effects Series:

Color Effects Series
Vivid Color	Cyanotype	Sepia	Black & White

Low-Light Tests

Low light photography is an area where there are really enormous differences between digital camera models. This test starts at a light level about equivalent to typical city street lighting at night (one foot-candle), and then progresses down from there, each successive test being at half the light level of the preceding one. You may also see the effect of poor low-light autofocus in some of these shots, although we use a different test setup to check autofocus performance more directly. (The results of which are reported on in the main Test Results section.) Things to look for here include:

• Exposure limit: What’s the darkest level a camera can handle at each ISO setting? If the leftmost images are reasonably bright, the camera should do fine with typical city night scenes. If you’re shooting in the more dimly-lit suburbs, you’ll need a camera capable of producing good images one, two, or three steps to the right of that.
• Autofocus Limit: How dark can you shoot and still get well-focused pictures?
• White Balance: Does the camera’s white balance suffer at low light levels? (Many do.)
• Noise Levels: Look at the photos, print them on your own photo printer. How large a print can you make at acceptable quality levels, at various ISO settings and light levels?
• Detail loss to anti-noise processing?: Do details in the white gauze and even in the lettering on the test targets suffer at lower light levels?

(Note: If you’d like to use a light meter to check light levels for subjects you might be interested in shooting, a light level of one foot-candle corresponds to a normal exposure of two seconds at f/2.8 and ISO 100.)

Â	1 fc 11 lux	1/2 fc 5.5 lux	1/4 fc 2.7 lux	1/8 fc 1.3 lux	1/16 fc 0.67 lux
ISO 50	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5
ISO 100	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5
ISO 200	1.63 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5
ISO 400	1/1 sec f3.5	1.69 sec f3.5	2 sec f3.5	2 sec f3.5	2 sec f3.5

Flash Range Test

Digital camera makers have gotten better with their flash-range ratings. In the early days, many cameras had rather “optimistic” flash range specs, to put it politely. These days, the manufacturers seem to be toeing the line. (No doubt at least in part because of tests like this.) Consistent with our philosophy of testing worst-case conditions, this test also involves some use of each camera’s zoom lens. Flash range is greater at wide angle focal lengths than at telephoto ones. If you’re shooting at the wide angle end of the lens’ range, you might get better flash range than what’s shown here. – But you’ll never get a nasty surprise if you let the test shots below be your guide to flash capability.

8 ft	9 ft	10 ft	11 ft	12 ft	13 ft	14 ft
1/60 sec f3.5 ISO 50	1/60 sec f3.5 ISO 50	1/60 sec f3.5 ISO 50	1/60 sec f3.5 ISO 50	1/204 sec f3.5 ISO 50	1/204 sec f3.5 ISO 50	1/204 sec f3.5 ISO 50

ISO-12233 (WG-18) Resolution Test

This is the standard ISO-12233 resolution target. It can be used with automated image-analysis tools like Imatest, but for our reviews, we generally just report the number of lines of resolution that we can see cleanly resolved in it.

The chart numbers correspond to hundreds of lines of resolution per picture height. – The reference to picture height is to provide a consistent measure of resolution, regardless of image aspect ratio. (Height to width ratio.)

In evaluating lines of resolution, it’s important to set as a threshold the point at which artifacts (jaggies or blurring of the lines) start to overcome the target details themselves. We tend to be a bit more conservative than some reviewers in the number of lines we’ll say a camera has: The point here isn’t to try to figure out the point at which you can just barely convince yourself that you’re still seeing the target lines, but rather to call a halt when the camera is no longer cleanly resolving the target detail. We also report the point at which the target lines entirely merge into oblivion, as the “extinction point.” This last point doesn’t relate to image resolution in any sense, but does correlate somewhat with perceived edge sharpness.

Things to look for on this target include: 

• Resolution: How many lines are resolved, both horizontally (across the vertically-oriented resolution wedge) and vertically (across the horizontally-oriented resolution wedge.)
• Where do artifacts start?: This is a more subtle measure. Somewhere well before the lines start to merge with each other, you’ll see subtle patterns of thickening and narrowing, artifacts caused by the camera’s sensor and image processing system. In an ideal world, these artifacts would be invisible. With most cameras, they are nearly so, but some models produce rather pronounced “jaggies.”
• Extinction point: As described above, the point at which the target lines fade away more or less completely. Generally correlates with perceived sharpness.
• Barrel/Pincushion distortion: Barrel distortion refers to a tendency for straight lines near the edges of the frame to bow outward in the middle. Pincushion distortion is just the opposite, lines near the edge bow inward in the middle. The amount of distortion is expressed as a percentage, dividing the amount of the bow by the length of the line over which you’re measuring it. Most lenses show significant amounts of barrel distortion at wide angle, on average about 0.8%. That doesn’t sound like a lot, but it’s very visible if you have straight lines near the edges of the frame. Pincushion distortion is usually seen at telephoto focal lengths, and varies widely between cameras, from none at all to as much as 0.6%.
• Chromatic Aberration: Chromatic aberration appears as a slight colored fringe around the edges of objects, usually just in the corners of the frame. If the coloration is intense, it can be very distracting in your images, particularly where dark objects are silhouetted against the sky. Obviously, the less of this the better.
• Corner Sharpness: Most cameras lose sharpness in the corners of the frame, at least somewhat, and particularly at larger apertures. Look at the resolution target elements in the corners to see how much the image softens there, and how far into the frame the softness persists.

Resolution Series, medium focal length

 

Resolution Test, Zoom Series