New/Modified Test Targets

As part of our initiative to provide fairly complete camera test data quickly, we've added a new test target and modified an existing one. The goal was to come up with two test shots that we could shoot and post for every camera, more or less as soon as it hits the lab. The combination of these two shots provides a tremendous amount of information about how a camera performs, covering aspects of color, resolution, tonality, and noise suppression processing, as well as noise/detail performance across the full ISO range. Here's a look at the two "new" targets (one all-new, the other modified), and how you can use them to pick apart a camera's imaging performance:

New Still Life target

The photo above shows our new Still Life target. We carefully assembled and arranged its contents to reveal an extraordinary amount of information about a camera's performance.

Here's what to look for in this target:

• Tone-on-tone detail & noise suppression: The cloth swatches in the pinwheel were chosen because they show a lot of tone-on-tone detail, across a broad range of colors. This is just the kind of detail that noise suppression processing tends to flatten out. If you look at the detail in these swatches as the ISO increases, you'll see just where different cameras start to lose subtle detail. -- The white and tan swatches and the dark swatches tend to be particularly revealing of this. The label of the vinegar bottle (second from the right) is another great place to look for lost detail from noise suppression, as the image of the person at the top of the label is actually a depiction of a mosaic. The dark colors in the background and in the figure's clothes contain detail that's very quickly lost when a camera's noise suppression system kicks in. Cameras with really high-quality, low-noise sensors that require little noise suppression will be able to hold onto the detail in these areas, but many others will show only a uniform swath of smudged color.
  
  Another place where you'll quickly see the effects of over-aggressive noise suppression is in the white salt grains of the salt grinder in the lower left. Cameras are often more conservative about suppressing noise in highlight areas (because our eyes tend to see less of it there), but many cameras seem to have a hard time holding onto the subtle shadings that distinguish the salt grains from each other, particularly at higher ISO levels.
  
  
• Fine Detail: You'll find a lot of fine detail in the label of the beer bottle on the right, in its fine cursive text, but the other bottle labels hold a lot of fine detail as well. Fine text is often a good visual indicator of resolution, because our brains have an excellent idea of what the text should look like, so are very quick to notice even minor loss of detail.
  
  For really fine detail, look to the circular scale/calculator on the right side of the scene. Some of the lines there are extremely fine indeed. Looking at results from many different cameras with this target, we found that camera noise-suppression systems often confuse the fine lines with image noise, and so flatten them out. There's also a nice range of fine text sizes in this chart as well, once again great visual cues for resolution and detail.
  
  
• Highlight Detail: Three elements in this scene show off (or show up) a camera's ability to hold onto highlight detail. As mentioned above, the salt grains (and reflections of the studio lights) in the salt mill are examples of fairly subtle highlight detail that cameras' anti-noise processing sometimes obliterate. The folded white cloth under the mug on the right side of the frame likewise shows a lot of white-on-white detail that is easy to lose, particularly if a camera's tone curve is too contrasty. As it turns out though, the most sensitive test of a camera's highlight abilities seems to be the hank of white embroidery thread in the upper right corner. These fibers are unusually bright and reflective, so its easy for a camera to blow out detail in them.
  
  
• Shadow Detail: Several elements of this subject are useful for evaluating shadow detail, particularly the black mug and the pieces of folded black velvet, both under and inside the mug. The bottoms of the beer bottles also provide some gradations of deep shadow, and the clump of peppers in the bottom of the pepper oil bottle had a fair bit of detail that's far down at the shadow end of the tone curve.
  
  We were actually surprised when we constructed this scene just how dark the velvet and sides of the beer bottles come out on a camera. Even with the bright studio lights shining directly on it, the velvet in particular stays way, way down at the shadow end of the tone curve. With most cameras and on most monitors, the velvet will simply appear as an unrelieved swatch of black. To see whether it contains deep detail or not, in most cases you'll have to open the file in an image editor and boost the brightness dramatically, to bring the detail up into a visible range.
  
  
• Preservation of "Shape" in Strong Colors: As you approach the extremes of a camera's color gamut (its range of recordable colors), it becomes more and more difficult for the camera to show fine gradations of tone, because one or more of the RGB color channels are close to saturation. It's not uncommon to see a brightly colored piece of clothing or a vibrant flower appear in digicam photos as just a blob of color, because the camera ran up against the limits of its color gamut. The brightly colored embroidery threads in the upper right portion of the Still Life target are good examples of situations where this might happen. Pay particular attention to the bright red and dark blue colors here, as these are both colors near the edge of the typical sRGB color gamut.
  
  
• Color accuracy and white balance: It's pretty small in there, but we've included a mini MacBeth chart, which displays very carefully controlled color swatches. Our Multi Target (see below) sports a full-sized MacBeth chart, but the one here serves as a good check of color balance and rendition, and is also useful for checking white balance on this particular shot.
  
  
• Image noise and detail vs ISO: As mentioned above, this target contains many elements useful for evaluating detail loss to anti-noise processing. We'll therefore always shoot a full set of test images of this target across each camera's ISO range, for every camera we test.
  

New "Multi" target

Our new "Multi" target above will look familiar to anyone who's read any of our reviews over the last eight years, as it's just a number of additional elements tacked onto the standard ISO-12233 test target. This is actually something of a transitional target, as we're working on an improved target that will incorporate many of these elements along with a few others. In its current form, this is just an ISO-12233 res chart with several other elements attached to it, covering portions of the target we don't generally use in our reviews. (We'll continue shooting the full res chart in SLR reviews for the benefit of interested readers, but all lesser cameras will now show just the Multi-Target instead.) The added elements largely mirror ones that were present in our "DaveBox" test target, which we've now semi-retired. Here's some of what you'll find in this target:

Res chart elements: In placing the new elements on the ISO target, we were careful to leave the highest-frequency hyperbolic resolution wedges (the sets of fine lines that fan out vertically and horizontally), since these are what most people look at on the ISO chart to judge camera resolution from. There are also enough of the slanted black parallelograms available to use them to measure a camera's Spatial Frequency Response characteristics with Imatest.

Gray Scale: While reflective gray scales don't cover the full dynamic range of higher-end cameras, advanced readers may be interested in using the gray scale here to evaluate noise performance vs brightness level, and/or examine the shape of a camera's tone curve.

MacBeth Chart: This is about as common a color standard as you can get these days, very widely available for only mildly-exorbitant cost, and quite well controlled in its production. It thus serves as a good basis of comparison between cameras and between test setups. Imatest also understands the MacBeth colors very well, and uses them to produce its color accuracy map that we feature in all our reviews.

Kodak Color Separation Target: We include this for reference because it's used by other reviewers out there, but caution our readers that it really isn't well-suited for use as an absolute color standard. As its name suggests, its actual intended purpose is as an aid in setting exposure levels for old-style film-based color separation, using panchromatic graphic arts film, RGB filters, and halftone screens. It's ideal for that use, but the colors aren't well-controlled enough and are too subject to fading for it to be usable as a true reference standard for color accuracy.

Kodak Q60 target: This is another target that's perhaps not well enough controlled for quantitative measurements between cameras, but one that does have several useful characteristics nonetheless. For instance, it provides a good reference for the handling of various colors representative of common skin tones (the tan and brown patches along its right side), as well as of lighter, less-saturated tints of both additive and subtractive primaries (red, green, blue, cyan, magenta, and yellow). The MacBeth chart's colors lean heavily toward highly saturated values, and there are no pastel tones present at all. In the past, we've found that cameras with contrasty tone curves sometimes have trouble with the pastel tones in the Q60 target, making it a valuable reference that we'll continue to include.