Sigma DP1 Merrill Review – Test Shots

Still Life

This is our new “Still Life” test target. We’re combining
some of the elements from previous shots (DaveBox and Res Chart) into
this and the “Multi Target” shot below, plus added a number
of elements that are very revealing of various camera characteristics
and foibles.

Here’s what to look for in this target:

• Tone-on-tone detail & noise suppression:
  The cloth swatches in the pinwheel were chosen because they show a lot
  of tone-on-tone detail, across a broad range of colors. This is just
  the sort of detail that noise suppression processing tends to flatten
  out. If you look at the detail in these swatches as the ISO increases,
  you’ll see just where different cameras start to lose subtle detail.
  — The white and tan swatches and the dark swatches tend to be particularly
  revealing of this. The label of the vinegar bottle (second from the
  right) is another great place to look for lost detail from noise suppression,
  as the image of the person at the top of the label is actually a depiction
  of a mosaic. The dark colors in the background and in the figure’s clothes
  contain detail that’s very quickly lost when a camera’s noise suppression
  system kicks in. Cameras with really high-quality, low-noise sensors
  that require little noise suppression will be able to hold onto the
  detail in these areas, many others will show only a uniform swath of
  smudged color.

  Another place where you’ll quickly see the effects of over-aggressive
  noise suppression is in the white salt grains of the salt grinder in
  lower left. Cameras are often more conservative about suppressing noise
  in highlight areas (because our eyes tend to see less of it there),
  but many cameras seem to have a hard time holding onto the subtle shadings
  that distinguish the salt grains from each other, particularly at higher
  ISO levels.

• Fine Detail: You’ll find a lot of fine detail
  in the label of the beer bottle on the right, in its fine cursive text,
  but the other bottle labels hold a lot of fine detail as well. Fine
  text is often a good visual indicator of resolution, because our brains
  have an excellent idea of what the text should look like, so
  are very quick to notice even minor loss of detail.

  For really fine detail, look to the circular scale/calculator on the
  right side of the scene. Some of the fine lines there are extremely
  fine indeed. Looking at results from many different cameras with this
  target, we found that camera noise-suppression systems often confuse
  the fine lines with image noise, and so flatten them out. There’s also
  a nice range of fine text sizes in this chart as well, once again great
  visual cues for resolution and detail.

• Highlight Detail: Three elements in this scene
  show off (or show up) a camera’s ability to hold onto highlight detail.
  As mentioned above, the salt grains (and reflections of the studio lights)
  in the salt mill are examples of fairly subtle highlight detail that
  cameras’ anti-noise processing sometimes obliterate. The folded white
  cloth under the mug on the right side of the frame likewise shows a
  lot of white-on-white detail that is easy to lose, particularly if a
  camera’s tone curve is too contrasty. As it turns out though, the most
  sensitive test of a camera’s highlight abilities seems to be the hank
  of white embroidery thread in the upper right corner. These fibers are
  unusually bright and reflective, so its easy for a camera to blow out
  detail in them.

• Shadow Detail: Several elements of this subject
  are useful for evaluating shadow detail, particularly the black mug
  and the pieces of folded black velvet, both under and inside the mug.
  The bottoms of the beer bottles also provide some gradations of deep
  shadow, and the clump of peppers in the bottom of the pepper oil bottle
  had a fair bit of detail that’s far down at the shadow end of the tone
  curve.

  We were actually surprised when we constructed this scene just how dark
  the velvet and sides of the beer bottles ended up being. Even with the
  bright studio lights shining directly on it, the velvet in particular
  stays way, way down at the shadow end of the tone curve. With most cameras
  and on most monitors, the velvet will simply appear as an unrelieved
  swatch of black. To see whether it contains deep detail or not, in most
  cases you’ll have to open the file in an image editor and boost the
  brightness dramatically, to bring the detail up into a visible range.

• Preservation of “Shape” in Strong Colors:
  As you approach the extremes of a camera’s color gamut (its range of
  recordable colors), it becomes more and more difficult for the camera
  to show fine gradations of tone, because one or more of the RGB color
  channels are close to saturation. It’s not uncommon to see a brightly
  colored piece of clothing or a vibrant flower appear in digicam photos
  as just a blob of color, because the camera ran up against the limits
  of its color gamut. The brightly colored embroidery threads in the upper
  right portion of the Still Life target are good examples of situations
  where this might happen. Pay particular attention to the bright red
  and dark blue colors here, as these are both colors near the edge of
  the typical sRGB color gamut.

• Color accuracy and white balance: It’s pretty
  small in there, but we’ve included a mini-MacBeth chart, which displays
  very carefully controlled color swatches. Our Multi Target (see below)
  sports a full-sized MacBeth chart, but the one here serves as a good
  check of color balance and rendition, and is also useful for checking
  white balance on this particular shot.

• Image noise and detail vs ISO: As mentioned above,
  this target contains many elements useful for evaluating detail loss
  to anti-noise processing. We’ll therefore always shoot a full set of
  test images of this target across each camera’s ISO range, for every
  camera we test. (See below.)

Multi Target

Our new “Multi Target” was first put into use in April, 2009, replacing the earlier “interim” design. This target incorporates a number of elements that combine aspects of the previous Multi target, as well as the previous Viewfinder Accuracy or “VFA” chart. Here’s some of what you’ll find in this target:

Framing marks: This chart evolved from the earlier Viewfinder Accuracy chart, so one of its major uses is to measure viewfinder accuracy. (See notes in the Viewfinder Accuracy section, at the bottom of this page, for more information on this.)

USAF resolution targets: An important use of this target is in evaluating lens quality, looking how well sharpness holds up as you move from the center to the corners of the frame. The little “USAF” resolution targets arranged at the center, in the corners, and along the diagonals are very useful for making fine judgements about blur, flare and aberrations in the image. We generally show crops of a USAF chip from a corner of the target and from the center, to show how lenses hold sharpness at wide and telephoto focal lengths.

Alignment “bullseyes:” We find these graphics from the graphic arts world (used to align sheets of film in the old film-based prepress days) very useful for seeing chromatic aberration in lenses. The bold black/white elements are good for seeing the colored fringes caused by CA in the corners of the frame.

MacBeth ColorChecker Chart: This is about as common a color standard as you can get these days, very widely available for only mildly exorbitant cost, and quite well controlled in its production. It thus serves as a good basis of comparison between cameras and between test setups. Imatest also understands the MacBeth colors very well, and uses them to produce its color accuracy map that we feature in all our reviews.

MacBeth ColorChecker SG Chart: The ColorChecker SG chart provides a wider range of colors, to give a more detailed map of a camera’s color handling. We haven’t begun using this chart in the color-accuracy graphs we routinely offer, but expect to do so at some point in the future. In the meantime, we sometimes refer to this chart internally, to get a more complete idea of what a camera’s color map looks like.

Log C/F Target: The progressive resolution pattern located just below the center of the target is a Log C/F (logarithmic contrast vs spatial frequency) chart. Digital camera noise reduction routines work by looking at levels of local contrast, flattening-out detail at progressively lower spatial frequencies as the local contrast decreases. (This is very commonly seen in human hair, grass, foliage, and other subjects with subtly-contrasting fine detail.) This chart lets Imatest analyze just how a camera makes the tradeoff between contrast, detail, and image noise.

Color Starbursts: The six circular starburst elements arranged around the target are intended to reveal de-mosaicing artifacts and color-dependent resolution issues. The six starbursts provide examples of each combination of RGB colors intersecting each other. (That is, red, green, and blue against black, plus red against green, green against blue, and blue against red.) Given that the most common sensor RGB color filter pattern (the so-called Bayer) pattern has twice as many green pixels as red or blue, you’ll generally see that the green/black starburst shows the best resolution, while the blue/red one shows the worst. The effects of different sensor geometries and color filter array patterns will be revealed here.

Musicians Image: Synthetic test patterns only tell you so much. While we have a lot of pictorial images in our other test targets, we thought it would be useful to include a small “natural” image here as well.

Resolution Target

Part of the impetus in developing the new Multi Target in April 2009 was to switch to using a new 2x target for the resolution measurements, since the original ISO 12233 chart we’d used since the site first began in 1998(!) had become inadequate for testing the highest-resolution cameras. We’d for quite some time had a “homemade” 2x target, employing a shrunk version of the ISO 12233 chart, shot at 1/2 size. The resolution numbers on that chart all needed to be doubled to convert to the actual values, though, so we decided to go with a commercial 2x target to eliminate possible confusion on the part of our readers. Numbers on this new 2x resolution chart now read directly in hundreds of lines/picture height. (Because almost all of the area of this new resolution chart is now meaningful for resolution measurements, there was no longer space on it to overlay the MacBeth and other color targets on our prior Multi chart; hence the simultaneous change in our Multi target.)

“Sunlit” Portrait
(This is our “Outdoor” Portrait test – read more about it
here.)

The lighting in this shot is deliberately awful, about what you’d expect
from noontime sunshine here in the Atlanta, GA area. (In fact, the color
balance has been chosen to pretty well match the hazy sunshine here in
mid-August.)

The reason for the harsh lighting is to provide a real “torture
test” of how cameras handle conditions of extreme contrast; and in
particular, how well they do holding onto highlight detail.

Look for:

• Overall color: Matching summer sunlight here in the South,
  the lighting in this scene is a bit more yellow-tinged than that in
  many parts of the country, or in the fall or winter. – So there may
  be an overall warm cast to the color. That said though, there’s a fair
  range of color represented in the bouquet, presenting a tough challenge
  for the cameras. For some reason, the blue flowers seem particularly
  hard to handle, with many cameras rendering them as purple. (In real
  life, they’re a light shade of navy blue, with just a bit of purple
  in them.)
• Skin tones: The overall slight warm cast will tend to leave
  the model’s skin tones a bit on the warm side as well. Nonetheless,
  look to see if her skin seems overly pink or if they have a too-bright
  tinge of yellow: Some cameras oversaturate skin tones (make their color
  too intense), leading to an almost sunburned look. A little oversaturation
  can make for a more “healthy-looking” complexion, but it doesn’t
  take much variation for skin tones to look unnatural.
• Highlight detail: When the model’s skin tones are at a more
  or less normal level of brightness, how much detail can you see in her
  shirt? Does it blow out entirely to white, or can you still see the
  creases and folds in the fabric?
• Overall contrast: Most consumer digital cameras produce bright,
  contrasty images, because that’s what most consumers like. Unfortunately,
  under bright sunlit conditions, many such cameras produce images with
  little or no highlight detail, and dark, plugged-up looking shadows.
• Shadow detail: The area under the flower bouquet is in quite deep shadow. Does the camera in question retain good detail here, with low image noise? To see, you may need to download the image and play with it in Photoshop(tm) or another imaging program. Brighten the image, and see how far detail extends into the shadows. Photo printers are generally much better at showing shadow detail than are CRTs or LCDs, so you’ll want a camera that preserves good detail here. The ability to boost brightness without encountering too much image noise is important if you ever have to “rescue” an underexposed image on the computer.
• Detail in areas of subtle contrast: Most digital cameras employ
  some sort of noise-suppression to remove electronic noise from their
  images. Noise suppression is a good thing, but only if it’s not overdone.
  Too much noise suppression will “flatten out” subtle detail
  in areas of reduced contrast. You can often see this in hair, where
  the individual strands become blurred, and the image takes on an almost
  watercolor effect. Look at the detail in the model’s hair, and compare
  how it looks with different cameras in the Comparometer.

To view the entire exposure series from zero to +1.0 EV, see files DP1MOUTAP0.HTM through /cameras/sigma-dp1-merrill-review/image/952414?section=samples on the thumbnail
index page.

Contrast Series
-5	-2	Default	+2	+5

Saturation Series
-5	-2	Default	+2	+5

Auto White Balance

Incandescent White Balance

Manual White Balance

The incandescent lighting used in most US homes actually has a very
strong yellow color to it. Our eyes have an amazing ability to ignore
color casts like this, something digital cameras struggle to emulate.

The incandescent lighting used for this shot is thus not only very common
here in the US, but also very difficult for most digital cameras to deal
with. While we probably want a little yellow color to remain in
the image (to convey some of the mood of the original scene), too much
will look unnatural and distort colors.

Most cameras’ auto white balance systems have a great deal of difficulty
with this shot, but many incandescent white balance settings struggle
as well. (It seems that many cameras’ incandescent settings are actually
calibrated to the tungsten lighting used in professional studio systems,
which isn’t nearly as warm-toned as typical household lighting.)

If you intend to do much shooting indoors after dark, pay careful attention
to this test, as cameras vary widely in this regard.

To view the entire exposure series from zero to +1.0 EV, see files DP1MINBAP0.HTM through DP1MINBAP3.HTM on the thumbnail
index page.

ISO Series
“ISO equivalent” refers to a camera’s light sensitivity. ISO
200 represents twice the sensitivity of ISO 100, meaning that you can
use a shutter speed that’s twice as fast. Higher ISO settings are often
required to get any picture at all when shooting after dark, but even
in full daylight, using a higher ISO can help you freeze fast action.
The problem is, increasing a digital camera’s ISO also increases image
noise. In practical terms, this means that higher-ISO images often can’t
be used to produce prints as large as lower-ISO ones. The tricky thing
here is that high-ISO images often look much different when printed at
various sizes than they do when viewed on-screen. In particular, for any
level of image noise, you’ll often find that while noise is quite evident
at larger print sizes, as you reduce the size of the prints, there will
come a point where it suddenly ceases to be an issue. We routinely print
high-ISO photos from the cameras we test on our studio printer (currently
a Canon i9900) at a range of sizes, and report our findings. If you’re
interested in investigating the effect of image noise for yourself, don’t
judge cameras’ performance by how their images look on your CRT/LCD, viewed
pixel-for-pixel. Rather, download the test shots linked in the table below
and output them on your own printer, so you can see how prints of various
sizes will actually look.

One additional note about this particular test series though: Because
these images are shot under household incandescent lighting, the camera
has to boost its blue-channel signal quite a bit to get back to a neutral
color balance. Since the blue channel is generally the one with the most
noise, this makes this shot a real acid test of noise performance. Noise
levels in high-ISO shots taken under daylight conditions usually won’t
show as much noise. (See the “Far Field” test for examples of
high ISO shots captured in daylight.)

Like several of our tests, these images are actually photos of a high-resolution poster, shot under studio lighting. The shots for this camera were captured with our third-generation House poster, which was assembled from 45 separate 11-megapixel images, shot with a very high-quality lens, and then stitched together into a single image. The resulting image amounts to about 450 very high-quality megapixels. This should have sufficient detail to comfortably challenge cameras up to at least 80-100 megapixels. (And even with higher-resolution cameras, we believe that the camera lenses themselves would be more likely to limit resolution than would the detail in the poster.)

Why did we choose to shoot a picture of a picture? The idea was to show a typical subject (a house and surrounding foliage) in a way that would be absolutely consistent from camera to camera. Any outdoor subject is going to vary considerably from day to day, as the lighting changes with the weather, atmospheric conditions, and season. Shooting a poster lets us compare images from cameras shot weeks, months, or even years apart, with the sure knowledge that nothing has changed from one shot to another.

Things to look for here are fine detail, as seen in the foliage and tree limbs against the sky, sharpness in the corners, and the preservation of subtle detail in the shaded brick patterns. – Many cameras with overactive noise suppression severely blur the brick patterns that are in shadow.

Far Field Test 2

While the House poster in the shot above provides absolute repeatability from test to test, it doesn’t offer the range of brightness (dynamic range) that the original scene had, nor does it contain the nearly infinite range of fine detail found in nature. For these reasons, we shoot an outdoor photo of a building, to provide a more challenging (if more variable) subject for the cameras. Until Summer of 2011, we used the original house from the House poster for this “Far Field” shot, but the ever-encroaching trees made it less and less useful. The coup de grace for that subject was when the neighbor across the street relandscaped his yard so we could no longer shoot from our original position.

In place of the original house, we’ve switched to using the Roswell, GA City Hall building, which offers many of the elements of the original Far Field subject, but in a setting where we’re guaranteed access into the future, and without the threat of trees eventually obscuring the building. In this shot, we look at how the camera handles rendering of subtle detail (the bricks, both in the sunlight and in the deep shadows under the entryway roof), how it handles bright highlights (the white columns and reflections from the dome), and shadow detail (in the dark areas beneath the shrubs on either side of the steps).

Note though, that because this is shot outdoors, the character of the light is unavoidably going to change quite a bit, depending on the atmospheric humidity and the time of year. – You thus shouldn’t rely on it for absolute comparisons between cameras, since it’s unlikely that conditions will be exactly the same from one test to the next.

Contrast Series
-5	-2	Default	+2	+5

Pine

Our current Far-Field shot lacks one feature that we found invaluable in the original one; namely the pine foliage that was in the background of the old subject. The pine foliage provided very fine detail that we could also count on being the same year-round. No other shots we capture provided this combination of fine detail at a distance, so we added a new shot (“PINE” in all the filenames) of a large pine tree near our office, shot at roughly the same scale as the pines in the old Far-Field shot, so comparisons can still be made against older images. Use this shot to see how well each camera handles really fine detail, both in high-contrast situations (the needles against the sky) and in low-contrast ones (the needles against each other). The tangles of small branches against the sky are also very revealing.

The combination of the FAR2 and PINE shots provide a wealth of information about cameras’ image-rendering prowess with natural subjects shot at “infinity,” filling testing needs that can’t be entirely met with studio shots alone.

Because most of its various elements are now contained or represented in the combination of the Still Life and Multi Target shots, we no longer routinely shoot the Davebox by itself, as would normally appear in this space. (We do however, still use it for our low light test below.)

Standard Macro Shot

Many are interested in close-up or “macro” photography. This test shows the best results we could obtain using each camera’s macro mode. What to look for:

• Minimum Macro Area: What’s the smallest area that the camera can photograph? – We calculate this and report on it in the Test Results section of each review.
• Softness in corners?: The images from most digital cameras get fairly soft when shooting in macro mode. Some are better than others though, you can use the texture of the paper fibers or the details in the dollar bill to compare corner sharpness between models.
• Flash performance in macro mode?: Macro mode flash performance varies widely between cameras. Some can’t throttle down the flash enough, others throttle down enough, but have very uneven lighting, while others have lenses that project into the path of the flash, casting strong shadows. The Macro with Flash shot here will show you what to expect. (Note: the Sigma DP1m does not have a built-in flash.)

Low-Light Tests

Low light photography is an area where there are really enormous differences
between digital camera models. This test starts at a light level about
equivalent to typical city street lighting at night (one foot-candle),
and then progresses down from there, each successive test being at half
the light level of the preceding one. You may also see the effect of poor
low-light autofocus in some of these shots, although we use a different
test setup to check autofocus performance more directly. (The results
of which are reported on in the main Test Results section.) Things to
look for here include:

• Exposure limit: What’s the darkest level a camera can handle
  at each ISO setting? If the leftmost images are reasonably bright, the
  camera should do fine with typical city night scenes. If you’re shooting
  in the more dimly-lit suburbs, you’ll need a camera capable of producing
  good images one, two, or three steps to the right of that.
• Autofocus Limit: How dark can you shoot and still get well-focused
  pictures?
• White Balance: Does the camera’s white balance suffer at low
  light levels? (Many do.)
• Noise Levels: Look at the photos, print them on your own photo
  printer. How large a print can you make at acceptable quality levels,
  at various ISO settings and light levels?
• Detail loss to anti-noise processing?: Do details in the white
  gauze and even in the lettering on the test targets suffer at lower
  light levels?

(Note: If you’d like to use a light meter to check light levels
for subjects you might be interested in shooting, a light level of one
foot-candle corresponds to a normal exposure of two seconds at f/2.8 and
ISO 100.)

	1 fc 11 lux	1/2 fc 5.5 lux	1/4 fc 2.7 lux	1/8 fc 1.3 lux	1/16 fc 0.67 lux
ISO 100	1.3 s f2.8	2.5 s f2.8	5 s f2.8	10 s f2.8	20 s f2.8
ISO 200	0.6 s f2.8	1.3 s f2.8	2.5 s f2.8	5 s f2.8	10 s f2.8
ISO 400	0.3 s f2.8	0.6 s f2.8	1.3 s f2.8	2.5 s f2.8	5 s f2.8
ISO 800	1/6 s f2.8	0.3 s f2.8	0.6 s f2.8	1.3 s f2.8	2.5 s f2.8
ISO 1600	1/13 s f2.8	1/6 s f2.8	0.3 s f2.8	0.6 s f2.8	1.3 s f2.8
ISO 3200	1/25 s f2.8	1/13 s f2.8	1/6 s f2.8	0.3 s f2.8	0.6 s f2.8
ISO 6400	1/50 s f2.8	1/25 s f2.8	1/13 s f2.8	1/6 s f2.8	0.3 s f2.8

Viewfinder Accuracy

Viewfinder accuracy is an important parameter, especially for shots
where framing is critical. The optical viewfinders on most digital cameras
match the (poor) accuracy of those on film cameras, typically showing
only about 85% of the actual final frame area. It’s likely that this is
a deliberate design choice by the camera engineers, to help avoid users
accidentally cutting off the heads of their subjects. We disagree with
this approach, or at least feel that it should be mitigated a bit, perhaps
by increasing the accuracy to 90 to 95%.

Unlike the optical viewfinders, the LCD viewfinders on most digital
cameras tend to be quite accurate. There are exceptions though, so
this test is still important.

Things to look for on this test chart are:

• Optical/Electronic viewfinder accuracy: When we shoot this target in the studio, we line things up so the center of the bright red outline on the target is just visible at the edges of the viewfinder frame. The resulting photo then very directly shows how accurate the viewfinder is. The fine black lines mark progressive increments of 1% of increased or decreased frame area. The bold black lines mark 5% increments. The lines let you get an approximate idea of frame accuracy visually, but we measure the actual pixel dimensions to derive the accuracy numbers we report in our reviews.
• LCD monitor accuracy: This is the same test, but framed with the LCD monitor instead of the optical viewfinder. As mentioned above, LCD monitors are usually more accurate than optical viewfinders, especially in point & shoot digicams.

LCD Monitor

Still Life

This is our new “Still Life” test target. We’re combining
some of the elements from previous shots (DaveBox and Res Chart) into
this and the “Multi Target” shot below, plus added a number
of elements that are very revealing of various camera characteristics
and foibles.

Here’s what to look for in this target:

• Tone-on-tone detail & noise suppression:
  The cloth swatches in the pinwheel were chosen because they show a lot
  of tone-on-tone detail, across a broad range of colors. This is just
  the sort of detail that noise suppression processing tends to flatten
  out. If you look at the detail in these swatches as the ISO increases,
  you’ll see just where different cameras start to lose subtle detail.
  — The white and tan swatches and the dark swatches tend to be particularly
  revealing of this. The label of the vinegar bottle (second from the
  right) is another great place to look for lost detail from noise suppression,
  as the image of the person at the top of the label is actually a depiction
  of a mosaic. The dark colors in the background and in the figure’s clothes
  contain detail that’s very quickly lost when a camera’s noise suppression
  system kicks in. Cameras with really high-quality, low-noise sensors
  that require little noise suppression will be able to hold onto the
  detail in these areas, many others will show only a uniform swath of
  smudged color.

  Another place where you’ll quickly see the effects of over-aggressive
  noise suppression is in the white salt grains of the salt grinder in
  lower left. Cameras are often more conservative about suppressing noise
  in highlight areas (because our eyes tend to see less of it there),
  but many cameras seem to have a hard time holding onto the subtle shadings
  that distinguish the salt grains from each other, particularly at higher
  ISO levels.

• Fine Detail: You’ll find a lot of fine detail
  in the label of the beer bottle on the right, in its fine cursive text,
  but the other bottle labels hold a lot of fine detail as well. Fine
  text is often a good visual indicator of resolution, because our brains
  have an excellent idea of what the text should look like, so
  are very quick to notice even minor loss of detail.

  For really fine detail, look to the circular scale/calculator on the
  right side of the scene. Some of the fine lines there are extremely
  fine indeed. Looking at results from many different cameras with this
  target, we found that camera noise-suppression systems often confuse
  the fine lines with image noise, and so flatten them out. There’s also
  a nice range of fine text sizes in this chart as well, once again great
  visual cues for resolution and detail.

• Highlight Detail: Three elements in this scene
  show off (or show up) a camera’s ability to hold onto highlight detail.
  As mentioned above, the salt grains (and reflections of the studio lights)
  in the salt mill are examples of fairly subtle highlight detail that
  cameras’ anti-noise processing sometimes obliterate. The folded white
  cloth under the mug on the right side of the frame likewise shows a
  lot of white-on-white detail that is easy to lose, particularly if a
  camera’s tone curve is too contrasty. As it turns out though, the most
  sensitive test of a camera’s highlight abilities seems to be the hank
  of white embroidery thread in the upper right corner. These fibers are
  unusually bright and reflective, so its easy for a camera to blow out
  detail in them.

• Shadow Detail: Several elements of this subject
  are useful for evaluating shadow detail, particularly the black mug
  and the pieces of folded black velvet, both under and inside the mug.
  The bottoms of the beer bottles also provide some gradations of deep
  shadow, and the clump of peppers in the bottom of the pepper oil bottle
  had a fair bit of detail that’s far down at the shadow end of the tone
  curve.

  We were actually surprised when we constructed this scene just how dark
  the velvet and sides of the beer bottles ended up being. Even with the
  bright studio lights shining directly on it, the velvet in particular
  stays way, way down at the shadow end of the tone curve. With most cameras
  and on most monitors, the velvet will simply appear as an unrelieved
  swatch of black. To see whether it contains deep detail or not, in most
  cases you’ll have to open the file in an image editor and boost the
  brightness dramatically, to bring the detail up into a visible range.

• Preservation of “Shape” in Strong Colors:
  As you approach the extremes of a camera’s color gamut (its range of
  recordable colors), it becomes more and more difficult for the camera
  to show fine gradations of tone, because one or more of the RGB color
  channels are close to saturation. It’s not uncommon to see a brightly
  colored piece of clothing or a vibrant flower appear in digicam photos
  as just a blob of color, because the camera ran up against the limits
  of its color gamut. The brightly colored embroidery threads in the upper
  right portion of the Still Life target are good examples of situations
  where this might happen. Pay particular attention to the bright red
  and dark blue colors here, as these are both colors near the edge of
  the typical sRGB color gamut.

• Color accuracy and white balance: It’s pretty
  small in there, but we’ve included a mini-MacBeth chart, which displays
  very carefully controlled color swatches. Our Multi Target (see below)
  sports a full-sized MacBeth chart, but the one here serves as a good
  check of color balance and rendition, and is also useful for checking
  white balance on this particular shot.

• Image noise and detail vs ISO: As mentioned above,
  this target contains many elements useful for evaluating detail loss
  to anti-noise processing. We’ll therefore always shoot a full set of
  test images of this target across each camera’s ISO range, for every
  camera we test. (See below.)

Multi Target

Our new “Multi Target” was first put into use in April, 2009, replacing the earlier “interim” design. This target incorporates a number of elements that combine aspects of the previous Multi target, as well as the previous Viewfinder Accuracy or “VFA” chart. Here’s some of what you’ll find in this target:

Framing marks: This chart evolved from the earlier Viewfinder Accuracy chart, so one of its major uses is to measure viewfinder accuracy. (See notes in the Viewfinder Accuracy section, at the bottom of this page, for more information on this.)

USAF resolution targets: An important use of this target is in evaluating lens quality, looking how well sharpness holds up as you move from the center to the corners of the frame. The little “USAF” resolution targets arranged at the center, in the corners, and along the diagonals are very useful for making fine judgements about blur, flare and aberrations in the image. We generally show crops of a USAF chip from a corner of the target and from the center, to show how lenses hold sharpness at wide and telephoto focal lengths.

Alignment “bullseyes:” We find these graphics from the graphic arts world (used to align sheets of film in the old film-based prepress days) very useful for seeing chromatic aberration in lenses. The bold black/white elements are good for seeing the colored fringes caused by CA in the corners of the frame.

MacBeth ColorChecker Chart: This is about as common a color standard as you can get these days, very widely available for only mildly exorbitant cost, and quite well controlled in its production. It thus serves as a good basis of comparison between cameras and between test setups. Imatest also understands the MacBeth colors very well, and uses them to produce its color accuracy map that we feature in all our reviews.

MacBeth ColorChecker SG Chart: The ColorChecker SG chart provides a wider range of colors, to give a more detailed map of a camera’s color handling. We haven’t begun using this chart in the color-accuracy graphs we routinely offer, but expect to do so at some point in the future. In the meantime, we sometimes refer to this chart internally, to get a more complete idea of what a camera’s color map looks like.

Log C/F Target: The progressive resolution pattern located just below the center of the target is a Log C/F (logarithmic contrast vs spatial frequency) chart. Digital camera noise reduction routines work by looking at levels of local contrast, flattening-out detail at progressively lower spatial frequencies as the local contrast decreases. (This is very commonly seen in human hair, grass, foliage, and other subjects with subtly-contrasting fine detail.) This chart lets Imatest analyze just how a camera makes the tradeoff between contrast, detail, and image noise.

Color Starbursts: The six circular starburst elements arranged around the target are intended to reveal de-mosaicing artifacts and color-dependent resolution issues. The six starbursts provide examples of each combination of RGB colors intersecting each other. (That is, red, green, and blue against black, plus red against green, green against blue, and blue against red.) Given that the most common sensor RGB color filter pattern (the so-called Bayer) pattern has twice as many green pixels as red or blue, you’ll generally see that the green/black starburst shows the best resolution, while the blue/red one shows the worst. The effects of different sensor geometries and color filter array patterns will be revealed here.

Musicians Image: Synthetic test patterns only tell you so much. While we have a lot of pictorial images in our other test targets, we thought it would be useful to include a small “natural” image here as well.

Resolution Target

Part of the impetus in developing the new Multi Target in April 2009 was to switch to using a new 2x target for the resolution measurements, since the original ISO 12233 chart we’d used since the site first began in 1998(!) had become inadequate for testing the highest-resolution cameras. We’d for quite some time had a “homemade” 2x target, employing a shrunk version of the ISO 12233 chart, shot at 1/2 size. The resolution numbers on that chart all needed to be doubled to convert to the actual values, though, so we decided to go with a commercial 2x target to eliminate possible confusion on the part of our readers. Numbers on this new 2x resolution chart now read directly in hundreds of lines/picture height. (Because almost all of the area of this new resolution chart is now meaningful for resolution measurements, there was no longer space on it to overlay the MacBeth and other color targets on our prior Multi chart; hence the simultaneous change in our Multi target.)

“Sunlit” Portrait
(This is our “Outdoor” Portrait test – read more about it
here.)

The lighting in this shot is deliberately awful, about what you’d expect
from noontime sunshine here in the Atlanta, GA area. (In fact, the color
balance has been chosen to pretty well match the hazy sunshine here in
mid-August.)

The reason for the harsh lighting is to provide a real “torture
test” of how cameras handle conditions of extreme contrast; and in
particular, how well they do holding onto highlight detail.

Look for:

• Overall color: Matching summer sunlight here in the South,
  the lighting in this scene is a bit more yellow-tinged than that in
  many parts of the country, or in the fall or winter. – So there may
  be an overall warm cast to the color. That said though, there’s a fair
  range of color represented in the bouquet, presenting a tough challenge
  for the cameras. For some reason, the blue flowers seem particularly
  hard to handle, with many cameras rendering them as purple. (In real
  life, they’re a light shade of navy blue, with just a bit of purple
  in them.)
• Skin tones: The overall slight warm cast will tend to leave
  the model’s skin tones a bit on the warm side as well. Nonetheless,
  look to see if her skin seems overly pink or if they have a too-bright
  tinge of yellow: Some cameras oversaturate skin tones (make their color
  too intense), leading to an almost sunburned look. A little oversaturation
  can make for a more “healthy-looking” complexion, but it doesn’t
  take much variation for skin tones to look unnatural.
• Highlight detail: When the model’s skin tones are at a more
  or less normal level of brightness, how much detail can you see in her
  shirt? Does it blow out entirely to white, or can you still see the
  creases and folds in the fabric?
• Overall contrast: Most consumer digital cameras produce bright,
  contrasty images, because that’s what most consumers like. Unfortunately,
  under bright sunlit conditions, many such cameras produce images with
  little or no highlight detail, and dark, plugged-up looking shadows.
• Shadow detail: The area under the flower bouquet is in quite deep shadow. Does the camera in question retain good detail here, with low image noise? To see, you may need to download the image and play with it in Photoshop(tm) or another imaging program. Brighten the image, and see how far detail extends into the shadows. Photo printers are generally much better at showing shadow detail than are CRTs or LCDs, so you’ll want a camera that preserves good detail here. The ability to boost brightness without encountering too much image noise is important if you ever have to “rescue” an underexposed image on the computer.
• Detail in areas of subtle contrast: Most digital cameras employ
  some sort of noise-suppression to remove electronic noise from their
  images. Noise suppression is a good thing, but only if it’s not overdone.
  Too much noise suppression will “flatten out” subtle detail
  in areas of reduced contrast. You can often see this in hair, where
  the individual strands become blurred, and the image takes on an almost
  watercolor effect. Look at the detail in the model’s hair, and compare
  how it looks with different cameras in the Comparometer.

To view the entire exposure series from zero to +1.0 EV, see files DP1MOUTAP0.HTM through /cameras/sigma-dp1-merrill-review/image/952211?section=test on the thumbnail
index page.

Contrast Series
-5	-2	Default	+2	+5

Saturation Series
-5	-2	Default	+2	+5

Auto White Balance

Incandescent White Balance

Manual White Balance

The incandescent lighting used in most US homes actually has a very
strong yellow color to it. Our eyes have an amazing ability to ignore
color casts like this, something digital cameras struggle to emulate.

The incandescent lighting used for this shot is thus not only very common
here in the US, but also very difficult for most digital cameras to deal
with. While we probably want a little yellow color to remain in
the image (to convey some of the mood of the original scene), too much
will look unnatural and distort colors.

Most cameras’ auto white balance systems have a great deal of difficulty
with this shot, but many incandescent white balance settings struggle
as well. (It seems that many cameras’ incandescent settings are actually
calibrated to the tungsten lighting used in professional studio systems,
which isn’t nearly as warm-toned as typical household lighting.)

If you intend to do much shooting indoors after dark, pay careful attention
to this test, as cameras vary widely in this regard.

To view the entire exposure series from zero to +1.0 EV, see files DP1MINBAP0.HTM through DP1MINBAP3.HTM on the thumbnail
index page.

ISO Series
“ISO equivalent” refers to a camera’s light sensitivity. ISO
200 represents twice the sensitivity of ISO 100, meaning that you can
use a shutter speed that’s twice as fast. Higher ISO settings are often
required to get any picture at all when shooting after dark, but even
in full daylight, using a higher ISO can help you freeze fast action.
The problem is, increasing a digital camera’s ISO also increases image
noise. In practical terms, this means that higher-ISO images often can’t
be used to produce prints as large as lower-ISO ones. The tricky thing
here is that high-ISO images often look much different when printed at
various sizes than they do when viewed on-screen. In particular, for any
level of image noise, you’ll often find that while noise is quite evident
at larger print sizes, as you reduce the size of the prints, there will
come a point where it suddenly ceases to be an issue. We routinely print
high-ISO photos from the cameras we test on our studio printer (currently
a Canon i9900) at a range of sizes, and report our findings. If you’re
interested in investigating the effect of image noise for yourself, don’t
judge cameras’ performance by how their images look on your CRT/LCD, viewed
pixel-for-pixel. Rather, download the test shots linked in the table below
and output them on your own printer, so you can see how prints of various
sizes will actually look.

One additional note about this particular test series though: Because
these images are shot under household incandescent lighting, the camera
has to boost its blue-channel signal quite a bit to get back to a neutral
color balance. Since the blue channel is generally the one with the most
noise, this makes this shot a real acid test of noise performance. Noise
levels in high-ISO shots taken under daylight conditions usually won’t
show as much noise. (See the “Far Field” test for examples of
high ISO shots captured in daylight.)

Like several of our tests, these images are actually photos of a high-resolution poster, shot under studio lighting. The shots for this camera were captured with our third-generation House poster, which was assembled from 45 separate 11-megapixel images, shot with a very high-quality lens, and then stitched together into a single image. The resulting image amounts to about 450 very high-quality megapixels. This should have sufficient detail to comfortably challenge cameras up to at least 80-100 megapixels. (And even with higher-resolution cameras, we believe that the camera lenses themselves would be more likely to limit resolution than would the detail in the poster.)

Why did we choose to shoot a picture of a picture? The idea was to show a typical subject (a house and surrounding foliage) in a way that would be absolutely consistent from camera to camera. Any outdoor subject is going to vary considerably from day to day, as the lighting changes with the weather, atmospheric conditions, and season. Shooting a poster lets us compare images from cameras shot weeks, months, or even years apart, with the sure knowledge that nothing has changed from one shot to another.

Things to look for here are fine detail, as seen in the foliage and tree limbs against the sky, sharpness in the corners, and the preservation of subtle detail in the shaded brick patterns. – Many cameras with overactive noise suppression severely blur the brick patterns that are in shadow.

Far Field Test 2

While the House poster in the shot above provides absolute repeatability from test to test, it doesn’t offer the range of brightness (dynamic range) that the original scene had, nor does it contain the nearly infinite range of fine detail found in nature. For these reasons, we shoot an outdoor photo of a building, to provide a more challenging (if more variable) subject for the cameras. Until Summer of 2011, we used the original house from the House poster for this “Far Field” shot, but the ever-encroaching trees made it less and less useful. The coup de grace for that subject was when the neighbor across the street relandscaped his yard so we could no longer shoot from our original position.

In place of the original house, we’ve switched to using the Roswell, GA City Hall building, which offers many of the elements of the original Far Field subject, but in a setting where we’re guaranteed access into the future, and without the threat of trees eventually obscuring the building. In this shot, we look at how the camera handles rendering of subtle detail (the bricks, both in the sunlight and in the deep shadows under the entryway roof), how it handles bright highlights (the white columns and reflections from the dome), and shadow detail (in the dark areas beneath the shrubs on either side of the steps).

Note though, that because this is shot outdoors, the character of the light is unavoidably going to change quite a bit, depending on the atmospheric humidity and the time of year. – You thus shouldn’t rely on it for absolute comparisons between cameras, since it’s unlikely that conditions will be exactly the same from one test to the next.

Contrast Series
-5	-2	Default	+2	+5

Pine

Our current Far-Field shot lacks one feature that we found invaluable in the original one; namely the pine foliage that was in the background of the old subject. The pine foliage provided very fine detail that we could also count on being the same year-round. No other shots we capture provided this combination of fine detail at a distance, so we added a new shot (“PINE” in all the filenames) of a large pine tree near our office, shot at roughly the same scale as the pines in the old Far-Field shot, so comparisons can still be made against older images. Use this shot to see how well each camera handles really fine detail, both in high-contrast situations (the needles against the sky) and in low-contrast ones (the needles against each other). The tangles of small branches against the sky are also very revealing.

The combination of the FAR2 and PINE shots provide a wealth of information about cameras’ image-rendering prowess with natural subjects shot at “infinity,” filling testing needs that can’t be entirely met with studio shots alone.

Because most of its various elements are now contained or represented in the combination of the Still Life and Multi Target shots, we no longer routinely shoot the Davebox by itself, as would normally appear in this space. (We do however, still use it for our low light test below.)

Standard Macro Shot

Many are interested in close-up or “macro” photography. This test shows the best results we could obtain using each camera’s macro mode. What to look for:

• Minimum Macro Area: What’s the smallest area that the camera can photograph? – We calculate this and report on it in the Test Results section of each review.
• Softness in corners?: The images from most digital cameras get fairly soft when shooting in macro mode. Some are better than others though, you can use the texture of the paper fibers or the details in the dollar bill to compare corner sharpness between models.
• Flash performance in macro mode?: Macro mode flash performance varies widely between cameras. Some can’t throttle down the flash enough, others throttle down enough, but have very uneven lighting, while others have lenses that project into the path of the flash, casting strong shadows. The Macro with Flash shot here will show you what to expect. (Note: the Sigma DP1m does not have a built-in flash.)

Low-Light Tests

Low light photography is an area where there are really enormous differences
between digital camera models. This test starts at a light level about
equivalent to typical city street lighting at night (one foot-candle),
and then progresses down from there, each successive test being at half
the light level of the preceding one. You may also see the effect of poor
low-light autofocus in some of these shots, although we use a different
test setup to check autofocus performance more directly. (The results
of which are reported on in the main Test Results section.) Things to
look for here include:

• Exposure limit: What’s the darkest level a camera can handle
  at each ISO setting? If the leftmost images are reasonably bright, the
  camera should do fine with typical city night scenes. If you’re shooting
  in the more dimly-lit suburbs, you’ll need a camera capable of producing
  good images one, two, or three steps to the right of that.
• Autofocus Limit: How dark can you shoot and still get well-focused
  pictures?
• White Balance: Does the camera’s white balance suffer at low
  light levels? (Many do.)
• Noise Levels: Look at the photos, print them on your own photo
  printer. How large a print can you make at acceptable quality levels,
  at various ISO settings and light levels?
• Detail loss to anti-noise processing?: Do details in the white
  gauze and even in the lettering on the test targets suffer at lower
  light levels?

(Note: If you’d like to use a light meter to check light levels
for subjects you might be interested in shooting, a light level of one
foot-candle corresponds to a normal exposure of two seconds at f/2.8 and
ISO 100.)

	1 fc 11 lux	1/2 fc 5.5 lux	1/4 fc 2.7 lux	1/8 fc 1.3 lux	1/16 fc 0.67 lux
ISO 100	1.3 s f2.8	2.5 s f2.8	5 s f2.8	10 s f2.8	20 s f2.8
ISO 200	0.6 s f2.8	1.3 s f2.8	2.5 s f2.8	5 s f2.8	10 s f2.8
ISO 400	0.3 s f2.8	0.6 s f2.8	1.3 s f2.8	2.5 s f2.8	5 s f2.8
ISO 800	1/6 s f2.8	0.3 s f2.8	0.6 s f2.8	1.3 s f2.8	2.5 s f2.8
ISO 1600	1/13 s f2.8	1/6 s f2.8	0.3 s f2.8	0.6 s f2.8	1.3 s f2.8
ISO 3200	1/25 s f2.8	1/13 s f2.8	1/6 s f2.8	0.3 s f2.8	0.6 s f2.8
ISO 6400	1/50 s f2.8	1/25 s f2.8	1/13 s f2.8	1/6 s f2.8	0.3 s f2.8

Viewfinder Accuracy

Viewfinder accuracy is an important parameter, especially for shots
where framing is critical. The optical viewfinders on most digital cameras
match the (poor) accuracy of those on film cameras, typically showing
only about 85% of the actual final frame area. It’s likely that this is
a deliberate design choice by the camera engineers, to help avoid users
accidentally cutting off the heads of their subjects. We disagree with
this approach, or at least feel that it should be mitigated a bit, perhaps
by increasing the accuracy to 90 to 95%.

Unlike the optical viewfinders, the LCD viewfinders on most digital
cameras tend to be quite accurate. There are exceptions though, so
this test is still important.

Things to look for on this test chart are:

• Optical/Electronic viewfinder accuracy: When we shoot this target in the studio, we line things up so the center of the bright red outline on the target is just visible at the edges of the viewfinder frame. The resulting photo then very directly shows how accurate the viewfinder is. The fine black lines mark progressive increments of 1% of increased or decreased frame area. The bold black lines mark 5% increments. The lines let you get an approximate idea of frame accuracy visually, but we measure the actual pixel dimensions to derive the accuracy numbers we report in our reviews.
• LCD monitor accuracy: This is the same test, but framed with the LCD monitor instead of the optical viewfinder. As mentioned above, LCD monitors are usually more accurate than optical viewfinders, especially in point & shoot digicams.

LCD Monitor