Sony A850 Imatest Results

We routinely use Norman Koren's excellent "Imatest" analysis program for quantitative, thoroughly objective analysis of digicam test images. I highly recommend it to our technically-oriented readers, as it's far and away the best, most comprehensive analysis program I've found to date.

My comments below are just brief observations of what we see in the Imatest results. A full discussion of all the data Imatest produces is really beyond the scope of this review: Visit the Imatest web site for a full discussion of what the program measures, how it performs its computations, and how to interpret its output.

Here's some of the results produced by Imatest for the Sony A850:

Interestingly, despite it supposedly having identical "guts" to the A900, it's clear that the image processing of the Sony A850 has been tweaked slightly. Like the Sony A900, the A850 showed very good color accuracy overall. Hue accuracy actually somewhat improved over that of the A900, with most of the hue shift occurring in the cyans, sky blues, reds and oranges. Average saturation was a bit higher in the A900, at 110% (oversaturated by 10% vs 5% for the A900, mostly in the deep blues and reds and some greens). Average "delta-C" color error was a bit better than the A900 though, at 4.78 for the A850 vs 5.27 after correction for saturation. All in all, a very good color response for an SLR, with the saturation boost likely to increase its appeal to consumer users. Mouse over the links below the illustration above to compare results with competing models.

Using the Adobe RGB color space (which provides a much wider gamut, or range of colors that can be expressed), the Sony A900 delivers more highly saturated color, with an average saturation of 112.9% and average saturation-corrected hue error of 4.7 "delta-C" units, which in this case is actually slightly more accurate than any of the competition. Again, mouse over the links below the illustration above to compare results with competing models. (Note that a slight change in the axes scales for this plot in the most recent version of Imatest results in minor displacements in the dots for both the "Ideal" and "Camera" data.)

 

This image shows how the Sony A900 actually rendered the colors of the MacBeth chart, compared to a numerically ideal treatment. In each color swatch, the outer perimeter shows the color as actually captured by the camera, the inner square shows the numerically correct color after correcting for the luminance of the photographed chart (as determined by a second-order curve fit to the values of the gray swatches), and the small rectangle inside the inner square shows the numerically correct color, without the luminance correction. This image shows the A850's very good hue accuracy, as well as a gamma curve that results in an overexposure of some colors, notably blues, cyans, reds, orange and some greens. (We've seen this in most cameras we test, increasing the "punch" of images by brightening saturated colors a little, while actually keeping both the hue and saturation level very close to technically accurate values.)

 

There's a lot in this particular graph, a lot more than we have room to go into here. (This set of plots has also changed a few revisions back in Imatest. Some of the plots that were once shown here are now shown in other Imatest output. Since we largely focus on the Noise Spectrum plot, we only show the graphic above, which includes that plot.)

In comparing these graphs with those from competing cameras, we've found that the Noise Spectrum graph at lower right is the most important. Cameras that manage to shift their noise spectrum to higher frequencies have much finer-grained noise structures, making their noise less visually objectionable. In the graph above, this would show up as a noise spectrum curve that remained higher on the right side, representing higher noise frequencies.

Here, we see the results at ISO 100, which is as a low extension of the normal ISO range for the A850. The luminance curve is very flat (meaning luminance noise is very fine-grained), however the color channels, particularly the red and blue, exhibit higher noise values, especially at lower frequencies. This graph reinforces our own visual observations of the A850's images, in which we saw that the luminance noise was relatively low, but that the chroma noise was much higher. It's interesting that the "spikes" in the noise spectrum that we normally see from Sony's JPEG processing don't appear to be present with the A850, so perhaps Sony's engineers have tweaked the JPEG encoder a bit.

Here again, Sony has clearly adjusted the image processing of the A850 vs that of the A900: Noise levels in the deepest shadows are somewhat higher than we was with the A900, but noise in the upper shadows through midtones is lower. The spectral energy of the blue and red channel noise is also higher in the lower frequency ranges than in the A900.

Here are the results at the A850's base ISO of 200. Again, the luminance curve is quite flat and the A850 does a good job of keeping plenty of the noise energy at high frequencies. The green channel follows the luminance channel more closely than at ISO 100, but the red and blue channels still show quite a bit more noise energy at lower frequencies than they did in the A900. Likewise, there's more noise in the darkest shadows, but less in the more visible parts of the tone curve. Bottom line, the Sony A850 overall noise levels are low at low ISO, with a frequency spectrum that has more energy on the higher frequency (right-hand) side of the curve in the green and luminance signals than do many cameras, but less so in the red. and blue.

Above is the same set of noise data at ISO 3,200. Here, the Noise Spectrum graph has a dramatic shift toward the left-hand, lower-frequency side than at ISO 200, coarsening the "grain" of the image noise patterns. The red channel also continues to have dramatically more noise energy at lower frequencies than the others. This can be seen in the A850's ISO 3,200 images as very noticeable blotches of chroma noise.

Interestingly, at ISO 3,200, the A850's noise spectrum and noise profile in primary colors (the upper right-hand plot of the group) are virtually identical to those of the A900; much more so than was the case at lower ISO settings.

 

Here's the same set of noise data at ISO 6,400. Very similar to ISO 3,200, with slightly more shift to the left indicating even coarser "grain" caused by the A850's noise reduction attempts to blur out the noise. Again, the red channel continues to have much more noise energy at lower frequency than the others. Here, we can also see hints of the spectral noise-frequency spikes previously mentioned.

 

This chart compares the Sony A850's noise performance over a range of ISOs against that of other current, full frame cameras, using default settings. While we continue to show noise plots of this sort because readers ask for them, we each time point out that the noise magnitude is only a small part of the story, the grain pattern being much more important. Also, this is a plot of luminance magnitudes, where the A850 does relatively well: A plot of chrominance noise would tell a different story. As you can see the A850's graph tracks the A900 very closely, as you'd expect. The magnitude of their image noise starts out slightly above most of the competition at ISO 100, and remains higher at all ISOs until the Sonys' high ISO noise reduction kicks in more aggressively at ISO 1,600. This brings the curve back down to the Sonys' ISO 800 level, and in-line with most of the competition, but a bit higher than the Canon 5D Mark II. There's quite a large difference at ISO 6,400, but still a pretty good performance for a camera with 24 megapixels. Keep in mind these are at default settings, so the shape and positions of the curves will be influenced by your settings.

 

Sony A900 Dynamic Range Analysis

A key parameter in a digital camera is its Dynamic Range, the range of brightness that can be faithfully recorded. At the upper end of the tonal scale, dynamic range is dictated by the point at which the RGB data "saturates" at values of 255, 255, 255. At the lower end of the tonal scale, dynamic range is determined by the point at which there ceases to be any useful difference between adjacent tonal steps. Note the use of the qualifier "useful" in there: While it's tempting to evaluate dynamic range as the maximum number of tonal steps that can be discerned at all, that measure of dynamic range has very little relevance to real-world photography. What we care about as photographers is how much detail we can pull out of the shadows before image noise becomes too objectionable. This, of course, is a very subjective matter, and will vary with the application and even the subject matter in question. (Noise will be much more visible in subjects with large areas of flat tints and subtle shading than it would in subjects with strong, highly contrasting surface texture.)

What makes most sense then, is to specify useful dynamic range in terms of the point at which image noise reaches some agreed-upon threshold. To this end, Imatest computes a number of different dynamic range measurements, based on a variety of image noise thresholds. The noise thresholds are specified in terms of f-stops of equivalent luminance variation in the final image file, and dynamic range is computed for noise thresholds of 1.0 (low image quality), 0.5 (medium image quality), 0.25 (medium-high image quality) and 0.1 (high image quality). For most photographers and most applications, the noise thresholds of 0.5 and 0.25 f-stops are probably the most relevant to the production of acceptable-quality finished images, but many noise-sensitive shooters will insist on the 0.1 f-stop limit for their most critical work.

The image below shows the test results from Imatest for an in-camera JPEG file from the Sony A900 with a nominally-exposed density step target (Stouffer 4110), and the A900's settings at their default positions, where Dynamic Range Optimization is set to Off.

These are good results for in-camera JPEGs, almost identical to the Sony A900, but not as good as recent results from other manufacturers. The tone curve shows excellent gradation in highlights, but the shadow end trails off a bit more abruptly. Noise levels are a little lower than the A900, either indicating Sony has tweaked JPEG processing, or perhaps it's just sample variation. Again, interesting that the "spikes" have reappeared in the noise spectrum plot at lower right.

Interestingly, the results with DRO set to Off were slightly better at the highest quality level (7.31 f-stops), however overall dynamic range improved very slightly to 10.7 f-stops. This is likely due to the fact that DRO standard increases shadow levels, which also makes noise slightly more evident, knocking the highest quality level down a bit to 7.3 f-stops.

Selecting DRO Plus (Auto) didn't have much of a result on the dynamic range numbers either. We tried some manual DRO levels (1, 3 and 5), but all resulted in reduced Imatest scores with tone curves that had a hump in the midtones.

We normally also analyze RAW files converted with Adobe Camera Raw to better see what dynamic range the sensor is capable of, but ACR does not yet support the Sony A850 at the time of this writing (late August 2009). Using another RAW converter would yield results that aren't directly comparable, so we'll try to update this section after Adobe adds support for the A850's RAW files.

Dynamic Range, the bottom line:

As expected, the Sony A850's results were almost dead-identical to the A900's, with JPEGs placing a bit below those from most other DSLR models we've tested recently. We expect results from ACR processed RAW files will be much better, just as they were with the A900: Despite its lackluster JPEG performance, the A900's dynamic range was among the best we'd measured to date, when processing images from its RAW files. (Note that we measure dynamic range for each camera using its base ISO: This might place some cameras like the A850/900 with base ISOs of 200 at a slight disadvantage relative to those with lower base ISOs.)

To get some perspective, here's a summary of the Sony A850's dynamic range performance, and how it compares to other digital SLRs that we also have Imatest dynamic range data for. (Results are arranged in order of decreasing dynamic range at the "High" quality level.):

Dynamic Range (in f-stops) vs Image Quality (At camera's base ISO) (Blue = RAW via ACR, Yellow=Camera JPEG, Green=Current Camera)
Model	1.0 (Low)	0.5 (Medium)	0.25 (Med-High)	0.1 (High)
Nikon D3X (Adobe Camera Raw 5.3b)	--	--	11.1	9.64
Nikon D700 (Adobe Camera Raw 4.5)	12.1	11.6	10.6	9.51
Sony A900 (ISO 200) (Adobe Camera Raw 4.6b)	--	12.1	10.7	9.36
Nikon D90 (Adobe Camera Raw 4.6b)	12.1	11.8	10.7	9.27
Fujifilm S3 Pro (Adobe Camera Raw 2)	12.1	11.7	10.7	9.00
Nikon D40x (Adobe Camera Raw 4.1)	12.0	10.9	10.3	8.90
Canon 5D Mark II (Adobe Camera Raw 5.2)	--	10.8	10.0	8.89
Sony A330 (Adobe Camera Raw 5.4)	--	--	10.1	8.86
Canon EOS-1Ds Mark III (Adobe Camera Raw 4.5)	11.5	10.7	9.96	8.84
Nikon D3 (Adobe Camera Raw 4.5)	11.7	11.0	10.0	8.75
Canon EOS-1D Mark III (Adobe Camera Raw 4.5)	11.7	10.7	9.99	8.73
Pentax K20D (Adobe Camera Raw 4.5)	11.4	10.6	9.82	8.56
8.5 Stops
Nikon D300 (Adobe Camera Raw 4.3.1)	11.4	10.9	9.87	8.45
Sony A200 (Adobe Camera Raw 4.3.1)	11.6	10.4	9.82	8.43
Nikon D60 (Adobe Camera Raw 4.4.1)	11.6	10.5	9.74	8.31
Nikon D40 (ISO 200) (Adobe Camera Raw 4.1)	11.9	10.9	9.89	8.30
Canon EOS-1Ds Mark III (Camera JPEG)	10.9	10.2	9.71	8.23
Pentax K100D (Adobe Camera Raw 3.6)	11.3	10.3	9.51	8.23
Pentax K200D (Adobe Camera Raw 4.4.1)	--	10.5	9.54	8.19
Pentax K10D (Adobe Camera Raw 3.7)	10.6	10.0	9.29	8.19
Sony A100 (Adobe Camera Raw 3.4)	11.3	10.5	9.69	8.16
Canon EOS-1Ds Mark II (Adobe Camera Raw 3)	11.2	10.3	9.40	8.14
Canon EOS 50D (Adobe Camera Raw 4.6)	11.2	10.5	9.49	8.06
Nikon D40x (Camera JPEG)	10.8	10.0	9.42	8.04
Canon Rebel XSi (Camera JPEG) (ALO on by default)	11.3	10.1	9.34	8.01
8.0 Stops
Fujifilm S3 Pro (Camera JPEG)	--	9.90	9.40	7.94
Nikon D3X (Camera JPEG) Advanced D-Lighting=Low)	--	--	--	7.91
Sony A350 (Adobe Camera Raw 4.4)	11.6	10.5	9.61	7.89
Canon EOS-1D Mark III (Camera JPEG)	--	10.2	9.70	7.88
Canon Rebel XS (Adobe Camera Raw 4.5)	--	10.3	9.27	7.88
Nikon D3 (Camera JPEG)	--	--	--	7.87
Canon Digital Rebel XTi (Adobe Camera Raw 3.6)	10.8	9.88	9.18	7.84
Canon EOS 5D (Adobe Camera Raw 3)	11.0	10.4	9.21	7.83
Canon EOS 50D (Camera JPEG) (ALO Off )	--	9.64	9.17	7.83
Nikon D90 (Camera JPEG)	--	--	--	7.77
Panasonic DMC-GH1 (Adobe Camera Raw 5.4b)	9.88	--	9.30	7.76
Panasonic DMC-GH1 (Camera JPEG iExposure=Standard)	8.76	--	--	7.76
Nikon D5000 (Camera JPEG), (Advanced D-Lighting=Low )	--	--	9.28	7.75
Pentax K-7 (Adobe Camera Raw 5.4)	10.6	9.93	9.07	7.73
Canon EOS 40D (Adobe Camera Raw 4.2)	11.2	10.1	9.26	7.72
Canon Rebel XSi (Adobe Camera Raw 4.4.1)	10.6	9.95	9.10	7.68
Canon EOS 50D (Camera JPEG) (ALO STD by default)	--	--	8.90	7.68
Nikon D700 (Camera JPEG)	--	--	9.05	7.67
Canon 5D Mark II (Camera JPEG) (ALO STD)	10.6	9.68	8.98	7.66
Nikon D5000 (Camera JPEG), (Advanced D-Lighting=Off)	--	--	8.96	7.65
Canon EOS-5D (Camera JPEG)	10.2	9.68	8.82	7.65
Olympus E-3 (Adobe Camera Raw 4.3)	10.3	10.1	9.29	7.64
Canon 5D Mark II (Camera JPEG) (ALO Off)	--	9.67	8.96	7.62
Nikon D60 (Camera JPEG)	10.5	9.62	8.89	7.62
Nikon D200 (Adobe Camera Raw 3)	10.6	9.65	8.96	7.61
Nikon D80 (Adobe Camera Raw 3.6)	11.1	10.4	9.42	7.51
7.5 Stops
Olympus E-500 (Adobe Camera Raw 3)	10.7	9.97	8.90	7.46
Olympus E-510 (Adobe Camera Raw 4.1)	10.0	9.43	8.64	7.46
Pentax K10D (Camera JPEG)	--	9.49	8.88	7.44
Nikon D300 (Camera JPEG)	--	--	8.70	7.44
Olympus E-420 (Adobe Camera Raw 4.1.1)	10.0	9.61	8.65	7.44
Canon Rebel T1i (Camera JPEG) (ALO=STD by default)	11.3	10.1	9.34	7.43
Nikon D2Xs (Adobe Camera Raw 3.6)	10.6	9.90	8.93	7.42
Canon EOS 40D (Camera JPEG)	10.6	9.52	8.78	7.42
Nikon D3X (Camera JPEG) (Advanced D-Lighting=Off)	--	--	--	7.37
Nikon D50 (Camera JPEG)	10.7	9.93	8.70	7.36
Sony A850 (ISO 200) (Camera JPEG) (DRO off )	10.2	9.54	8.51	7.33
Panasonic DMC-G1 (Adobe Camera Raw 5.2)	10.7	9.78	8.70	7.32
Sony A330 (Camera JPEG) (DRO Off)	10.1	9.37	8.53	7.31
Sony A900 (ISO 200) (Camera JPEG) (DRO off )	10.2	9.75	8.49	7.31
Sony A850 (ISO 200) (Camera JPEG) (DRO Plus )	10.2	9.55	8.66	7.30
Sony A330 (Camera JPEG) (DRO Standard)	10.1	9.37	8.59	7.30
Sony A850 (ISO 200) (Camera JPEG) (DRO Standard )	10.2	9.57	8.45	7.30
Sony A200 (Camera JPEG) (DRO on by default)	10.4	9.43	8.91	7.29
Canon EOS 20D (Camera JPEG)	10.3	9.66	8.85	7.29
Canon EOS 30D (Camera JPEG)	10.3	9.50	8.57	7.29
Nikon D40 (ISO 200) (Camera JPEG)	10.4	9.80	8.89	7.28
Sony A330 (Camera JPEG) (DRO Plus)	10.1	9.38	8.54	7.28
Sony A900 (Camera JPEG) (DRO on)	10.1	9.76	8.47	7.26
Canon Rebel XS (Camera JPEG)	10.3	9.4	8.61	7.22
Olympus E-520 (Adobe Camera Raw 4.5)	11.0	9.46	8.70	7.20
Sony A350 (Camera JPEG) (DRO on by default)	10.3	9.55	8.85	7.19
Nikon D80 (Camera JPEG)	10.1	9.43	8.48	7.12
Canon Digital Rebel XT (Camera JPEG)	10.3	9.51	8.61	7.11
Nikon D200 (Camera JPEG)	--	9.07	8.36	7.11
Panasonic DMC-G1 (Camera JPEG, iExposure = Low)	--	9.29	8.50	7.09
Panasonic DMC-G1 (Camera JPEG, iExposure = Standard)	--	9.30	8.54	7.07
Olympus E-300 (Camera JPEG)	10.8	9.26	8.48	7.07
Olympus E-410 (Adobe Camera Raw 4.1)	10.2	9.40	8.24	7.05
Olympus E-500 (Camera JPEG)	10.0	9.14	8.16	7.05
Canon Digital Rebel XTi (Camera JPEG)	9.83	9.10	8.27	7.04
Canon EOS-1Ds Mark II (Camera JPEG)	10.3	9.38	8.60	7.04
Panasonic DMC-G1 (Camera JPEG, iExposure = High)	10.3	9.23	8.54	7.04
Panasonic DMC-G1 (Camera JPEG, iExposure = Off)	--	9.33	8.52	7.03
Pentax K200D (Camera JPEG)	--	9.50	8.30	7.01
7.0 Stops
Canon Digital Rebel (Camera JPEG)	10.1	9.11	8.47	6.97
Nikon D2Xs (Camera JPEG)	9.82	8.98	8.23	6.97
Panasonic DMC-L10 (Adobe Camera Raw 4.2)	10.4	9.34	8.48	6.91
Sigma DP1 (Camera JPEG)	--	8.95	8.13	6.91
Pentax *istDs (Camera JPEG)	10.2	10.0	8.87	6.90
Sony A100 (Camera JPEG)	10.2	9.24	8.39	6.89
Pentax K100D (Camera JPEG)	10.3	9.30	8.39	6.73
Pentax K20D (Camera JPEG)	10.2	9.21	8.09	6.66
Pentax K-7 (Camera JPEG)	9.59	8.87	8.03	6.54
6.5 Stops
Nikon D2x (Camera JPEG)	--	8.93	7.75	6.43
Olympus E-3 (Camera JPEG)	9.32	9.06	8.50	6.42
Panasonic DMC-L10 (Camera JPEG)	--	8.94	8.00	6.38
Olympus E-420 (Camera JPEG)	9.18	8.82	7.93	6.37
6.0 Stops
Olympus E-410 (Camera JPEG)	--	--	7.60	5.99
Nikon D70s (Camera JPEG)	9.84	8.69	7.46	5.85
Nikon D70 (Camera JPEG)	9.81	8.76	7.58	5.84
Olympus E-520 (Camera JPEG)	9.32	8.68	7.74	5.74
Olympus E-P1 (Camera JPEG Gradation = Normal)	--	8.85	7.74	5.47
< 5.0 Stops
Olympus E-510 (Camera JPEG)	7.70	7.16	5.87	3.55

The results shown in the table are interesting. One of the first things that struck me when I (Dave) initially looked at test data for a wide range of DSLRs, was that here again, purely analytical measurements don't necessarily correlate all that well with actual photographic experience. There's no question that the Fuji S3 Pro once deserved its place atop the list, as its unique "SR" technology did indeed deliver a very obvious improvement in tonal range in the highlight portion of the tonal scale relative to competing models of its day. (Amazing that it's now surpassed by even consumer-level models using today's technology.) I was also surprised to see the analytical results place the original Olympus E-300 as highly as they did, given that our sense of that camera's images was that they were in fact noisier than those of many other DSLRs that we looked at. In the other direction, I was quite surprised to see the Nikon D2x place as low on the listings as it did, given that we found that camera's shadow detail to be little short of amazing.

One thing that's going on here though, is that we test each camera at its base ISO setting, which should produce best-case noise levels. This is in fact what many photographers will be most interested in, but it does perhaps place some of the Nikons (like the D40) and the Sony A850/900 at a disadvantage, as their lowest ISO setting is 200, as compared to the ISO 100 settings available on most other models.

 

Sony A900 Resolution Chart Test Results

The chart above shows consolidated results from spatial frequency response measurements in both the horizontal and vertical axes. The "MTF 50" numbers tend to correlate best with visual perceptions of sharpness, so those are what I focus on here. The uncorrected resolution figures are 2,919 line widths per picture height in the horizontal direction (corresponding to the vertically-oriented edge), and 2,840 lines along the vertical axis (corresponding to the horizontally-oriented edge), for a combined average of 2,689 LW/PH. Correcting to a "standardized" sharpening with a one-pixel radius increased both vertical and horizontal resolution, resulting in an average of 2,941 LW/PH. While good, this is slightly below the A900's corrected score of 3,397 LW/PH.

To see what's going on, refer to the plots below, which show the actual edge profiles for both horizontal and vertical edges, in both their original and corrected forms. Here, you can see that unlike the A900, the A850 applies sharpening in the horizontal direction that Imatest considers close to ideal. While Imatest may consider this "ideal," the effect of the sharpening operator extends a fair distance away from the edge itself (close to 3 pixels, based on the graph below), which will tend to blur the finest detail. (As always, you're better off dialing down the in-camera sharpening and applying your own sharpening after the fact in Photoshop or other image-processing software.)

Imatest reports that the vertical direction (horizontal edge) is undersharpened by 6.96%, but in our view, it's actually much more what you'd want to come out of a high-resolution camera like the Sony A850. The Sony A850's images respond fairly well to tight sharpening in Photoshop, but sharpening results will be better in scenes with natural textures than those with abrupt edges (as with text on signs, building features in architectural shots etc), where you'll quickly find halos around high-contrast objects. For optimum results with post-capture sharpening, best to shoot with the in-camera sharpening turned down or (better yet) work from RAW with a good RAW converter.