Sony A230 Imatest Results

We routinely use Norman Koren's excellent "Imatest" analysis program for quantitative, thoroughly objective analysis of digicam test images. I highly recommend it to our technically-oriented readers, as it's far and away the best, most comprehensive analysis program I've found to date.

My comments below are just brief observations of what we see in the Imatest results. A full discussion of all the data Imatest produces is really beyond the scope of this review: Visit the Imatest web site for a full discussion of what the program measures, how it performs its computations, and how to interpret its output.

Here's some of the results produced by Imatest for the Sony A230:

The Sony A230 showed very good color accuracy, as well as fairly accurate saturation levels. Hue accuracy was quite good, with small hue shifts occurring in the cyans, blues, reds, oranges and yellows. The shift in orange and yellow-orange was somewhat greater, though; the camera doesn't discriminate well between yellow and yellow-orange hues. Average saturation was 106.2% (6.2% oversaturated), which is more true to life than the typical entry-level SLR. (Only the Canon XS did better in this group.) Average "delta-C" color error was only 5.48 after correction for saturation, which while very good, wasn't quite as good as the Canon XS. (Delta-C is the same as the more commonly referred to delta-E, but delta-C takes into account only color differences, ignoring luminance variation.) Overall, a good color response for an SLR, especially a consumer oriented model. With very similar color to the other A-series models announced at the same time (the A330 and A380), the Sony A230 pushes strong reds, dark blues and some greens just a little, but actually undersaturates bright yellows, light greens, and cyan tones slightly: Overall saturation is closer to real life than that of most consumer SLRs we test. Mouse over the links below the illustration above to compare results with competing models. Mouse over the links below the illustration above to compare results with competing models.

 

Like most other cameras, when using the Adobe RGB color space (which provides a much wider gamut, or range of colors that can be expressed), the Sony A230 delivers more saturated colors, although the change in saturation here is very slight, relative to the sRGB results. Average saturation was 107.6%, a little higher than the sRGB results. Average saturation-corrected hue error was 4.91 "delta-C" units, a little more accurate than the sRGB results. Again, mouse over the links below the illustration above to compare results with other consumer SLRs.

 

This image shows how the Sony A230 actually rendered the colors of the MacBeth chart, compared to a numerically ideal treatment. In each color swatch, the outer perimeter shows the color as actually captured by the camera, the inner square shows the numerically correct color after correcting for the luminance of the photographed chart (as determined by a second-order curve fit to the values of the gray swatches), and the small rectangle inside the inner square shows the numerically correct color, without the luminance correction. This image shows the very good hue accuracy of the Sony A230, and also a tendency to boost the brightness (luminance) of highly saturated colors. We we've observed this brightness boost in many cameras we've tested; it seems to be a way to get brighter-looking color without completely overdoing the saturation. In the neutrals, the A230 tracks the numerically exact values fairly closely in the highlights through midtones, but increases contrast by pushing the shadows a bit deeper than the technically accurate numbers. As noted above, the overall color accuracy of the Sony A230 is better than that of most cameras we test, although its competitor the Canon Rebel XS edges it out across most of the spectrum (except the reds, which the XS pushes a good bit more).


 

There's a lot in this particular graph, a lot more than we have room to go into here. (This set of plots has also changed a few revisions back in Imatest. Some of the plots that were once shown here are now shown in other Imatest output. Since we largely focus on the Noise Spectrum plot, we only show the graphic above, which includes that plot.)

In comparing these graphs with those from competing cameras, we've found that the Noise Spectrum graph at lower right is the most important. Cameras that manage to shift their noise spectrum to higher frequencies have much finer-grained noise structures, making their noise less visually objectionable. In the graph above, this would show up as a noise spectrum curve that remained higher on the right side, representing higher noise frequencies.

The Sony A230 does a fair job of keeping much of the noise energy at high frequencies, so what little low-ISO image noise that's there is pretty fine-grained as a result. The color channels are quite a bit noisier than the luminance, however, especially the red and blue channels at lower frequencies. (This seems quite common these days.) This manifests itself as small, colored blotches in monochromatic midtones and shadows, and is only visible when pixel-peeping to a pretty extreme degree.

As we saw with previous Sony SLRs, there are spikes that occur in the Noise Spectrum plot every 0.125 or 1/8 cycles per pixel. The source of the noise spikes appears to be the sometimes slightly lighter pixels along the edges of the 8x8 pixel blocks used in the A230's JPEG encoding. Overall noise levels are so low however that the noise spikes aren't really visible at ISO 100 under normal conditions.

 

Here's the same set of noise data at ISO 1,600. Here, the Noise Spectrum graph is shifted towards the left-hand, lower-frequency side than it was at ISO 100, though it's hard to tell because of the change in scale. (Interestingly, the red and blue channel's noise spectrum is still much higher at low frequencies, something most noise reduction systems tend to minimize at higher ISOs.) This indicates a coarsening of the "grain" of the image noise patterns by quite a bit, and indeed that's what we see when inspecting the A230's ISO 1,600 images.

 

Here's the same set of noise data at ISO 3,200. The Noise Spectrum graph is similar, though the A230's noise reduction has further reduced the chroma noise to more closely follow luminance noise, but it's still higher, especially at low frequencies. Notice also that the noise levels are quite a bit higher. (Particularly evident in the pixel noise plot in the upper right corner of the illustration, and in the midtones and below in the graph in the lower left hand corner.)

 

This chart compares the Sony A230's noise performance over a range of ISOs against that of other current, entry-level consumer models. While we continue to show noise plots of this sort because readers ask for them, we each time point out that the noise magnitude is only a small part of the story, the grain pattern being much more important. Here, we can see that the Sony A230's luminance noise magnitude follows that of the A330 very closely, which is no surprise, since they use the same sensor and image processor. The Sony A230's noise starts out average relative to most of its "competition", and remains about average compared to the others up to 800. (The Nikon D3000 being the exception, as its High ISO Noise Reduction defaults to Off or Minimal, depending on the ISO.) At ISO 1,600, the A230's graph takes a deep dip, down to below the ISO 400 level, indicating strong noise reduction has kicked in (at the inevitable cost to fine detail). At ISO 3,200, the A230's graph is still much lower than the competition. This may seem like good performance, but examining high ISO images from the A230 reveals that a lot of fine detail has been blurred away by noise reduction. Do keep in mind these measurements are at default settings, so the shape or position of the curve is influenced by the settings you choose to use. Turning off the high ISO reduction (the Sony A230 has only Off and On options) would have certainly raised the values in the graph above, but would also result in more detail retention. Note that the Sony A230 forces high ISO noise reduction off when in continuous mode, and forces it on in some exposure modes, such as Auto or Scene Selection.

 

Sony A230 Dynamic Range Analysis

A key parameter in a digital camera is its Dynamic Range, the range of brightness that can be faithfully recorded. At the upper end of the tonal scale, dynamic range is dictated by the point at which the RGB data "saturates" at values of 255, 255, 255. At the lower end of the tonal scale, dynamic range is determined by the point at which there ceases to be any useful difference between adjacent tonal steps. Note the use of the qualifier "useful" in there: While it's tempting to evaluate dynamic range as the maximum number of tonal steps that can be discerned at all, that measure of dynamic range has very little relevance to real-world photography. What we care about as photographers is how much detail we can pull out of the shadows before image noise becomes too objectionable. This, of course, is a very subjective matter, and will vary with the application and even the subject matter in question. (Noise will be much more visible in subjects with large areas of flat tints and subtle shading than it would in subjects with strong, highly contrasting surface texture.)

What makes most sense then, is to specify useful dynamic range in terms of the point at which image noise reaches some agreed-upon threshold. To this end, Imatest computes a number of different dynamic range measurements, based on a variety of image noise thresholds. The noise thresholds are specified in terms of f-stops of equivalent luminance variation in the final image file, and dynamic range is computed for noise thresholds of 1.0 (low image quality), 0.5 (medium image quality), 0.25 (medium-high image quality) and 0.1 (high image quality). For most photographers and most applications, the noise thresholds of 0.5 and 0.25 f-stops are probably the most relevant to the production of acceptable-quality finished images, but many noise-sensitive shooters will insist on the 0.1 f-stop limit for their most critical work.

The image below shows the test results from Imatest for an in-camera JPEG file from the Sony A230 with a nominally-exposed density step target (Stouffer 4110), and the A230's contrast and other exposure-related settings at their default values. Note that the A230 selects different levels of DRO depending on the exposure mode, so we've included plots from all three settings (Off, Standard and Advanced or Plus) here.

Here, we can see that the tone curves show pretty good gradation in the highlights and the shadows trail off nicely for all three settings. (The order above, from top to bottom is Off, Standard and Advanced/Plus). As we almost always find with dynamic-range enhancement modes on consumer cameras, the total dynamic range actually remains constant (the camera just flattens the highlight end of the tone curve a little) and is pretty good, with a range of between 10.4 and 10.7 f-stops, while the scores at the highest level (7.26 to 7.30 f-stops) are about average for an SLR with an APS-C sensor.

 

Processing the Sony A230's RAW (.ARW) files through Adobe Camera Raw (ACR) version 5.5 increased dynamic range by more than a 1.5 f-stops at the highest quality level (8.95 f-stops), compared to the in-camera JPEG (7.30 f-stops) while total dynamic range reported increased from 10.7 to 12.1 f-stops. These results were obtained by using ACR's automatic settings; then manually tweaking from there. (The auto settings by themselves gave a value of 8.51 stops of DR at the highest quality level.) It's worth noting here that ACR's default noise reduction settings reduced overall noise somewhat (see the plot in the lower left-hand corner) relative to the levels in the in-camera JPEG, which would tend to boost the dynamic range numbers for the High Quality threshold. Also, the extreme highlight recovery being performed by ACR here would likely produce color errors in strong highlights of natural subjects.

Dynamic Range, the bottom line:

The net result was that the Sony A230's JPEGs scored about average in Imatest's dynamic range analysis when compared against other current APS-C sensor models. Sony A230 RAW files converted with ACR did much better, scoring in the top 10 cameras we've tested to date.

To get some perspective, here's a summary of the Sony A230's dynamic range performance, and how it compares to other digital SLRs that we also have Imatest dynamic range data for. (Results are arranged in order of decreasing dynamic range at the "High" quality level.):

Dynamic Range (in f-stops) vs Image Quality (At camera's base ISO) (Blue = RAW via ACR, Yellow=Camera JPEG, Green=Current Camera)
Model	1.0 (Low)	0.5 (Medium)	0.25 (Med-High)	0.1 (High)
Nikon D3X (Adobe Camera Raw 5.3b)	--	--	11.1	9.64
Nikon D700 (Adobe Camera Raw 4.5)	12.1	11.6	10.6	9.51
Sony A900 (Adobe Camera Raw 4.6b)	--	12.1	10.7	9.36
Nikon D90 (Adobe Camera Raw 4.6b)	12.1	11.8	10.7	9.27
Fujifilm S3 Pro (Adobe Camera Raw 2)	12.1	11.7	10.7	9.00
Sony A230 (Adobe Camera Raw 5.5)	11.7	11.1	10.1	8.95
Nikon D40x (Adobe Camera Raw 4.1)	12.0	10.9	10.3	8.90
Canon 5D Mark II (Adobe Camera Raw 5.2)	--	10.8	10.0	8.89
Sony A330 (Adobe Camera Raw 5.4)	--	--	10.1	8.86
Canon EOS-1Ds Mark III (Adobe Camera Raw 4.5)	11.5	10.7	9.96	8.84
Nikon D3 (Adobe Camera Raw 4.5)	11.7	11.0	10.0	8.75
Canon EOS-1D Mark III (Adobe Camera Raw 4.5)	11.7	10.7	9.99	8.73
Nikon D3000 (Adobe Camera Raw 5.5)	--	10.8	10.1	8.61
Pentax K20D (Adobe Camera Raw 4.5)	11.4	10.6	9.82	8.56
8.5 Stops
Nikon D300 (Adobe Camera Raw 4.3.1)	11.4	10.9	9.87	8.45
Sony A200 (Adobe Camera Raw 4.3.1)	11.6	10.4	9.82	8.43
Nikon D60 (Adobe Camera Raw 4.4.1)	11.6	10.5	9.74	8.31
Nikon D40 (Adobe Camera Raw 4.1)	11.9	10.9	9.89	8.30
Canon EOS-1Ds Mark III (Camera JPEG)	10.9	10.2	9.71	8.23
Pentax K100D (Adobe Camera Raw 3.6)	11.3	10.3	9.51	8.23
Pentax K200D (Adobe Camera Raw 4.4.1)	--	10.5	9.54	8.19
Pentax K10D (Adobe Camera Raw 3.7)	10.6	10.0	9.29	8.19
Sony A100 (Adobe Camera Raw 3.4)	11.3	10.5	9.69	8.16
Canon EOS-1Ds Mark II (Adobe Camera Raw 3)	11.2	10.3	9.40	8.14
Canon EOS 50D (Adobe Camera Raw 4.6)	11.2	10.5	9.49	8.06
Nikon D40x (Camera JPEG)	10.8	10.0	9.42	8.04
Olympus E-P1 (ISO 200, Adobe Camera Raw 5.5)	11.5	10.4	9.26	8.04
Canon Rebel XSi (Camera JPEG) (ALO on by default)	11.3	10.1	9.34	8.01
8.0 Stops
Fujifilm S3 Pro (Camera JPEG)	--	9.90	9.40	7.94
Nikon D3X (Camera JPEG) Advanced D-Lighting=Low)	--	--	--	7.91
Sony A350 (Adobe Camera Raw 4.4)	11.6	10.5	9.61	7.89
Canon EOS-1D Mark III (Camera JPEG)	--	10.2	9.70	7.88
Canon Rebel XS (Adobe Camera Raw 4.5)	--	10.3	9.27	7.88
Nikon D3 (Camera JPEG)	--	--	--	7.87
Canon Digital Rebel XTi (Adobe Camera Raw 3.6)	10.8	9.88	9.18	7.84
Canon EOS 5D (Adobe Camera Raw 3)	11.0	10.4	9.21	7.83
Canon EOS 50D (Camera JPEG) (ALO Off )	--	9.64	9.17	7.83
Nikon D90 (Camera JPEG)	--	--	--	7.77
Panasonic DMC-GH1 (Adobe Camera Raw 5.4b)	9.88	--	9.30	7.76
Panasonic DMC-GH1 (Camera JPEG iExposure=Standard)	8.76	--	--	7.76
Nikon D5000 (Camera JPEG), (Advanced D-Lighting=Low )	--	--	9.28	7.75
Canon Rebel T1i (Adobe Camera Raw 5.4b)	11.2	10.2	9.16	7.73
Canon EOS 40D (Adobe Camera Raw 4.2)	11.2	10.1	9.26	7.72
Canon Rebel XSi (Adobe Camera Raw 4.4.1)	10.6	9.95	9.10	7.68
Canon EOS 50D (Camera JPEG) (ALO STD by default)	--	--	8.90	7.68
Nikon D700 (Camera JPEG)	--	--	9.05	7.67
Canon 5D Mark II (Camera JPEG) (ALO STD)	10.6	9.68	8.98	7.66
Nikon D5000 (Camera JPEG), (Advanced D-Lighting=Off)	--	--	8.96	7.65
Canon EOS-5D (Camera JPEG)	10.2	9.68	8.82	7.65
Olympus E-3 (Adobe Camera Raw 4.3)	10.3	10.1	9.29	7.64
Canon 5D Mark II (Camera JPEG) (ALO Off)	--	9.67	8.96	7.62
Nikon D60 (Camera JPEG)	10.5	9.62	8.89	7.62
Nikon D200 (Adobe Camera Raw 3)	10.6	9.65	8.96	7.61
Nikon D80 (Adobe Camera Raw 3.6)	11.1	10.4	9.42	7.51
7.5 Stops
Olympus E-500 (Adobe Camera Raw 3)	10.7	9.97	8.90	7.46
Olympus E-510 (Adobe Camera Raw 4.1)	10.0	9.43	8.64	7.46
Pentax K10D (Camera JPEG)	--	9.49	8.88	7.44
Nikon D300 (Camera JPEG)	--	--	8.70	7.44
Olympus E-420 (Adobe Camera Raw 4.1.1)	10.0	9.61	8.65	7.44
Canon Rebel T1i (Camera JPEG) (ALO=STD by default)	11.3	10.1	9.34	7.43
Nikon D2Xs (Adobe Camera Raw 3.6)	10.6	9.90	8.93	7.42
Canon EOS 40D (Camera JPEG)	10.6	9.52	8.78	7.42
Nikon D3X (Camera JPEG) (Advanced D-Lighting=Off)	--	--	--	7.37
Nikon D50 (Camera JPEG)	10.7	9.93	8.70	7.36
Panasonic DMC-G1 (Adobe Camera Raw 5.2)	10.7	9.78	8.70	7.32
Nikon D3000 (Camera JPEG)	10.2	9.64	8.69	7.31
Sony A330 (Camera JPEG) (DRO Off)	10.1	9.37	8.53	7.31
Sony A900 (Camera JPEG) (DRO off by default )	10.2	9.75	8.49	7.31
Sony A230 (Camera JPEG) (DRO Plus)	10.2	9.36	8.53	7.30
Sony A200 (Camera JPEG) (DRO on by default)	10.4	9.43	8.91	7.29
Canon EOS 20D (Camera JPEG)	10.3	9.66	8.85	7.29
Canon EOS 30D (Camera JPEG)	10.3	9.50	8.57	7.29
Sony A230 (Camera JPEG) (DRO Off)	10.1	9.40	8.52	7.29
Nikon D40 (Camera JPEG)	10.4	9.80	8.89	7.28
Sony A230 (Camera JPEG) (DRO Standard)	10.1	9.51	8.51	7.26
Sony A900 (Camera JPEG) (DRO on)	10.1	9.76	8.47	7.26
Canon Rebel XS (Camera JPEG)	10.3	9.4	8.61	7.22
Olympus E-520 (Adobe Camera Raw 4.5)	11.0	9.46	8.70	7.20
Sony A350 (Camera JPEG) (DRO on by default)	10.3	9.55	8.85	7.19
Panasonic DMC-GF1 (Adobe Camera Raw 5.5)	10.2	9.62	8.62	7.16
Nikon D80 (Camera JPEG)	10.1	9.43	8.48	7.12
Canon Digital Rebel XT (Camera JPEG)	10.3	9.51	8.61	7.11
Nikon D200 (Camera JPEG)	--	9.07	8.36	7.11
Panasonic DMC-G1 (Camera JPEG, iExposure = Low)	--	9.29	8.50	7.09
Panasonic DMC-G1 (Camera JPEG, iExposure = Standard)	--	9.30	8.54	7.07
Olympus E-300 (Camera JPEG)	10.8	9.26	8.48	7.07
Olympus E-410 (Adobe Camera Raw 4.1)	10.2	9.40	8.24	7.05
Olympus E-500 (Camera JPEG)	10.0	9.14	8.16	7.05
Canon Digital Rebel XTi (Camera JPEG)	9.83	9.10	8.27	7.04
Canon EOS-1Ds Mark II (Camera JPEG)	10.3	9.38	8.60	7.04
Panasonic DMC-G1 (Camera JPEG, iExposure = High)	10.3	9.23	8.54	7.04
Panasonic DMC-G1 (Camera JPEG, iExposure = Off)	--	9.33	8.52	7.03
Pentax K200D (Camera JPEG)	--	9.50	8.30	7.01
7.0 Stops
Panasonic DMC-GF1 (Camera JPEG, iExposure = Off)	--	9.33	8.44	6.99
Canon Digital Rebel (Camera JPEG)	10.1	9.11	8.47	6.97
Nikon D2Xs (Camera JPEG)	9.82	8.98	8.23	6.97
Panasonic DMC-L10 (Adobe Camera Raw 4.2)	10.4	9.34	8.48	6.91
Sigma DP1 (Camera JPEG)	--	8.95	8.13	6.91
Pentax *istDs (Camera JPEG)	10.2	10.0	8.87	6.90
Sony A100 (Camera JPEG)	10.2	9.24	8.39	6.89
Pentax K100D (Camera JPEG)	10.3	9.30	8.39	6.73
Pentax K20D (Camera JPEG)	10.2	9.21	8.09	6.66
6.5 Stops
Nikon D2x (Camera JPEG)	--	8.93	7.75	6.43
Olympus E-3 (Camera JPEG)	9.32	9.06	8.50	6.42
Panasonic DMC-L10 (Camera JPEG)	--	8.94	8.00	6.38
Olympus E-420 (Camera JPEG)	9.18	8.82	7.93	6.37
6.0 Stops
Olympus E-410 (Camera JPEG)	--	--	7.60	5.99
Nikon D70s (Camera JPEG)	9.84	8.69	7.46	5.85
Nikon D70 (Camera JPEG)	9.81	8.76	7.58	5.84
Olympus E-520 (Camera JPEG)	9.32	8.68	7.74	5.74
Olympus E-P1 (Camera JPEG, ISO 200, Gradation = Normal)	--	8.85	7.74	5.47

Comparing the Sony A230 to the rest of the market, its dynamic range in camera-produced JPEGs is pretty much in line with competing models. Adobe Camera Raw managed to boost the DR pretty dramatically, but as noted above, the extreme processing it's applying in the highlights would almost certainly result in color errors in strong highlights in highly-colored natural subjects; the actual dynamic range of the sensor is likely much closer to what we see in the JPEGs.

 

Sony A230 Resolution Chart Test Results

The chart above shows consolidated results from spatial frequency response measurements in both the horizontal and vertical axes. The "MTF 50" numbers tend to correlate best with visual perceptions of sharpness, so those are what we focus on here. The uncorrected resolution figures are 1,861 line widths per picture height in the horizontal direction (corresponding to the vertically-oriented edge), and 1,777 lines along the vertical axis (corresponding to the horizontally-oriented edge), for a combined average of 1,819 LW/PH. Correcting to a "standardized" sharpening with a one-pixel radius only changed the resolution score by only a tiny bit, for an average of 1,801 LW/PH, which is nearly identical to the Sony A330's results. The corrected numbers put the A230 pretty much on par with its 10-megapixel competitors.

To see what's going on, refer to the plots below, which show the actual edge profiles for both horizontal and vertical edges, in both their original and corrected forms. Here, you can see that an almost ideal amount of in-camera sharpening is applied in the horizontal direction (oversharpened by only 1.4 %), while in the vertical direction, Imatest reports only slight undersharpening of 0.47%. This is why standardized sharpening didn't change the MTF 50 numbers by much. Despite the reported slight undersharpening, though, there are still sharpening artifacts present (the "bumps" in the curves below): For the best results with images that will be sharpened in an image-processing program like Photoshop, you'll want to dial down the in-camera sharpening somewhat to eliminate these. (Or, just work from RAW files.)

These are very good results. Still, (as is almost always the case), you'll extract the most detail from the Sony A230's images by careful processing of its ARW RAW files.