ir-logo.gif

PMA 2000 - Digital SLR mini-shootout

AN IMAGING RESOURCE SPECIAL REPORT

Go to Imaging Resource Home | PMA 2000 Index | News | Tips | Cameras | Scanners | Newsletter


A "Mini-Shootout" of Pro Digital SLRs!

By: Dave Etchells, Imaging Resource
Additonal perspective by Rob Galbraith, see section at bottom!

As we alluded to in our special coverage of the new Fuji S1 Pro SLR digicam, we also managed to get our hands on several other pro cameras at PMA, and conducted an informal test of resolution and image noise with them. We felt this test would be particularly interesting in light of the controversy surrounding Fuji's S1 camera, which creates a ~6 million pixel file, using a ~3.2-2.5 (?) million pixel SuperCCD sensor. Fuji claims that the 6 million pixel final file size is entirely justified, due to the unique geometry of their SuperCCD sensor design, while detractors have maintained that this is mere hogwash and marketing hype, essentially saying "a pixel is a pixel", and "there ain't no free lunch." We suspect that both sides will find support for their positions in this test: We felt that the S1's resolution clearly outshown that of the pro cameras using conventional 3 megapixel sensors, but did in fact fall short of the Kodak DCS 660, which has a full 6 million pixel conventional CCD.

We also looked at noise performance from each of the cameras, at a variety of ISO ratings. Here, we felt we found a bigger difference in the S1's output, with noticeably less noise at a given ISO level (particularly very high ISOs) than the cameras with conventional CCDs. We felt this was particularly impressive for a product that's still in early prototype stages, as our experience has been that noise levels in production cameras are generally lower, due to higher levels of electronics integration, and the use of lower-power semicustom ICs.


What's Wrong With This Test?
Before going any further, or looking at any of the images we captured, we feel obligated to point out the limitations of this test. (There are a LOT of them!) Here's the short list:



OK, enough babbling, let's see some results!
The shot at right shows a reduced view of the booth we used as a test subject. We'll show a few areas in detail, to illustrate both resolution and noise performance. The signage at front center provides a useful reference for resolution, as the various strokes and lines of the script text clearly show the resolving power of the various cameras.

Interesting, but probably less useful are the cross-members of the girder structure supporting the central sign on the booth, and the black sign hanging on the far wall that you can see peeking out behind and between the girders, just above and to the left of center in the image. The cables hanging the distant sign might have made an excellent test of fine detail handling, but their distance from the main focal point of the shot means that depth of field issues prevent us from drawing any firm conclusions.

Finally, the small gray box just below center in the image, to the right of the white chemical pails at the front of the booth provides an excellent uniform gray object to study image noise on.


The "Text" resolution test.
We liked this test the best for showing resolution and detail, because the strokes of the script in the sign make it easy to discern just how much each camera is actually resolving. Here are cropped results from each camera. (To make comparisons easier, we've done an "auto levels" operation in Photoshop to brighten up the images and give them a more uniform tonal range. We felt this was legitimate, as it wouldn't disturb the resolution, only the tonal range.)


Kodak DCS 620
(2.0 megapixels)

Kodak DCS 660
(6 megapixels)

Well, no question who wins this contest: The DCS 660 is obviously sharper than any of the others: The "no free lunch" people are at least somewhat correct, in that a conventional 6 megapixel CCD definitely appears to record more detail than a 3+ megapixel SuperCCD, with Fuji's clever interpolation.

But don't write off the S1's larger file size as mere marketing hype just yet: Compare the details of the strokes and small elements of the script text between the S1 and either of the conventional 3 megapixel cameras. Look in particular at how open (or not) the centers of the letters "o", "a", "g", and even the small loop of the "e". It's hard to argue that there isn't more detail present than in either of the conventional 3 megapixel CCD images.

Fuji S1
(6 megapixel file, 3+ megapixel SuperCCD sensor)

Nikon D1
(3 megapixels)

Kodak DCS 330
(3 megapixels)



Further Manipulations (Added 2/12/00)
A number of readers wrote in, saying that the D1 image actually looked about as detailed as the S1 when it was upsampled to the same size in Photoshop(tm). This was an obvious thing we should have done in the first place, and so include the images below for your perusal. The top one is the D1 resampled to the same size as the S1 image. When we did so, we noticed that the D1 image had quite a bit of sharpening applied to it in-camera, so we applied some sharpening to the Fuji image in Photoshop, to try to get the best equivalence between the two. The results? Pretty interesting - While we still think we see a bit more detail in the S1 image, it's clearly not the blow-out you'd expect from "6" megapixels vs 2.7.


Nikon D1 image, upsampled 157%, bicubic interpolation

Fuji S1 image, unsharp masked in Photoshop(tm),
radius 0.6 pixels, 200%



The "Fine Lines" Resolution Test
This one is a bit less conclusive/obvious, thanks to the problems of depth of field and focusing distance mentioned earlier. Still, it may have some value, so we're presenting it here:


Kodak DCS 620
(2.0 megapixels)

Kodak DCS 660
(6 megapixels)

While more equivocal (as noted), this is an interesting result: The DCS 660 still wins the resolution derby, but the S1 again does pretty well. The 3 megapixel cameras don't do too badly with the fine wires from the hanging sign, and the D1's picture retains a surprising delicacy in these fine lines.

Fuji S1
(6 megapixel file, 3+ megapixel SuperCCD sensor)

Nikon D1
(3 megapixels)

Kodak DCS 330
(3 megapixels)



Resolution Conclusion
I'm sure we'll get some argument/discussion on it, but the first-order results seem pretty clear to me: The S1 clearly bests conventional 3-megapixel cameras in the realm of resolution (at least within the limits of this test), but falls short of a "true" 6 megapixel design.


Comments on this article
Click here to go to a page where you can leave your own observations/comments, etc.


Image noise!
Given that this page is already painfully large, click here to go to a page containing the cropped selections from the camera files to evaluate image noise.



Special: Perspective from Rob Galbraith

As you might guess, this article elicited a LOT of comments and emails. Among the more interesting correspondence was an email thread between myself (Dave Etchells) and Rob Galbraith. Rob is one of the premier digital photojournalists worldwide, with vast experience in that market/application. (He maintains an exceptionally well put-together web site at www.robgalbraith.com.)

Rob graciously agreed to let me share our dialog here, so the rest of our readers could benefit from his perspective. Here's a direct excerpt from our email (my comments back to Rob are further indented below...):

At 9:37 AM -0700 2/8/00, Rob Galbraith wrote:
Hi Dave,

*The DCS 620 frame appears that it could be shot with the anti-aliasing filter installed, while the DCS 660 is clearly not. This may be making the DCS 620 frame look softer and less detailed than it actually is. Even so, my experience with the DCS 620 and a good lens like Nikon's 80-200mm f2.8 is that it will produce greater clarity than that, especially with the text test. If you shot the samples with a wide angle lens, then the DCS 620 results might be more in line with what I'm accustomed to seeing.

I believe the 220 did have the antialiasing filter in place, the 660 obviously did not. You're correct about the lens, they were all taken with a 28-105mm zoom (zoom, yuck!) I'll make note of that fact. (The Nikon D1 had the same lens on it, don't recall what the lens was on the S1.)


*It's possible that the differences in resolution only might be accounting
for the differences in apparent detail, at least between the 2MP DCS 620,
2.7MP Nikon D1 and the Fuji Super CCD at it's 3.x MP resolution.

I think there's more to it than that: I see a more pronounced difference between the S1 and the D1 than 2.7 - 3.2 or whatever MP would make. At the same time, the S1 looks to have more resolution than the DCS 330, which has a 3+ megapixel sensor as well.


*Perhaps the single biggest difference might be lens and focal length. You indicate that for the sake of consistency, different lenses and focal lengths were used. I understand the need for different focal lengths, but doesn't using different lenses just about kill the validity of the test itself? I think so.

You're right: As noted, the 620 and 660 had the identical lens on them, the D1 the same lens model, just a different unit. S1 is unknown. (I can try to find out - I remember it being a zoom with "one touch" zoom control) Definitely makes a difference, I don't think it kills the test entirely though.


*Sharpness differences could exacerbated by an underpowered tripod.

You've gone to great pains to explain the uncontrolled nature of the test, and I appreciate that, and why. But given that many of your readers are going to make lasting impressions of these cameras based solely on these frames, I'm concerned that the test is not up to the usual Imaging Resource standards and does more harm than good ultimately.

I agree about the "not up to the usual standards", but hope that the readers will actually read what I've written. I may be wrong (you're in a better position to say actually, given your focus and readership), but I tend to think that people considering spending $4-10K on a camera will be fairly sophisticated about the impact of these various factors. The other half of the test (image noise) of course isn't sensitive to optical issues like these, although it may be a bit unfair (as noted) to subject the S1 to such a high level of scrutiny so early in its product life...

What I'd obviously *really* like is to get all the cameras here to do a full-blown test, with identical lenses, locked-down massive tripod, standard res targets, with/without antialiasing filters, etc, etc. The ideal would be to use a single, very sharp fixed focal-length lens (maybe my Nikkor 50mm f/1.4, stopped down to around f/8 or so?), and just move the tripod in or out to adjust framing. This would eliminate the vagaries of lens focal length, or even zoom setting on the same lens. I'd also hope to get away from this one-dimensional view of the products as well, look at them in their fullness, considering and reporting on the myriad other features that should/would go into a photographer's decision to buy one or the other.

I'm going to work on the various mfrs involved to see if I can't set something like that up later this spring

>One more thing: sharpening in the camera. I didn't see any mention if that
>was on/off, and what setting. It could also have an impact.

Obviously, one more parameter for the full test, with huge potential impact. I used the default setting on cameras with an option. As I recall, the 330 and 620 were set to "medium". (Recall Mike DeLuca explicitly making that setting on one of the Kodak cameras at least.)

This was a very ad-hoc, uncontrolled test, but I think the results were/are still useful. There were some clear differences between units that I don't think can at all be attributed to just lens differences. (Although the DCS 620 clearly deserves another shot, minus anti-aliasing filter and with a lens more suited to it.)

Thanks again very much for your input, your perspective and experience is invaluable in this area. - Let me know if it'd be OK for me to share this conversation thread via the article itself




Original Image Files
Here's a list of links to the original test files from all the cameras:


Fuji S1, ISO 400 (1380k)
Fuji S1, ISO 800 (1476k)
Fuji S1, ISO 1600 (1680k)


Kodak DCS 330, ISO 125 (1000k)
Kodak DCS 330, ISO 250 (1168k)
Kodak DCS 330, ISO 400 (1296k)


Kodak DCS 620, ISO 200 (1112k)
Kodak DCS 620, ISO 400 (1492k)
Kodak DCS 620, ISO 1600 (2680k)


Kodak DCS 660, ISO 80 (3384k)


Nikon D1, ISO 200 (1092k)
Nikon D1, ISO 400 (1212k)
Nikon D1, ISO 800 (1168k)
Nikon D1, ISO 1600 (1048k)


Copyright © 2000 by Imaging Resource, Inc.

Go to Imaging Resource Home | Back To PMA 2000 Index | News | Tips | Cameras | Scanners | Newsletter