Autofocus Reality Part 3A: Canon Lenses

Sometimes I get to write cool, new stuff. Sometimes there’s just grunt work to be done, usually of my own making. This is one of those times.

To bring you up to date if you’ve landed here first, this is how I got here. Autofocus Reality Part 1 showed that on center-point, single-shot autofocus, standard phase detection was less accurate than manual focus and live view contrast-detection AF. No surprise there.

Autofocus Reality Part 2 showed that two of Canon’s newer lenses, the 24mm f/2.8 IS USM and 28mm f/2.8 IS USM, seemed to have much more accurate phase-detection AF when shot on 5D Mark III bodies, but not on 5D Mark II bodies. That was pretty surprising. To me at least.

That made the next step very obvious and not particularly fun.

We need to find out what other Canon lenses have the more accurate AF. If there are any, we’ll see if we can determine when this change occurred. Then we need to compare the other Canon bodies.

This post will simply compare Canon lenses (shot on the 5D Mk III) to see if we can find which ones autofocus more accurately.

Canon's EOS 5D Mark III digital SLR. Photo copyright © 2012, Imaging Resource. All rights reserved.

Assuming what we’re seeing is real, I made the assumption that recently released lenses would be more likely to autofocus accurately. (I have a background in science. Meaning, until someone besides me reproduces this finding, it’s just a blip. Perhaps there are other explanations to what I found. Maybe the Imatest charts I use are just more accurate at 24mm or something.)

So I started testing the most recent releases and then worked backwards. That, I should point out, makes a big assumption.

My method assumes that the lenses are designed and developed in the order they are released. That’s probably not 100 percent accurate, particularly given Canon’s habit of announcing things about a generation before they actually appear on the street.

Today’s Test Subjects
I set up the following lenses for testing on 5D Mk III cameras over several days, grouping them by year of release:

• 2012 – 28mm f/2.8 IS, 24mm f/2.8 IS, 40mm f/2.8

• 2011 – 300mm f/2.8L IS II (I would assume all the “II” telephotos will behave the same way. The others are too long for me to test accurately in our lab).

• 2010 – 70-200mm f/2.8L IS II, 70-300mm f/4-5.6L IS

• 2009 – 100mm f/2.8L IS Macro

• 2008 – 24mm f/1.4 Mk II

Previously I’ve been putting up graphs of the individual results. They’re pretty and all, but they take up a lot of space.

A quicker way to compare for this article will be to look at the standard deviation (SD), a measurement of how much the numbers vary. In all of the previous testing, live view AF groups with 10 test shots had SDs between 10 and 20, as did phase-detection autofocus on the Canon 24mm and 28mm f/2.8 IS lenses.

Phase-detection AF sample groups on all other lenses had SDs of well over 20, usually over 30. I added one older lens, the Canon 70-200 f/2.8L NON IS, because I would be testing zooms for the first time and wanted a baseline. This is a rather blunt tool, so it is not going to be 100 percent accurate scientifically.

I’ll add that I don’t particularly have the time to perform--nor do I expect you have the inclination to read--a scientific statistical analysis of the data involved. I consider this just a screening test: I’m just looking to see if there might be something worth investigating further.

Results
For brevity I’ll simply place the results in a table instead of showing a bunch of graphs.

The standard deviations are the average SD of a minimum of three separate runs of each lens for each type of focus. I have started off with some of the lenses from the previous AF posts and continued on to today’s results. I’ve listed the lenses by year of release, since I assumed more recent lenses were more likely to have improved AF.

Lens	LV SD	AF SD	Year
Canon 50mm f/1.4	9	34	1993
Canon 70-200 f/2.8	15	31	1995
Canon 50mm f/1.2	15	27	2007
Canon 24mm f/1.4	12	29	2008
Canon 100mm f/2.8 IS L	13	34	2009
Canon 70-200 f/2.8 IS II	14	24	2010
Canon 70-300 IS L	11	13	2010
Canon 24mm f/2.8 IS	10	12	2012
Canon 40mm f/2.8	13	16	2012

At a glance if you look down the LV SD column, which is the standard deviation of the live view contrast-detection focus runs, you see the variation is pretty consistently in the teens for all of the lenses tested.

Looking next at the standard deviation of autofocus we see there are accurate focusing lenses, which I’ve marked in bold, and other lenses. The accurate focusing lenses are very, very close to live view results.

I didn’t show it in this graph, but the 28mm f/2.8 IS is also an accurate focusing lens. There’s a bit of a surprise in that the 70-300mm L appears to focus with the accurate group, while the 70-200mm f/2.8 IS is kind of on the border. Not in the teens certainly, but it’s the best of all the other lenses.

Both of these were released in 2010, and I’m not sure what this means.

It might be that the 70-300 seems more accurate because it’s being tested at f/4 rather than f/2.8, but we’ve tested other lenses at f/4 and they weren’t this accurate. The 70-200 f/2.8 IS L may be better than the major group of other lenses but not quite as good as the newer lenses. Or maybe not. Testing three copies probably isn’t sensitive enough to make that call.

The Canon EF 300mm f/2.8L IS II USM lens. Photo provided by Canon USA Inc.

And the 300mm f/2.8L IS II
There was one last task to be done. The new Canon Supertelephoto II lenses are all recent releases: 2011 or 2012. They replace lenses designed in the 1990s, which were considered some of the best lenses Canon had made. So we have to check them.

We tested the 300mm f/2.8 IS II for a couple of reasons, but first among them was that our Imatest setup starts struggling at 400mm. I felt the 300mm should be representative of the group.

Because I felt supertelephotos were different lenses than the others we tested, we checked both the 300mm f/2.8 IS and IS II on both the Canon 5D Mk II and 5D Mk III.

As expected, neither lens proved to focus more accurately with the 5D II camera, so I won’t go into that further.

We will post graphs, like in the previous articles, of the results of a sample series (one copy of each lens on the same 5D III) as an illustration--they are pretty and all. But I repeated the tests with other copies and received the same results.

First is the 300mm f/2.8 IS. There was a bit of front-focus (the difference in average between the two groups), which I could have helped fix with microfocus adjustment, but I didn’t have the time.

The standard deviation for manual focus was 20. For phase detection AF, it was 35. I’m not certain why there was difficulty with contrast-detection AF with this lens (most other lenses were in the teens as the table above shows), but it showed about the same number for all three copies tested. Clearly, though, phase detection AF was less accurate, as was the case with all other older designs.

The 300mm f/2.8 IS Mark II, though, did behave differently on the 5D Mk III.

As expected, the 300mm f/2.8 IS II has more accurate phase-detection AF than the older version 300mm f/2.8. In the example above, it is absolutely as good as the contrast-detection live view focus. If you prefer the standard deviation numbers, the 300mm f/2.8 IS II had an average SD of 17 with live view and 17 with phase autofocus.

As an aside, in case you haven’t noticed, I had to change the scales of the resolution graph for the 300mm f/2.8 IS II. It’s an amazingly high resolution lens.

The older 300mm f/2.8 may look bad in comparison, but those numbers are very good. The original version is sharper, for example, than the 70-200 f/2.8 IS II. But the new version is amazingly good.

So What Did We Learn Today?
Well, we learned that on 5D Mk III bodies, the newest Canon lenses seem to autofocus more accurately, nearly as accurately as live view contrast-detection focus. And of course, standard phase-detection AF is much faster and more convenient than live view.

As best as I can tell, the three lenses released in 2012 (24mm, 28mm, and 40mm f/2.8), the 70-300 IS L lens released in late 2010, and the new Mk II supertelephoto lenses are more accurate.

Maybe the 70-200 f2.8 IS II is, but probably it isn’t. I’m not able to make the call since it seems to be somewhere in between the “best” group and the other lenses.

Oh, yeah: we learned the 300 f/2.8 IS II is amazingly, bitingly sharp.

Next up, obviously, will be comparing with other Canon bodies.

I think we can safely assume the 1Dx will be at least as good as the 5D Mk III. We can also assume the 5D Mk II and previous cameras with the same AF system are not. It will be interesting to see where the 1D Mk IV, 1Ds Mk III and 7D fit in this picture.

(Roger Cicala is the founder of LensRentals.com. Visit LensRentals.com to check out that cool lens you've been hankering for, and for some of the best customer service on the Internet!)