Camera Labs

Well, time for an upgrade. But no definitive insights yet.
I'm currently doing a series of testshots the following way:
- nail testchart to the wall (my new DIN A3-sized one on photo-paper comes in handy here)
- stand back so that you're a good 10m or so away, handheld
- switch camera to Shutter priority at 1/125sec
- zoom lens to 400mm
- take two series of shoots, one with OS="on", one with OS="off"
- Take each series in sequences of three shots, put camera down between each sequence to relax your arm and shoulder, breath normal
- Take at least 9 shots within each series

After downloading the shots to the computer, I evaluate sharpness and classify it with 0-5 stars in Lightroom. 0 stars meaning totally cr*p, 5 starts meaning sharp, no blur to be seen. I'll give you some examples later.

With handholding I'm getting a distribution of images from 0 stars to 5 stars due to shake. The interesting point now is the following: If I use OS, is the distribution of the image quality of a series of shots more favourably skewed towards the sharp end in comparison to a series without using OS?
Well, you would normally expect that the OS will eliminate shake to a certain degree and thus see more sharp images in a series than without OS.

All I can tell you for now is that I get some strange statistics with this lens. In one series I definitely got a better IQ distribution with OS="off" than with OS="on"
But I'm shooting and reshooting again to make sure that this is not a one-off statistical hick-up.

But some things I can already tell for sure with this test set-up:
- When I handhold this >2kg lens+camera combo I can really see me shaking! So I need a OS/IS/VR that is working!
- sharpness variations from different apertures is not the critical point in this set-up.
- focus errors is also a non-issue here as the magnification is relatively low, due to the distance from the test-chart.
So what we'll be really seeing in this set-up is efficiency of OS/IS with this lens.

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Now I think I've done enough. I've shot an additional 300 images and evaluated the sharpness of each and every shot, ranking it from 0 to 5 stars. The base-lining is explained in great detail here. So I have now a reference sample of 139 shots @400mm focal length without OS as the basis against which we can judge the improvements from the OS at various shutter speeds ranging from 1/60 sec to 1/250 sec.

Now let me get one thing out of the way first: I mostly shot with OS set to "1" (=normal operation) but also tried 23 shots (@400mm) with the OS set to "2" (only vertical operation) plus another 17 shots at focal lengths <400mm with OS=2. At 1/125 sec and 1/60 sec this was a total desaster as I didn't get a single shot better than 2 stars rating! At 1/250 sec and shorter focal lengths there were quite a few excellent shots though.

Turning back to OS=1 for the rest of the post. Here are the results for 73 shots with the Sigma 120-400mm @400mm with OS=1 compared to the other results of 400mm lenses w/o OS (or OS=0="off").

Relative as % of total shots at a given speed:
sharpness|....5.......4.......3.......2.......1.......0.......sum
-------------|----------------------------------------------------
1/250sec.|.18%..38%..22%.....8%..10%....4%.....100% OS=0
1/250sec.|.16%..34%..47%.....0%....3%....0%.....100% OS=1 (=32 shots)

1/125sec.|...3%....3%....8%...22%..32%..32%.....100% OS=0
1/125sec.|...0%....3%....0%...28%..34%..34%.....100% OS=0 (only 120-400mm)
1/125sec.|...0%....0%....0%...41%..44%..16%.....100% OS=1 (=32 shots)

1/60sec...|...0%....4%....4%.....4%..22%..67%.....100% OS=0
1/60sec...|...0%....0%...33%...22%..22%..22%.....100% OS=1 (=9 shots)

I'll continue with the analysis of these figures tomorrow...

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Now what do the figures show us?
((If this bores you, fast forward to the next post, where I'll give you the summary))

Let's start with the effect of OS at 1/60 sec: The OS reduced the percentage of unusable shots (0 stars) from 2 in 3 (67%) to below 1 in 4 (22%) and on the other side increased the the percentage of usable shots (>= 3 stars) from a meager 1 in 12 (8%) to a respectable 1 in 3 (33%). If you calculate the average sharpness over all shots with 1/60 sec and OS="off" you get 0.6 stars, with OS="on" that raises to 1.7 stars on average. So the average sharpness increased by 1 star when using the OS.
I'll come to the comparison with non-OS shots at higher speed shortly.

At 1/125 sec the share of unusable shots (0 stars) was halved (from 32/34% down to 16%) but the average sharpness increased only slightly to 1.3 stars from 1.0 stars if you compare only the results from the 120-400mm. If you take the (mixed) results including the shots with the fixed-focal into consideration, average sharpness did not increase. This means that at 1/125 sec I can shoot the fixed-focal 400mm/5.6 w/o OS on average with less shake than the zoom with OS
You can also see this clearly in the distribution of shots with at least 3 star sharpness: w/o OS I got 3+3+8=14% shots in this category, with OS I got 0 (!) shots with this sharpness

At 1/250 sec average sharpness was increased slightly from 3.3 stars to 3.6 stars. The major improvement can be seen in the reduction of shots with less than 3 stars from 8+10+4=22% to 3%. Those shots were mostly lifted to 3 star sharpness as you can see the jump from 22% in this category to 47%. That means at 1/250 sec with OS="on" almost every shot (97%) has a sharpness of 3 stars and better. So you're on the safe side for most practical purposes.
But if you're interested in only the best sharpness (5+4 stars) you can see that the use of OS did nothing to improve the hit-rate in this category. In fact it slightly reduces the hit-rate from 56% to 50%, but that is statistically not significant.

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Summary (with regard to the efficiency of the OS at 400mm):

1. Only at the lowest shutter speed of 1/60 sec was the OS able to improve results more than a reduction in shutter speed by 1 stop to 1/125 sec would have helped. But it clearly did not reach a 2 stop improvement.

2. Unfortunately at 1/60 sec even with OS you lose about half the shots and the other half is only good for printing 5x7 or viewing images at 50% max. (which would reduce the effective resolution of your brand new DSLR to about 3 MPix )*

3. At shutter speeds of 1/125 sec or shorter, the effect of the OS is below 1 stop and can hardly be seen.

4. In the viewfinder the effect of OS is the biggest. The image stabilizes quite nicely and you can aim/compose much better. This is in stark contrast to the disappointing results you get from your images.

Phew: It's out now and all over now, at least with respect to OS . And don't ask me to reveal any OS-details with regard to the other focal lengths <400mm. I know it would be nice to do a comparison at 200mm with the Nikon 18-200mm VR

But enough for now! I'll return with some real images and other stuff to judge the optical qualities of this lens later...

-------
* Sidenote: The effect of blur from shake is unfortunately different from other reasons for unsharpness. So don't expect to get the same results from sharpening a blurred picture than you normally get for small focus errors or optical weaknesses of the lens

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Now here's something to relax after all these brain-jogging, mind-numbing statistics



Top: Nikon 18-200mm f/3.5-5.6 VR at 200mm
Middle left: Sigma 400mm f/5.6 APO macro, right Tamron 500mm f/8.0
Bottom: Sigma 120-400mm f/4.5-5.6 OS at 400mm plus lens-shade
As you can clearly see the 500mm Tamron is smaller than the lens-shade of the Sigma 120-400

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Plus some "real" shots for you to evaluate. Click on the images then choose "all sizes" in flickr to see larger versions. All images were shot with OS=1:

#1:
Shot at 400mm, f/5.6, 1/250 sec, ISO 400

#2:
Shot at 400mm, f/5.6, 1/250 sec, ISO 720

#3:
Shot at 300mm, f/8.0, 1/250 sec, ISO 400

#4:
Shot at 400mm, f/5.6, 1/250 sec, ISO 800

All images are based on the standard "vivid"-settings. No extra sharpening. All were developed using some contrast enhancements apart from #3 which was only cropped.
If you look at the larger versions of #2 and #4 you can see the "halo"-effect around (or more like "behind") highlights. This is an attribute of this lens that makes me a little nervous as it outshines/covers details around highlights. Although sometimes it is a nice effect.

A positive observation is that I did not encounter CAs or color fringeing, even at those bright highlights!

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Well, today was the day to decide whether the Sigma 120-400mm is a keeper for me or not.

I really was not so sure after publishing the results from the OS-efficiency test. Although the OS was not that great the lens still had some impressive resolution under ideal conditions and it was a zoom which made it much more flexible than my 400mm fixed-focal.

But upon further investigation of the best testshots @400mm from the shake/blur-test (=5 stars only!) I came across some really nasty sharpness drop-off when looking at the borders, even at f/8.0
Those were in the order of a drop to 2-star sharpness and much uglier than compared to the slight drop-off the fixed-focal produced at the borders. And I am not even talking about corners or the results such a lens will produce on a high-res FF/FX body (shudder!).

This effect is clearly less pronounced at shorter focal lengths, but as I want this lens for 400mm performance I have now finally decided to send it back and stay with my fixed-focal

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Hi Thomas,
Many thanks for your thorough review. Like you, I'm mainly interested in this lens for the 400mm, and it looks like it has lackluster performance at that focal length. My question for you is: What would you recommend for a 300 to 400mm lens in the sub-$1000 category, if not this lens? Do you have any experience with the 100-300mm f/4 Sigma? It lacks the OS and 400mm, but has the HSM... I'm also using a D300 and I'm looking to replace my Tamron 70-210 f/2.8 with something that has more reach for shooting birds and wildlife.
Thanks again for your work!

Hello Protohominid, and welcome to the friendly Camera Labs forum!
Nice gear you have
But as myself you're still searching for a good long-reach tele-zoom. Welcome to the club!
I've already tested a lot of alternatives as you can see here. I sent all the lenses back, because none did satisfy me enough for the money: The Tamron 200-500, the Tokina 80-400 and now the Sigma 120-400 didn't satisfy me in the IQ-department and the Nikon was lacking AF-S and an updated VR.
As of today I see only three viable alternatives:
1. Nikon 70-300 VR
2. Nikon 80-400 VR
3. Sigma 100-300/4.0
As I did not have a chance to test these lenses I can only repeat what other people (like Gordon Laing and Klaus Schroiff) found out:
1. Good VR, sharp up to 200mm, 300mm a little weak in the corners
2. Medium VR, not as sharp as #1 up to 200mm, better at 300mm but then a drop-off at 400mm wide open which turn into nice IQ at f/8
3. Better than #2 in the whole range, lacking VR but offering 1 real stop larger aperture (at least from 200mm up). Even with a 1.4x converter at 300x1.4=420mm optically as good as #2. But again: no IS here.

What would/will I do? Well I have a very good 400/5.6 and 180/2.8 so I think I will do nothing and wait for a renewed Nikon 80-400 VR. Because having a zoom at those lengths is very valuable as you cannot zoom as easily "with your feet" as with the shorter fixed-focals. And I can sell the 400/5.6 when I get the zoom (and if it is almots as good as the fixed-focal). Plus the IS/VR should give at least some insurance against losses.
But what if I had no lens in this range? I think I'd even buy the Nikon 80-400VR today to fill the gap. It is simply the best overall value of the bunch, including being the light and quite compact. But then, there is a price to pay, and who wants to invest 1600 something and get his resale-value scraped by a newer/better version of the same lens

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Here's the proof of my claim that the borders at 400mm are much worse in IQ then the center (all crops 100% for pixel-peeping).

At f/5.6
Center: Border:

Unfortunately the situation was not a little better at f/8.0.
Can you imagine how the IQ deteriorates even further on a FF/FX-body

Compare this to the equivalent shot with the Sigma 400/5.6 fixed focal at f/5.6:
Center: Corner:

Much better, huh?! And that is in the corner (!) not simply border...

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Thanks for another excellent review Thomas!

Hi, any chance of a link to the images that decided you against the lens?

Thanks

Hello thelittlechef, and welcome to the friendly Camera Labs forum!
The proof is in my last post here. You can clearly see how the border performance is much worse than the center-IQ with the zoom at 400mm. The fixed-focal 400mm holds up much better.

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Have just seen the review in the German magazin ColorFoto. At 400mm the lens gets just 56.5 out of 100 points (70pts being the average of many tested lenses!) showing a high decentering defect of 30% at the long end. Nothing said though about the effectiveness of the OS. So you have to take my review for the best approximation of what the OS is worth
Btw the Sigma 150-500 was awarded 63.5 points at 500mm...

Another interesting measure the lab applied was the evaluation of the AF: The measured the best AF-performance of the Sigma 120-400 at 400mm to achieve 93% of the values of perfect manual focus and the worst giving only 77% of the results of perfect manual focus.
This is quite interesting as it is my observation (relying heavily on AF) that you have to be very careful how good the AF really is!

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses

Well Eric,

I've not only found flaws in the OS but also in sharpness (esp. in the corners) and some halo-like effect on bright lights (probably coma). So all in all I would not recomend this zoom even if you have in-body stabilization.
If you're quite content with your 75-300 at 300 mm you can crop and enlarge another 33% from the longest focal length and be almost as happy as with the 120-400. If you find the 75-300mm lacking in sharpness at 300mm you could try finding a used Sigma 400mm/5.6 (which I own and am very fond of - see review there)

_________________
Thomas (beware: Nikon-fanboy and moderator!) My Lens Reviews, My Pictures, My Photography Blog
D800+assorted lenses