How we test cameras and lenses

To help you make the right choice when it comes to buying new equipment, we use a wide range of tests designed to push photographic equipment to its limits. Our reviews are designed so it's easy to delve straight into the results or skip them entirely if you'd prefer the final verdict. If you belong to the former, here's more technical information on how we do the tests.

All the tests below are carried out with each camera mounted on a Manfrotto tripod with a model 410 geared head for precision adjustments. The shutter is then fired using the self-timer function to eliminate the possibility of camera shake.


Outdoor resolution and real-life noise

Studio-based tests performed with charts at close range under artificial light have their place in all camera and lens reviews, but there's no substitute for comparing shots taken outdoors under natural light with the lens focused to infinity. Outdoor results are crucial for evaluating cameras and lenses but we believe they're of limited use when viewed in isolation. This is why at Cameralabs we always take a number of identical outdoor shots with both the product on test and one or more of its key rivals within moments of each other. This allows us to not only show how the camera or lens on test performs under common conditions, but crucially how it compares to a rival product.

Since outdoor conditions constantly vary, we have to take new rival comparison shots for every new camera or lens we test. Sourcing a relevant rival product and photographing exactly the same outdoor composition for every new camera or lens on test requires considerable effort, but we believe it's essential for a complete and fair evaluation.

In our latest reviews at Cameralabs, we publish results from two main tests performed under real-life conditions. The first tests for resolution and sharpness across the frame, while the second tests for noise levels and detail at different ISO and Noise Reduction settings. Both tests are performed with the camera on review and one or more of its key rivals within moments of each other to give you a unique indication of how they compare in real life.

We additionally include a number of Gallery shots for every product, showcasing its features and revealing any compromised aspects. We always include 100% crops for each sample and describe not just the settings used, but how the camera or lens performed during the shot. We also make an effort to show the products used in a variety of environments as they would in real life. The photos are taken in and around the Queenstown area in New Zealand's South Island, with the main outdoor comparison shot (seen above) taken from the top of the Skyline Gondola.

Additional real-life tests

Image stabilisation has become a common feature and to test it we again head to the outdoors and photograph a distant scene with the lens focused to infinity. The camera is then put into Shutter Priority mode and a sequence of shots taken with and without stabilisation. The test is repeated several times.

Anti-dust capabilities are now present on almost every DSLR, and while impossible to scientifically and consistently measure their effectiveness, we believe some comment is better than nothing. Our anti-dust methodology consists of leaving the DSLR body face-up without a lens, inside and outside for ten minutes each; we can’t know how much dust entered the body during this time, nor even how much was present to start with, but we know such a process would result in dust being a problem for most models. We then power the camera on and off twice to activate the cleaning process. 

We then take a series of photos at every aperture setting of a plain white surface at close range with a lens at an equivalent of around 70mm and manually focused to infinity. Dust marks normally become most apparent at the smallest apertures (eg f16 and f22), but it’s also important to test at more common apertures. If dust marks are hard to see on the original images, we apply Auto Levels in Photoshop to reveal them. Dust results are shown in the Features pages of our reviews.

Finally to test continuous autofocus and tracking performance on reviews of higher-end models, we photograph on-coming vehicles approaching at 50kph along a straight stretch of road. You can find details of the AF performance in the Features pages of our reviews.

Studio resolution

To measure resolving power of a camera or lens we photograph an enhanced version of the industry standard ISO 12233 test chart under artificial daylight conditions. In the studio resolution section of our results pages we feature 100% crops of the converging vertical and horizontal line sections which can be used to determine horizontal and vertical resolution respectively. The scale on the chart represents 100 lines per picture height, lpph, so a reading of 18 corresponds to a resolution of 1800 lpph. The enhanced chart allows measurements up to 4000 lpph to be made. A typical 10 Megapixel digital SLR should resolve between 2000 and 2200 lpph depending on the lens.

Unless otherwise stated, the published results are taken using standard or bundled lenses and with the camera's best-quality JPEG mode. While we additionally perform the tests with RAW recording modes and a variety of other lenses, we've chosen to present the results from bundled lenses and JPEG recording modes as we believe this is how most cameras will be used.

All lenses are tested using specific digital SLR bodies. While this approach of course doesn't isolate the lens, it does demonstrate how it will perform with specific bodies under real-life conditions. As such we acknowledge we are testing a complete system including body sensor and image processing. We believe this is ultimately more useful than presenting pure figures on lens performance alone before trying to apply them to a body you own or are considering buying in the future. For example, it allows you to determine whether a premium lens really can be justified on a budget body - a crucial decision all digital SLR owners consider at some point. As new bodies are released, we will of course update and supplement the tests for key lenses.