Camera testing is a complicated and expensive process. To help with the high cost we share the results between a number of consumer organisations around the world. All the tests and procedures described in this article are carried out, or overseen by professional testers in consumer laboratories in Europe.
To try to keep up with the rapidly evolving digital camera market, we report on batches of cameras every couple of months. Our lab is constantly testing cameras, but each batch can take some months to complete. We only test products we can purchase from retail outlets, which means we're often a bit slower than magazines that take samples from manufacturers, but at least we know that what we're testing is representative of what you're likely to see on the shelves.
The descriptions that follow are of the Basic and High-end tests that make up the scores you'll find in our reviews of basic compact digital cameras and High-end cameras which includes SLRs.
All cameras are subject to the 'Basic' tests that follow. Cameras that fit with our criteria for 'High-end' cameras are also put through the High-end tests.
The criteria for a High-end camera are as follows:
• Manual override for focus and exposure controls
• RAW format recording (to allow you more control over the image once it’s taken)
• Manual white balance control
• Hot shoe for an external flash
Although the two tests share a number of elements, the weighting for each test (Basic and High-end) is different and it isn't possible to compare scores from one test to the other directly. However, a score of 70% or more is considered a good score and anything less than 40% is a poor score.
In each test description there's a run down of all the basic functions or features we appraise and how we decide what it will score in each category.
Before any actual testing begins, we do an exhaustive list of each cameras features and specifications. For the most part we don't rely on manufacturers specifications. For example, we take measurements that include all protruding parts when the camera is off and weigh them with their memory, batteries and neck straps etc. that come as standard. Manufactures specifications often leave these sorts of things out, but our intention is to try to give you information that relates to actual use.
All computer related tests are performed on an up to date PC with Microsoft WIN XP Professional SP3 operating system. The five monitors used are profiled and calibrated EIZO ColorEdge CG21 or EIZO S2433W and both monitors and video cards are adjusted to sRGB colour space.
In order to compare the test results to those of the previous tests using this method, in all relevant tests, especially those where a subjective score is given (for example the outdoor shots), samples from the last test are re-tested in certain aspects and used as anchor for scoring.
Unless otherwise noted, subjective tests are evaluated by five people, among them at least three interested users of cameras who are trained in assessing images.
Each panel member has their own monitor to provide best possible viewing conditions.
We hope this article helps to explain why our results sometimes differ from others who rely on an individual opinion to appraise these complex products. See also our digital camera buying guide. Check our article on compact digital camera reviews for more details.