Can we compare cheap cameras with expensive?
Is it fair to compare cameras with very different price points on the quality of the images they capture?
We took three images from our testing as examples. One of these photos was taken by a camera worth about $500, and the other two were shot with much more expensive cameras.
Which one's which?
These images have been deliberately changed to make it difficult to know which camera shot them. The EXIF data has been removed and they've been cropped a little and resized to make them easier to display. In every other way they're unaltered, and are the same as when they came from the camera as JPEGs.
Which of these images do you think is from the $500 camera?
Low light photo 1:
Low light photo 2:
Low light photo 3:
The test was carried out under the same conditions for each camera. The light level in the test target area was set to simulate a poorly lit room using halogen lamps. Each of the cameras we tested were positioned at the same distance from the test area, their zoom lenses adjusted so that the whole test area fit in the frame.
The cameras were set to fully AUTO function and checked to make sure they could focus on the scene. Focus and exposure areas were left at the camera's default setting. Image quality was on the highest possible setting and image size set to the largest option available.
In low light tests we don't make judgements based on the depth of field, because different cameras focus on different parts of the target. But we did make judgements on colour management, noise levels and exposure of the image, and we also noted how much blur there was in the slow moving card at the back of the target area.
And the winner is…
When we polled our members to see which image they thought had come from the cheaper camera, this is what they guessed:
- Low light photo 1 – 37%
- Low light photo 2 – 34%
- Low light photo 3 – 28%
With such an even split, it was clear that the cheaper camera wasn't easily identifiable, because the images weren't so different that everyone could see which had done a better job. We can now reveal that the cheaper $500 camera produced low light photo 2, and that on our test scale it actually scored better than the other two cameras for this part of the test.
This is only one of the many tests that make up our image quality score though, and for overall picture quality the cameras that produced images one and three do get better scores.
So what does that tell us?
When you remove the dollar value and brand of the camera from the equation, the viewers of our test had to rely on their understanding of what makes a good photo to judge which image they thought was the best. Our testers had the advantage of seeing the test target in both daylight and other light sources, and as a result they judged that low light photo 2 – the one taken by the cheapest camera – had the best combination of white balance, exposure and noise in this instance.
The other two cameras clearly hadn't compensated for the overly warm light, so if you were shooting with them yourself you'd have to do some work to correct that - assuming you knew how to and could be bothered. Otherwise, if you were looking solely for high quality, low light images and only wanted to think about pointing and shooting, you'd be better off going for the cheaper camera.
When we manually set the ISO to ISO3200 and did a manual white balance and exposure with the cameras that shot images one and two, the results were very different. The image taken with manual settings with the second (cheaper) camera of the bunch was much noisier and lost quite a bit of edge definition as a result.
It's not possible to judge the quality of a camera based on just one or two images. We produce dozens of images from every camera we test, in a variety of circumstances designed to replicate normal use. We also do technical tests that give an indication of how well the camera performs some more specific tasks, such as testing resolution to see how much detail a camera can record. We do this in both auto and manual modes for the cameras in our high-end tests.
If you're interested in finding out more about how basic and high-end digital cameras compare, check out the the full test.