We test a wide range of smartphones every year. This is an insight into how the tests are done.
All the tests and procedures described in this article are carried out or overseen by professional testers in specialist smartphone testing laboratories in Europe. The test method is decided by representatives of consumer organisations from around the world who have many years' experience and specialised expertise in smartphone testing.
The lab is constantly testing smartphones, but each batch can take some weeks to complete. This is because we only test products that are purchased from retail outlets. We might be a bit slower than reviewers who take samples from manufacturers, but this way we know that what we're testing is representative of what you're likely to see on the shelves.
We fast-track testing for models we feel you want to know about, for example the latest Google, Samsung or Apple smartphone, rather than simply including it in our next scheduled batch test.
An anechoic chamber is used for our general sound quality measurements.
The overall score is made up of:
- Display: 15%
- Performance: 15%
- Battery life: 10%
- Main camera: 10%
- Video quality: 10%
- Music: 5%
- Selfie camera: 5%
- Calling sound quality: 5%
- GPS: 5%
- Touchscreen: 5%
- Ease of use: 5%
- Durability: 5%
- Privacy and security 5%
Standby times are rounded to the nearest half-day, while time-in-use is rounded to the nearest minute.
Testers rate the following aspects of each phone:
- Web browser performance How easy it is to start the browser, navigate to sites, enter addresses and use favourites and bookmarks. We conduct a practical test on content downloads, and award extra points for Wi-Fi connectivity.
- Sound quality We measure a number of acoustic criteria, such as frequency response, for both incoming and outgoing calls. In addition, a user panel subjectively rates the sound quality of calls.
- Camera performance The phone's digital camera capabilities to produce an image in different lighting situations. Photo images are printed and rated for resolution, colour reproduction and noise. We also look at how easy the phone is to use in picture-taking mode.
- Music quality The phone's performance as a music player. We test the phones with the headphones supplied and with a set of reference headphones. Also, how easy it is to navigate through playlists, music transfers, volume control and available memory.
- Video quality Two one-minute video recordings are made of a scene featuring people with colourful clothes, other objects with fine details and realistic artificial plants at different distances, both at rest and in motion. The smartphone is strapped to a tripod, and about a third of the video is recorded while panning and another third while zooming.
- Touchscreen quality Includes the speed of refresh and accuracy of operation.
- Battery life Time-in-use results recorded in hours, and standby times recorded in days.
- GPS Measured for accuracy and speed.
- Synchronisation How easy it is to use the phone to keep contacts and messages up to date when working with contacts and information stored on another computer or online.
- TextingAssessed by writing and receiving text and checking the suitability of the display for texting with various programs and apps such as Messenger, SMS, email and Facebook.
- Display quality Size of the display and quality of the image with regard to sharpness and colour clarity.
- Durability We simulate falls from a table or out of a shirt pocket up to 25 times; store the phones in hot, damp conditions; and carry out a rain test.
- Privacy and security Privacy and security performance includes the ability to easily reset the phone and wipe personal information as well as the provision of an easy to read T&C document supplied when setting up the phone for the first time. We also check to see if the automatic update process is transparent and opt in rather than opt out.