We test a wide range of smartphones every year. This is an insight into how the tests are done.
All the tests and procedures described in this article are carried out or overseen by professional testers in specialist smartphone testing laboratories in Europe. The test method is decided by representatives of consumer organisations from around the world who have many years' experience and specialised expertise in smartphone testing.
The lab is constantly testing smartphones, but each batch can take some weeks to complete. This is because we only test products that are bought from retail outlets. We might be a bit slower than reviewers who take samples from manufacturers, but this way we know that what we're testing is representative of what you're likely to see on the shelves.
We fast-track testing for models we feel you want to know about, for example the latest Google, Samsung or Apple smartphone, rather than simply including it in our next scheduled batch test.
An anechoic chamber is used for our general sound quality measurements.
The overall score, called the CHOICE Expert Rating, is made up of:
- Display: 15%
- Performance: 15%
- Battery: 10%
- Main camera: 10%
- Video quality: 10%
- Music: 5%
- Selfie camera: 5%
- Calling sound quality: 5%
- GPS: 5%
- Touchscreen: 5%
- Ease of use: 5%
- Durability: 5%
- Privacy and security 5%
Standby times are rounded to the nearest half-day, while time-in-use is rounded to the nearest minute.
Testers rate the following aspects of each phone:
- Web browser performance How easy it is to start the browser, navigate to sites, enter addresses and use favourites and bookmarks. We conduct a practical test on content downloads, and award extra points for Wi-Fi connectivity.
- Sound quality We measure a number of acoustic criteria, such as frequency response, for both incoming and outgoing calls. In addition, a user panel subjectively rates the sound quality of calls.
- Camera performance We assess the phone's digital camera capabilities to produce an image in different lighting situations. Photo images are printed and rated for resolution, colour reproduction and noise. We also look at how easy the phone is to use in picture-taking mode.
- Music quality The phone's performance as a music player. We test the phones with the headphones supplied and with a set of reference headphones. Also, how easy it is to navigate through playlists, music transfers, volume control and available memory.
- Video quality Two one-minute video recordings are made of a scene featuring people with colourful clothes, other objects with fine details and realistic artificial plants at different distances, both at rest and in motion. The smartphone is strapped to a tripod, and about a third of the video is recorded while panning and another third while zooming.
- Touchscreen quality Includes the speed of refresh and accuracy of operation.
- Battery life This is the battery life you can expect from your mobile after charging for 15 minutes using the supplied charger. If a smartphone includes a fast charger we'll test and show the resulting battery life.
- Overall battery performance Aside from battery life, overall battery performance is determined by a number of factors including, time taken to get a full charge, battery life after a 15 minute charge, and support for quick charging and wireless charging. If a mobile performs well in other areas relating to charging then the overall Battery score can be higher. This would include the ability to wireless charge or the ability to achieve 70% or more power on a 15 minute charge.
- GPS Measured for accuracy and speed.
- Synchronisation We look at how easy it is to use the phone to keep contacts and messages up to date when working with contacts and information stored on another computer or online.
- Texting Assessed by writing and receiving text and checking the suitability of the display for texting with various programs and apps such as Messenger, SMS, email and Facebook.
- Display quality Size of the display and quality of the image with regard to sharpness and colour clarity.
- Durability We simulate falls from a table or out of a shirt pocket up to 25 times; store the phones in hot, damp conditions; and test it in simulated rain.
- Privacy and security We assess the ability to easily reset the phone and wipe personal information, and rate it for providing an easy-to-read Ts & Cs document when setting up the phone for the first time. We also check to see if the automatic update process is transparent and opt-in rather than opt-out.