Every year we buy and test as many TVs as we can fit in our lab; our latest TV test report has more than 100 models, although the companies constantly refresh their range so some of the TVs we test may be hard to find after a few months on the shelves. We test in batches of seven to nine at a time, because as they get bigger they take up more and more space. The tests take around three weeks and are designed to give each TV a fair go at providing a good picture, but our primary focus is to get as close as we can to real world experiences. We want to watch what you watch, not just what will make the television display look good in-store.
If you want to see what brands have been performing best in the last 12 months of TV testing, check out our Best Brand - TV report.
Changes to TV testing
Until recently most of the free-to-air channels broadcast their highest rating shows on their primary channel in standard definition (SD). This meant much of what you were watching, including major sporting events, was in SD. Therefore SD performance was a large part of the overall rating when we tested TVs.
CHOICE has found many TVs with very high resolution struggle to up-scale SD sources to their native resolution, leading to low scores in our testing for SD. Although many of the TVs deliver better than OK pictures with an HD source, poor results for SD performance means their overall score is quite low. Thankfully most of the Australian networks have finally moved over to HD broadcasting for their primary channels. This move, as well as the increase in high definition (HD) and even some 4K content available on online subscription services such as Netflix, Stan and Presto, has forced an adjustment to the CHOICE TV test weightings to better reflect the type of TV you are more likely to watch. While commenting on 4K content, we started to assess 4K content but will not be including it in our overall scoring until later in the year as we feel 4K will become more relevant as more content becomes available.
We've changed the weightings for our performance tests to reflect the increase in available HD content. Online streaming routinely delivers HD-quality video and most of the free-to-air stations now broadcast HD content on their main channel.
There has been a lot of talk about 'Smart TVs' and we have increased our testing in this area in preparation for the time when it becomes an intrinsic part of using the TV in the home. The initial testing has found that this area is still generally poorly implemented by TV manufacturers. Our testers have found the experience to be better when using the same smartphone brand as the TV, but this in itself is not a good sign as the experience should be as good with any mobile.
We've also reassessed our reference TV to better reflect consumer assessments on what is considered an 'average' TV. This means there'll be no changes in the rankings of any of the TVs we've tested, only the overall scores of all the TVs.
We will continue to accept one-off loans for large TVs over 70-inches as we could not not justify spending over $15,000 of your money (CHOICE is funded only by our members) for a TV and so will accept the occasional review model of 70-inch plus models (which we send back to the company following the review).
Our expert testers
Our technical staff are some of the most experienced TV testers in the country with 30 years' experience testing televisions between them. They've seen the progression from bulky cathode ray tube TVs, through the very expensive plasma sets, on to LCD and now OLEDs. In that time they've sat on Australian Standards Committees to make sure the television you buy is safe and have had a major influence in getting MEPS (mandatory energy performance standards) implemented so the TV you buy today uses significantly less energy than the sets of old.
How we choose what we test
From April to July every year manufacturers release many more models than we, or anyone else, can test. We liaise with the manufacturers to get information on what will be released and plan our testing based on that information along with market information from companies like GFK which provides insights into what sold well in the previous year.
Once the models are officially released we select what we'll buy based on the popularity of price points and sizes from the previous years, as well as what will be promoted in the stores. Our buyers help keep us informed on what is being promoted in the major retailers so we can buy the sets that you'll see when you're out shopping.
Occasionally we'll include a brand that's outside the mainstream to check that we're not missing a bargain, or something better than the main brands can deliver.
If you've got your eye on a model that's not in our test, make sure you read our guide to testing a TV in-store.
How we test
The most basic and important part of our testing is that it's a comparison of the relative merits of each television against the others in the test. To this end we have a number of reference TVs that we go back to, to check our scoring is consistent. This way our scoring is based on a direct comparison to a known quality level for each batch.
The final scores are based on hundreds of individual scores which our testers produce as they work their way through the test method.
Each TV is given some time to warm up and the tester then uses the TV's settings to get its colour as close as possible to the industry colour balance standard (D65). We do this because we know the broadcasters and makers of movies use this standard when producing TV shows and movies. Many TVs are set to a much cooler temperature in the factory because bluer pictures tend to look brighter, sharper and more contrasty.
We test under the following broad categories:
We gather a lot of information about the television including its weight, measuring the available screen area, connections (Wi-fi, Bluetooth etc.) and any other materials supplied with the TV such as 3D glasses and cables etc.
This is one of the major components of the TV test and includes both a technical and subjective assessment of how well the TV can handle a number of different source materials.
The TVs are placed side by side and connected to a sophisticated delivery system that provides them all with the same source material at the same time. This makes it possible for testers to go seamlessly from one TV to another to compare output. At least one reference TV is included in the array and the panel uses it as a basis for determining scores.
Subjective tests are done by a panel of at least three experienced viewers. The TV's brand and model are obscured, so the panel can make assessments based on what they see, not what they might expect to see. For each test the panel assesses high definition sources from a Blu-ray disc, a recording of both a TV show and sporting footage (including long shots of players on a field moving quickly) broadcast at 1080i. This is repeated with standard definition footage.
The panel is looking for colours that have good saturation and that don't look false or have a particular overall cast. We assess the sharpness and noise in the picture and look for obvious faults such as plastic looking skin tones, poor black levels, backlight bleeding, noise and pixilation around moving objects.
We also assess some technical footage, designed to test the TV's ability to deal with noise in the signal at both high and standard definition.
Testing on 4K content has been introduced at the beginning of 2017 and will continue until there is a significant number of TV models to report on within the overall score.
Objective tests include test patterns and footage which checks the TVs for overscan (how much picture is not being shown), jaggies (diagonal lines that appear as jagged or broken lines), as well as judder and blur at a number of different frame rates.
Ease of use
After picture quality, this is the second most important part of the test. We cover a lot of the TV's functionality, but it's impossible to check everything a smartTV can do, so we check the things we've found you'll probably use most.
If you've ever lost your remote control you'll know how essential it is for using your TV. Some come with two remotes and in this case we test them both, but the score is based on the best one.
The tester checks the size and spacing of the buttons, dials or touch pads to make sure they're easy to reach, logically grouped and not too close together. Labels should be big enough to read and of sufficient contrast to be easy to read in low light. The remote's overall size and shape is assessed for how well it fits in the hand and how easy it is to use with one hand.
On-screen user interface
The on-screen user interface should be easy to read, easy to follow, tell the user what is happening (e.g. tells the user to wait while TV is scanning for DTV broadcasts), and show clear prompts, which button to press, or what to do next.
The tester assesses the on-screen user interface for having the following attributes:
- A simple and well organised means of selecting the main functions or options.
- Easy to read characters.
- Intuitive descriptor words, phrases or symbols.
- Staying on the screen long enough.
- Having a guide as to which keys to use.
- Easy to organise frequently viewed channels, either by using the Favourites function or Channel skip/erase.
- Doesn't obstruct a significant part of the viewing area when adjusting the picture controls.
The tester measures the time it takes the TV to complete an automatic tuning scan. The Closed Captioning feature is assessed paying attention to the readability of the text, whether the user can adjust the transparency, and whether the closed captions state is retained when changing channels.
The tester assesses the browser for its ease of scrolling, entering data and whether it can play Flash video format. We check for some common apps (YouTube, Netflix etc.) to see if they're prominently displayed or have to be searched for. TV catch-up apps are also checked and points are lost if they're not all available. The layout of the screens that get you to and from apps are also assessed for their logic and accessibility.
USB recording functionality
The tester records what options and controls are available. In particular we look for:
- how recordings can be scheduled (via EPG or a separate scheduler)
- correct times and naming of programs that are recorded
- whether recordings can be re-named
- whether recordings begin and end on time
- the comprehensiveness and usefulness of the manual (if there is one)
- how easy it is to navigate in the recording menus.
Recordings are made using the USB connection and scheduled at a future time to make sure the TV can cope if it's left in standby mode.
The tester plays a number of high quality voice and music recordings on a DVD player connected to the TV and compares it a reference system. The assessment is made at a volume that the tester feels is sufficient for listening in a normal room and also at the TVs maximum volume (assuming the latter is louder).
The tester is listening for:
- clear voices
- sufficient frequency response to provide some drama
- rattles, humming or obvious distortion.
Test criteria explained
Prices are recommended retail, as supplied by manufacturers – you should be able to better this price in any store.
Overall score is made up of: high definition (HD) picture quality 35%, standard definition (SD) picture quality 20%, user interface score 10%, remote control score 10%, smartTV score 10%, EPG score 5%, sound quality score 5% and energy use score 5%. DVR performance is scored, but not included in the overall score because not all TVs have this feature.
USB inputs = the number of USB2 and USB3 inputs. The latter will be more important as more 4K devices come to market.
High definition includes footage from a Blu-ray movie, technical assessment of noise and free-to-air broadcast sports footage.
Standard definition includes footage from DVD movies, technical assessment of noise levels, jaggies etc. and free-to-air broadcast sports footage.
Our test lab
The CHOICE TV lab is a purpose-built testing facility that allows us to directly compare up to a dozen TVs at a time. It may not look like your lounge room, but its controlled lighting, temperature, humidity and neutral grey walls combine to make it possible to give every TV we test a good chance of delivering its best performance.
We use a state of the art signal delivery system that provides each TV with the same input at the same time. This makes it possible for our testers and viewing panels to directly compare the TVs with each other. It also makes it possible to quickly switch from one input type to another so we can assess the differences between them.
Our surveys of your viewing habits tell us that the majority of people watch TV in subdued halogen lighting. Our test lab has both halogen lighting as well as bright fluorescent lights, so we can replicate this condition as well as seeing how they look when in brighter conditions.
The lab is also equipped with a high quality Blu-ray player and video recorders along with a moderately priced stereo sound system. The latter is used as a reference when assessing TV sound quality.
We also have a number of calibrated instruments, which we can use to accurately measure white and black points on the TVs, or make complete colour accuracy assessments when necessary.
Ready to buy?
If you're ready to start comparing models, take a look at our latest TV reviews. If you want to do a bit more research about what to look for in a telly, check out our TV buying guide.