Almost every home in Australia has at least one TV. In fact the majority of homes will have two or more. New technology is giving us brighter, more saturated pictures with more detail, and bigger screen sizes. But size isn't the only factor when it's time for us to test TVs.
How we choose what we test
From April to August every year TV makers release many more models than we, or anyone else, can test. We liaise with the manufacturers to get information on what will be released and plan our testing based on that information along with market sales information which provides insights into what sold well in the previous year.
- Once the models are officially released we select what we'll buy based on the popularity of price points and sizes from the previous years, as well as what will be promoted in the stores.
- Our buyers help keep us informed on what is being promoted in the major retailers so we can buy the sets that you'll see when you're out shopping.
- Occasionally we'll include a brand that's outside the mainstream to check that we're not missing a bargain, or something better than the main brands can deliver.
If you've got your eye on a model that's not in our test, make sure you read our guide to testing a TV in-store.
- Our latest TV test report has more than 200 models, although the companies constantly refresh their range so many of the TVs we test may be hard to find after a few months on the shelves. We test in batches of five to seven at a time, because as they get bigger they take up more and more space. There is also the fatigue factor for our viewing panel when looking at too many models in one session.
- The tests take around three weeks and are designed to give each TV a fair go at providing a good picture, but our primary focus is to get as close as we can to real world experiences.
- We want to watch what you watch, not just what will make the television display look good in-store.
If you want to see what brands have been performing best in the last 12 months of TV testing, check out our Best Brand - TV report.
Changes to TV testing
Flat screen TVs enjoyed a surge in popularity in the early 2000s with the most popular display size of 32-inches (87cm). The model of choice became 42-inches (107cm) within a couple of years and remained so until the 55-inch (140cm) TV toppled the 42-inch model last year as the new standard size for the average lounge room. This year, 65-inch models are predicted to be the volume seller.
UHD (also called 4K) and the appearance of OLED screens adds even more quality to the mix, with more accurate blacks and HDR for more detail in darker scenes providing an incentive for some consumers to look for an upgrade. This is why we test how other resolutions such as SD and HD perform on the latest UHD (4K) TVs.
- Until recently most of the free-to-air channels broadcast their highest rating shows on their primary channel in standard definition (SD).
- This meant much of what you were watching, including major sporting events, was in SD. Therefore SD performance was a large part of the overall rating when we tested TVs.
- CHOICE has found many TVs with very high resolution struggle to up-scale SD sources to their native resolution, leading to low scores in our testing for SD.
- Although many of the TVs deliver better than OK pictures with an HD source, poor results for SD performance means their overall score is quite low.
From SD to HD (and even 4K)
Thankfully most of the Australian networks have finally moved over to HD broadcasting for their primary channels. This move, as well as the increase in high definition (HD) and even 4K content on online subscription services such as Netflix, Stan and Foxtel, has forced an adjustment to the CHOICE TV test weightings to better reflect the type of TV you are more likely to watch.
- We are now assessing 4K content and will be including it in our overall scoring during 2019 as we feel 4K has become relevant enough to more people as content becomes available on multiple platforms. While it only makes up 5% of our overall score, this may increase as the amount of 4K content available to Australians improves.
- We've also changed the weightings for our performance tests to reflect the increase in available HD content. Online streaming routinely delivers HD-quality video and most of the free-to-air stations now broadcast HD content on their main channel.
There has been a lot of talk about 'Smart TVs' and we have changed our testing in this area as the TV becomes more of a hub rather than a smart destination in the home.
- The latest models continue to offer smart features such as app support and access to streaming services such as Netflix and Stan, however additional support for voice assistance such as asking the TV questions via Google ('OK Google') and Amazon ('Alexa') has meant the TV is not only smart but also integrated into a smart home.
- Our testers have found the experience to be less than perfect, but we have noticed improvements in the latest models helping the TV to become something of a smart hub in the modern home.
We've also reassessed our reference TV to better reflect consumer assessments on what is considered an 'average' TV. This means there'll be no changes in the rankings of any of the TVs we've tested, only the overall scores of all the TVs.
We will continue to accept one-off loans for large TVs over 75-inches as we could not not justify spending over $15,000 of your money (CHOICE is funded only by our members) for a TV and so will accept the occasional review model of 75-inch plus models (which we send back to the company following the review).
Our expert testers
Our technical staff are some of the most experienced TV testers in the country with 35 years' experience testing televisions between them.
- They've seen the progression from bulky cathode ray tube TVs, through the very expensive plasma sets, on to LCD and now OLEDs and QLEDS.
- In that time they've sat on Australian Standards Committees to make sure the television you buy is safe and have had a major influence in getting MEPS (mandatory energy performance standards) implemented so the TV you buy today uses significantly less energy than the sets of old.
How we test
The most basic and important part of our testing is that it's a comparison of the relative merits of each television against the others in the test. To this end we have a number of reference TVs that we go back to, to check our scoring is consistent. This way our scoring is based on a direct comparison to a known quality level for each batch.
- The final scores are based on hundreds of individual scores which our testers produce as they work their way through the test method.
- Each TV is given some time to warm up and the tester then uses the TV's settings out of the box.
We test under the following broad categories:
We gather a lot of information about the television including its weight, measuring the available screen area, connections (Wi-fi, Bluetooth etc.) and any other materials supplied with the TV such as cables etc.
This is one of the major components of the TV test and includes both a technical and subjective assessment of how well the TV can handle a number of different source materials.
- The TVs are placed side by side and connected to a sophisticated delivery system that provides them all with the same source material at the same time. This makes it possible for testers to go seamlessly from one TV to another to compare output.
- At least one reference TV is included in the array and the panel uses it as a basis for determining scores.
- Subjective tests are done by a panel of at least three experienced viewers.
- The TV's brand and model are obscured, so the panel can make assessments based on what they see, not what they might expect to see.
- For each test the panel assesses high definition sources from a Blu-ray disc, a recording of both a TV show and sporting footage (including long shots of players on a field moving quickly) broadcast at 1080i. This is repeated with standard definition footage.
- The panel is looking for colours that have good saturation and that don't look false or have a particular overall cast. We assess the sharpness and noise in the picture and look for obvious faults such as plastic looking skin tones, poor black levels, backlight bleeding, noise and pixilation around moving objects.
We also assess some technical footage, designed to test the TV's ability to deal with noise in the signal at both high and standard definition.
Testing on 4K content has been introduced at the beginning of 2017 and is now reported on within the overall score.Objective tests include:
- test patterns and footage which checks the TVs for overscan (how much picture is not being shown)
- jaggies (diagonal lines that appear as jagged or broken lines)
- judder and blur at a number of different frame rates.
Ease of use
After picture quality, this is the second most important part of the test. We cover a lot of the TV's functionality, but it's impossible to check everything a smartTV can do, so we check the things we've found you'll probably use most.
If you've ever lost your remote control you'll know how essential it is for using your TV. Some come with two remotes and in this case we test them both, but the score is based on the best one.
- The tester checks the size and spacing of the buttons, dials or touch pads to make sure they're easy to reach, logically grouped and not too close together.
- Labels should be big enough to read and of sufficient contrast to be easy to read in low light.
- The remote's overall size and shape is assessed for how well it fits in the hand and how easy it is to use with one hand.
On-screen user interface
The on-screen user interface should be easy to read, easy to follow, tell the user what is happening (e.g. tells the user to wait while TV is scanning for DTV broadcasts), and show clear prompts, which button to press, or what to do next.
The tester assesses the on-screen user interface for having the following attributes:
- A simple and well organised means of selecting the main functions or options.
- Easy to read characters.
- Intuitive descriptor words, phrases or symbols.
- Staying on the screen long enough.
- Having a guide as to which keys to use.
- Easy to organise frequently viewed channels, either by using the Favourites function or Channel skip/erase.
- Doesn't obstruct a significant part of the viewing area when adjusting the picture controls.
The tester measures the time it takes the TV to complete an automatic tuning scan. The Closed Captioning feature is assessed paying attention to the readability of the text, whether the user can adjust the transparency, and whether the closed captions state is retained when changing channels.
The tester assesses the browser for its:
- ease of scrolling, entering data and whether it can play Flash video format.
- We check for some common apps (YouTube, Netflix etc.) to see if they're prominently displayed or have to be searched for.
- TV catch-up apps are also checked and points are lost if they're not all available.
- The layout of the screens that get you to and from apps are also assessed for their logic and accessibility.
The tester plays a number of high quality voice and music recordings on a DVD player connected to the TV and compares it a reference system. The assessment is then made by a group of three to score the TV at a volume that the trialists feel is sufficient for listening in a normal room and also at the TVs maximum volume (assuming the latter is louder).The tester is listening for:
- clear voices
- sufficient frequency response to provide some drama
- rattles, humming or obvious distortion.
Test criteria explained
- Prices are recommended retail, as supplied by manufacturers – you should be able to better this price in any store.
- Overall score is made up of: high definition (HD) picture quality 30%, standard definition (SD) picture quality 20%, 4K/UHD picture quality 5%; user interface score 10%, remote control score 10%, smartTV score 10%, EPG score 5%, sound quality score 5% and energy use score 5%.
- USB inputs = the number of USB2 and USB3 inputs. The latter will be more important as more 4K devices come to market.
- 4K/UHD includes footage from a 4K movie with HDR enabled on the TV and the player (XBox with support for 4K and HDR).
- High definition includes footage from a Blu-ray movie, technical assessment of noise and free-to-air broadcast sports footage.
- Standard definition includes footage from DVD movies, technical assessment of noise levels, jaggies etc. and free-to-air broadcast sports footage.
Our test lab
The CHOICE TV lab is a purpose-built testing facility that allows us to directly compare up to a dozen TVs at a time. It may not look like your lounge room, but its controlled lighting, temperature, humidity and neutral grey walls combine to make it possible to give every TV we test a good chance of delivering its best performance.
We use a state of the art signal delivery system that provides each TV with the same input at the same time. This makes it possible for our testers and viewing panels to directly compare the TVs with each other. It also makes it possible to quickly switch from one input type to another so we can assess the differences between them.
Our surveys of your viewing habits tell us that the majority of people watch TV in subdued halogen lighting. Our test lab has both halogen lighting as well as bright fluorescent lights, so we can replicate this condition as well as seeing how they look when in brighter conditions.
The lab is also equipped with a high quality Blu-ray player and video recorders along with a moderately priced stereo sound system. The latter is used as a reference when assessing TV sound quality.
We also have a number of calibrated instruments, which we can use to accurately measure white and black points on the TVs, or make complete colour accuracy assessments when necessary.