The in(k)s and outs of printer testing
We all print a wide range of documents at home and expect them to be high quality. Most other electronic gadgets we buy can be tried out in the store, but you can't really to that with printers, and with the major manufacturers marketing dozens of models it's increasingly difficult to pick out the ones that do just what we want. If you have to rely on advice, make sure it's expert and independent.
Add to this the costs of consumables. Ink and toner cartridges frequently cost more than the printers they serve – litre for litre, printer ink is one of the most expensive liquids you can buy. How much is a printer really going to use? Especially inkjet printers that use ink just getting ready to print?
Our tests aim to tell you what each printer can do, and just how much it'll cost you to use it over time.
To test as many printers and MFPs as possible to the depth that's required, we share costs and expertise with our sister consumer organisations around the world, using professional labs in Europe that specialise in printer testing. Each year they test hundreds of printers, and we pick out the models you can buy in Australia for our reports.
So our testers have a huge range of experience, which gives us consistency not only in the quantifiable measures like speed and ink consumption, but the more subjective area of the quality of output. Our test labs use a panel of people, comparing results against standard benchmarks to minimise any bias.
How we choose what we test
Just like CHOICE, our partners in Europe buy the printers they test from the retailers to make sure they're testing the same product the consumer buys. All the models they choose are commonly available worldwide, or have local variants. This means we can report on far more products than we could test here in Australia, and at greater depth, but it does mean that some brands that have a lower profile around the world, or are more business-orientated, have to miss out.
We measure the speed and quality of output of all the printers we test, how easy they are to use, their networking capability and their power consumption. We also assess multi-function printers for the speed and quality of their scanning and copying.
The overall score for a standard printer is made up of printing (67.5%), ease of use (22.5%) and networking (5%) and ink costs (5%) scores.
For multi-function printers the overall score is made up of printing (54%), ease of use (18%), scanning and copying (9% each), networking (5%) and ink costs (5%). Power consumption is a small factor in overall cost, so we don't add it to the total, but it's included in the reports.
Printing quality is always a matter of judgement, so we make that judgement as objectively as possible with five trained panellists looking at four standard documents to represent common printing habits:
- a typical black-and-white letter (15%)
- an 8x10-inch high quality photo (40%)
- a coloured spreadsheet (15%)
- a colour document with a mixture of graphics and text (20%)
The results are also measured for moisture resistance (10%).
The photos are printed on the best-quality glossy photo paper recommended by each manufacturer, using the printer's 'best quality' setting. The same setting is used for the mixed document, printed on premium matte paper. The other documents are printed on standard paper with standard settings. All the results are compared with three benchmark results scored at three, six and nine, to assess clarity, contrast and brightness, colour and overall impression. For colour lasers, we assess plain-paper copies of photos in the same way, but we don't include the photo component for mono laser printers.
Moisture resistance is measured by seeing if a highlighter pen smears text and also whether ink runs or bleeds by dripping water onto 24-hour-old photo prints.
We measure the time it takes to print the four documents above, allocating 25% of the total for the photo, 25% for the colour document, and 25% each for the letter and the spreadsheet. Timings are taken in 'ready' mode, measured from selecting 'OK' to the emergence of the copy. Several copies of each document are measured.
Scanning and copying
We scan and copy a magazine page, an 8x10-inch colour photo and a 3D object. Scan quality is assessed on a computer monitor by three expert viewers comparing the results to a high-quality reference scan. The same panel assesses copy quality, comparing the copy to the original magazine page.
To determine the life of ink and toner cartridges, we don't use the cartridges that come with the printer – they're often only partially full. Instead, we buy full sets of high-capacity cartridges and run the printers almost continuously, printing as many text, spreadsheet and photo pages as possible from separate sets of cartridges until at least one cartridge runs out of ink. Where possible, any 'low ink' warnings are bypassed. The weight of the cartridges before and after printing determines the weight of ink used, and hence, together with local cartridge prices, the cost per page of each type of document. We use commonly available online prices for genuine manufacturers' cartridges bought individually. We then use our 'average use scenario' (1000 mono prints, 250 colour document prints and 250 full-colour photo prints over the course of a year) to calculate long-term running costs.
Ink consumption for maintenance
Inkjet printers run a 'maintenance cycle' whenever they decide the print-heads need to be cleaned before use, and this uses ink. To measure the costs of this, we use a separate printer from the normal life test over a three-week period, printing two copies of the black and white text document on three mornings each week, and the coloured spreadsheet on one morning and two afternoons. The printer is idle for three days each week and is switched off between uses. We weigh the cartridges before and after the test and compare the weights of ink consumed to those used for the same number of copies of the same documents in the continuous test. The ratio between the two measurements provides the excess ink used as a percentage.
Ease of use
Our ease of use measure the major functions of the printers. The scenarios are:
- Set up and installation Using the 'quick start' guide, we fit any trays and covers delivered with the printer and install the ink cartridges, make connections, install drivers and other software provided and set up the system. We consider accessibility, simplicity, adequacy of instructions, labelling and time taken.
- Printing standard documents from a computer Loading paper, changing cartridges and various printing options including double-sided printing (where applicable) and changing print quality settings. We consider how easy it is to change settings and display help and how the printer copes with paper-out or empty cartridge conditions. We check if you can continue to print colour when the black cartridge runs out or black when the colour cartridges do, and we check the single-sheet feed and output tray capacity.
- Printing a 6x4 photograph from a computer Change paper size and colour cartridges if required, select correct settings and print photo.
- Handling of paper jams and replacement of ink cartridges How easy it is to remove paper if there's a print jam and potential problems with changing ink or toner cartridges, including leakage and difficult locks and catches.
- Networking Set up wired and wireless connections; assess the steps required and the adequacy of the instructions.
Each aspect is assessed by at least two technicians and the following rating is given: 1 for worse than expected, 2 for as expected and 3 for better. Any unusual aspects are noted and specific yes/no questions are answered.
Want to know more?
See our printer buying guide.
Ready to buy?
See our multifunction, inkjet and laser printer reviews.