How we test robot vacuums

What happens behind the lab doors when we test robot vacs.

Dishing the dirt

If you think your stick vacuum is a convenient companion for a quick clean up, you're going to be pretty impressed with the idea of a robot vacuum cleaner. Ideal for any time-poor person, just set up your robot vac and it'll get the job done for you – or so the ads say. Robot vacuums are increasing in popularity but they still need to make improvements in some areas, so you may not want to part with your regular barrel or stick vac just yet.

Our expert tester has tested robot vacs from when they first hit the market, and has seen their rise in popularity. With each test, more manufacturers jump on the bandwagon in the aim to produce the best robot vac yet. Here we'll tell you how he puts robot vacs through their paces to give you the results that help you choose what to buy.

Our expert testers

With 10 years of testing experience, our tester Matthew Tung is passionate about determining a product's technical performance. He tests a wide range of products in the CHOICE labs from cots and strollers to electric blankets and blood pressure monitors, as well as standard corded vacuums. He also sits on Australian Standards committees.

How we choose what we test

With so many to choose from, what makes us choose one robot vacuum to test over another? As with most of our product testing, our aim is to test the most popular models on the market and what you're most likely to see in the retailers.

We survey manufacturers to find out about their range of models, we check market sales information and we also check for any member requests to test specific models. From this information we put together a final list that goes to our buyers. They then head out to the retailers and buy each product, just as a normal consumer would. We do this so we can be sure they're the same as any consumer would find them and not 'tweaked' in any way.

Updating our test method 

From time to time, we're forced to update our test method due to practical realities outside our control. Recently, we discovered it was no longer possible to purchase the specific carpet we'd been using to test our vacuums, so we had to start using a new test carpet. As it wouldn’t be fair to compare vacuums tested on our new carpet with results from the old carpet, we've had to start our testing from scratch. This means we've retired all our old robot vacuum reviews and replaced them with a new batch of robot vacuums tested using our new carpet and rated under our new scoring system. Because of these changes, you may notice the scores for some of the previously tested models have changed slightly.

How we test

Dirt removal from hard floors

Matthew scatters a measured amount of sand evenly over a vinyl floor area and lets the robot run for 25 minutes, or until the vacuum completes its cleaning cycle, measuring the dirt pick-up at intervals throughout. He places objects (like chairs and a bench) in the room to make the set-up look similar to a home. With normal vacuums we pre-load the dust collector with dust, but robot vacs have a very small dirt receptacle which is normally emptied after a cleaning session, so we don't pre-load them.

Dirt removal from carpet

Matthew sprinkles sand evenly onto a section of carpet and lets the robot run for a set period of time on the carpeted area. The sand collected is weighed and recorded.

Cleaning corners and edges

Sand is spread into a right-angled corner, and the robot vacuum cleaners are assessed to see how close to the corner they can get to pick up the dirt.

Pet hair removal

Matthew embeds cat hair into the carpet and allows the vacuum to run for six minutes. He rates the vacuums based on how much cat hair is left behind after the test. This is a visual assessment.

Ease of use

Robot vacuums are very easy to use and operate, so ease of use isn't scored. However, Matthew will make comments about how easy it is to clean the dirt receptacle, which you can view in the good and bad points for each product in the comparison table. 

Test criteria explained

Overall score

The overall score is made up of:

  • Hard floor dirt pick-up (40%)
  • Dirt removal from carpet (30%)
  • Cleaning corners and edges (15%)
  • Pet hair removal (15%)

Our test lab

Our testing laboratory is up to date with the latest reference machines and calibrated measurement tools for our testers to bring you the right results.

Ready to buy?

Ready to buy a robot vacuum? Take a look at our results.

More about vacuums

Leave a comment

Display comments