How we test security software


How we evaluate antivirus security software for desktop and mobile to help you choose the best protection for your computer devices

Our expert testers


You can’t try out security software in the store, or for the most part even buy it there. These days it’s all online downloads, whether from the manufacturer’s website or from an app store. And installing and configuring it correctly for your purposes takes time.

We collectively use professional labs in Europe that specialise in security software. We choose software that you can buy in Australia for our reports.

As security software is available worldwide, CHOICE joins with our sister consumer organisations worldwide to test desktop security software. Sharing costs and specialist expertise helps us test as many programs as possible to the depth that's required, using dedicated test labs located in Europe. Product pricing is sourced using the product's Australian website wherever possible and prices are in Australian dollars (or converted to Australian dollars where necessary).

Testing considerations

To ensure an even playing field for testing, each software product is updated to the latest version available at the time of testing. The malware samples used for the performance tests are the same for all products and the tests are performed at the same time for all of them. 
 
Testing is done using a freshly installed and updated version of Windows 10 or macOS. For Windows testing, Google Chrome is used as the default Web browser, but with its built-in URL filters (anti-phishing, antimalware) disabled and without signing in to Google. For macOS, Safari is used but with its built-in URL filters disabled. Each test computer includes the latest version of Office, Acrobat Reader VLC, Firefox and 7zip.

Desktop protection

The overall score comprises protection (65%), ease of use (25%), and system impact (10%).

Ease of use assessment takes into account the installation, configuration and normal day-to-day use of the product; the clarity and simplicity of the user interface; built-in and online user help such as guidance, help menu and options; and ease of keeping the software up to date.
Protection Score testing includes on-demand scanning, both connected to and disconnected from the internet; real-world protection, using known malicious websites including so-called drive-by websites, plus malware infections from connected media such as a USB flash drive; current malware detection, which includes the speed at which software is updated to address emerging threats; URL protection, which includes blocking online malware, especially phishing websites such as false banking and social media sites; firewall effectiveness; alerts and quarantine.

System Impact Score is the result of assessing the software's use of disk space and memory, plus its effect on boot time and common tasks such as program launch and use, file copying and file compression and decompression.

Android protection

All mobile security apps are tested by specialist independent security test lab AV-Test. Tests are conducted in a worldwide unique, fully-automated Android test system and using the latest smartphones. 

Scoring is based on how well the app protects from malware and takes into account system impact – its effect on device speed, memory use and battery consumption, plus other features.

Testing includes on-demand scanning, invoked by a user at any time, and dynamic detection, which identifies and remedies malware as it occurs. Tests are run while simulating typical real-world user activities such as installing apps and launching websites, plus usage of everyday apps such as Adobe Reader and YouTube.

The overall score comprises protection score (50%), system impact score (50%). The protection score covers how effectively it shields the device from malicious software (malware). The system impact score reflects the extent to which running the software affects overall performance. A higher score is better.


Leave a comment

Display comments