06.How we test
Our testing includes both performance and ease of use in installing and using the hardware and software. The software ease of use is particularly important as this is what the user has to deal with most. All six wireless solutions were purchased on a prepaid plan, which is convenient and avoids the network lock-in that you find with ongoing mobile phone plans. All the USB modems could take an external antenna, though we found this didn’t generally make much difference to performance.
Ease of use Includes the set up of the hardware (30%) and the setup and use of the software (70%). The hardware assessment includes connecting each wireless broadband device to our test laptop, and installation and configuration of any supplied software. We assess how easy it is to physically install the device and the SIM card (if not pre-installed) and get it working, user help guides, troubleshooting information and support options and ease of uninstalling everything.
The software is assessed using a clean version of the test operating system for each wireless broadband unit on test, to prevent any conflicts between different software programs.
We assess the ease of software installation; simplicity and ease of use of the main interface; whether there is a software tool for checking data usage if so, does it provide a breakdown of figures for upload and download; can users access their user account via the connection manager software; is SMS included and how easy it is to use; and does the software provide a listing of SMS contacts.
Performance We test each wireless broadband unit at different geographical locations covering Sydney and the greater west. At each location, each device is connected with its respective ISP and connection speed tests are run several times at intervals using a Sydney-based server.
We record the average results over five separate test runs. We also collect data on ping times to provide a reference point but this data is not used in scoring. We also record additional test data using a mobile antenna rather than the fixed antenna, to determine if there are any differences in performance when using the two different types of antenna. The final performance score is weighted 70% for download speed and 30% for upload speed, as download speed is faster, more heavily used and more critical to overall performance.
All the devices on test work with Mac OS X and with Windows XP, Vista and 7, but are tested under Windows 7 Home Premium (64-bit).