04.Lack of accuracy
"While Automated Valuation Models (AVMs) may be useful for lending institutions in a portfolio and mortgage situation, I have grave reservations about their general use by consumers."
So says Peter Rossini, Program Director of the University of South Australia’s School of Commerce. Peter has published several research papers and studies of automated property valuation models.
The competing websites get most of their data from the same sources — state government property sales registers. They supplement this with information about recent sales from their estate agent contacts, and details of properties’ attributes. Complex statistical modelling of various types (including hedonics, which takes into account individual property attributes, such as the number of bedrooms and presence of a parking space) is then used to generate estimations and values. Each model uses its data differently, putting more or less weight on particular components.
However, each company knows that only a portion of the automated estimates it sells is within the right range. Residex
, for example, says estimates will generally “only be in an acceptably tight range in around 70% plus of the cases regardless of who undertakes the calculation”. If correct, that means around 30% of estimations from all companies are off the mark. Residex
concedes that “where there is the potential for such significant errors it is possible to unintentionally mislead the public.” For that reason, the company’s reports provide “rules and test values for the users to confirm and identify a value from and to help the user become more familiar with the valuation process”. RP Data
points out the limitations too. “We don’t want people thinking they can rely on an automated valuation,” it says.
Research in the UK, where automated valuations are also used, acknowledged the overall good performance of the tested models, but found that low-value properties are more likely to be overvalued, and high-value properties are more likely to be undervalued. It also found more variation (errors) in less homogenous segments of the property market, such as regional or rural areas, or suburbs where properties aren’t similarly designed in a uniform pattern — which is a much more common situation here than there.
In the UK, automated systems provide an indication of confidence in the accuracy of the valuation (sometimes referred to as confidence levels or uncertainty levels). In Australia, however, only RP Data
reports give an indication of the accuracy you can expect, which varies from suburb to suburb. Its ‘Forecast Standard Deviation’ (FSD) is a 68% probability that the report’s estimated value and the ‘true valuation’ of the property are within a certain range. An FSD of 15, for example, should mean there’s a 68% chance of the estimated value being within plus or minus 15% of the true value. Australian Property Monitors
claims that on average, half its reports are within 10% of the property’s sale price, and that 80% are within 20% of the actual value. Residex doesn’t display its confidence level for individual property estimations to the public, only to banks.
So property websites recognise their own limitations, and legal disclaimers point out that consumers shouldn’t rely solely on their figures. “Our reports are a guide or tool, but you need to get a valuation later,” says Residex
. “A valuation is much more detailed and accurate than our Property Explorer reports could ever be 100% of the time.” Australian Property Monitors
echoes those comments. "Our reports should be used as part of a suite of research undertaken when buying a property. Customers shouldn't rely on our reports in isolation for financial decisions. Buyers also need building inspections and other due diligence. We have specific disclaimers to protect us because we want customers to use the report as an estimate not a valuation. Only a licensed valuer can give a valuation."