Deepfakes are digital clones of trusted people’s faces, bodies and voices that scammers use to steal money and information
Research shows these synthetic talking heads are harder to spot than many Australians think
Being wary of suspicious online advertisements and looking for glitches on peoples’ faces in videos can help keep you safe
Could you spot an AI-generated face? Research shows many of us think we can, but actually we can’t.
This poses a risk when criminals are deploying digital clones of trusted individuals in online advertising and video calls as part of elaborate schemes designed to con you out of your savings and sensitive personal information.
Relentless advances in generative artificial intelligence (AI) technology mean these synthetic shills are looking more and more like people you might be willing to send money or information to.
Gary* wasn’t fully aware of the power of generative AI when, in 2023, he came across a video ad on Facebook appearing to feature Elon Musk.
“There was nothing really in the news about anything like that and AI hadn’t really moved forward. There was Elon Musk, as far as I knew,” the NSW resident explains.
The ad saw the world’s richest man spruik an investment opportunity in cryptocurrency while other prominent individuals nodded along.
“He was surrounded by [Anthony] Albanese, [Julia] Gillard, [Gina] Rinehart and a few other well-known Australians and TV commentators, and he was saying: ‘I’m only giving this out to 45 people’.”
Gary lost money to an investment scam promoted by a deepfake of Elon Musk. Image: Cybertrace
Gary registered his interest and was soon contacted by phone. He says he and his wife were persuaded to invest hundreds of thousands of dollars into the venture.
He says he only realised it was a scam when he wasn’t able to withdraw any of their money.
Looking back, Gary admits the video had signs of being AI-generated and says he still comes across similar clips on social media.
High-profile entrepreneurs like Musk are a favourite for deepfake scammers wanting a famous face to lend credibility to their malicious schemes.
At the end of last year, NAB intervened to stop a customer from sending $100,000 to someone appearing to be Kevin Costner
And it’s not just talking tech heads drawing people in – at the end of last year, NAB revealed it had intervened to stop a customer from sending $100,000 to someone appearing to be Kevin Costner.
The bank said the customer had been speaking to an AI-generated clone of the Hollywood actor via video call and believed the money was going to help him buy property in Australia.
Cutting-edge technology platforms, especially generative AI services that are accessible and cheap (or even free) to use, have delivered everyday people the ability to create digital audio and video clones of real people.
AI can generate realistic-looking images and videos of people from scratch. Image: CSIRO
But a clone is only considered a “deepfake” when the video, audio or image misrepresents that person and shows them doing or saying something they didn’t.
“A deepfake means there is untruth to what they are saying. [Scammers] are trying to portray people, individuals who might not be giving consent,” says Dr Sharif Abuadbba, a researcher studying AI and other emerging technologies at the CSIRO.
“The technology underneath is ‘deep learning’ [and] a deepfake is a particular-use case which is for malicious purposes,” adds fellow AI researcher Professor Sanjay Jha from the School of Computer Science and Engineering at UNSW.
As we’ve highlighted previously, deepfakes can be audio-only – a copy of the voice of a loved one, for example, that a scammer might use to impersonate them and plead for money via a voice message on your phone.
Video deepfakes take this method one step further and clone not just the voice, but also someone’s face and body.
A video deepfake can resemble a person you’re likely to trust in order to convince you to send money or sensitive personal details or to persuade you to believe disinformation.
They can also be made to look like the individual they’re shared with – depicted in a pornographic way in order to blackmail them.
As well as celebrities, scammers have been known to create video deepfakes of entirely fictional people who a victim might believe they’ve built a romantic relationship with.
A video deepfake can resemble a person you’re likely to trust in order to convince you to send money or sensitive personal details
Experts say criminals are targeting individuals with these deepfake videos that are tailored specifically to appeal to them.
“Let’s say you think you’re developing a romantic relationship and you start talking to [that person] on video… the deepfake can look like the picture you first saw of them that you think is genuine,” says Monica Whitty, a professor of human factors in cybersecurity at Monash University.
“Video and voice are used by cyber criminals because they develop trust with the victim,” she adds.
AI platforms have become quite good at generating copies of human faces and voices or creating new identities from scratch that consumers can find difficult to identify as fake.
In research conducted last year, the Commonwealth Bank showed over 1900 Australians a series of images of human faces, some real and some generated by AI.
Almost 90% of those surveyed had said they would be able to spot synthetic faces, but in a sign of consumer over-confidence, only 42% were able to distinguish between the real and AI-generated images.
Still images from a video featuring a real actor (left) and a deepfake recreation of the same scene (right). Image: Meta
Why all AI isn’t a scam
It’s worth noting that the AI technology behind many deepfakes can also be used benevolently.
For example, Professor Jha from UNSW created his own computer program that could generate digital copies of himself to deliver legitimate lectures.
He believes the technology could be helpful in teaching in different languages and disseminating training to people in different countries.
Established advertisers are also using generative AI to create synthetic people to appear in ads for legitimate products.
Even as generative AI platforms get better and better, there are still things you can look for, practices to be aware of and strategies to adopt to protect yourself from deepfake scams.
Be sceptical
Double-checking the source of any contact you receive online is the first barrier you can build to make yourself a harder target for scammers.
A celebrity is unlikely to ever contact you asking for money, and someone you’re building a romantic relationship with should be keen to move beyond video calls and online messaging and meet in real life.
A celebrity is unlikely to ever contact you asking for money
“When you’re dating, you need to meet that person face to face within the first couple of weeks,” says Professor Whitty. “Otherwise, you need to move on before you’re groomed [into providing money].”
If the contact is someone you have met before, but you’re suspicious, go back to them using details you’ve used previously or found yourself, even if they claim they’ve lost access to their normal phone number or social media account.
Look for technical glitches
Despite advances in AI, many synthetic copies of voices and faces are still far from perfect.
“Depending on the software they use, AI models can be very basic or very sophisticated,” says Professor Jha.
There’s a chance you’ll be able to spot some of the following tell-tale glitches that can arise in the process of making a deepfake:
Check if it looks too good: AI clones sometimes have an airbrushed, over-polished look. Make sure hair, lighting and skin tone looks believable.
Read their lips: The audio in an AI video may not always match the mouth movement of the person depicted. Watch for instances of dodgy lip-syncing.
Once more with feeling: Look for unusual facial expressions that don’t match the tone of what’s being said. Beware of unnatural blinking or flickering around the eyes.
Look at the body parts: AI struggles with hands – if these appear in the video, check that they look realistic. Look also at faces for any unusual asymmetries. Ask someone to turn their head 45 degrees to the left and right or put an outstretched hand in front of their face if you’re unsure.
Get it on the big screen
Signs of a deepfake can be hard to spot if you’re using a small screen.
These quirks can be hard to spot if you’re watching a clip or taking a video call in a small format, such as on a smartphone or a shrunk-down window in the corner of your computer screen.
Moving to a larger screen and making the image as big as possible might help to highlight some of the above-mentioned signs of a deepfake.
Know AI etiquette
AI experts creating digital copies of people consider it good practice to include a watermark or some other sort of disclaimer to make it clear to viewers when something has been made with AI. Scammers are unlikely to ever follow these sorts of protocols with their content.
It’s also worth remembering that official social media accounts of trusted organisations or individuals are unlikely to post videos or photos featuring AI-generated clones of themselves or their representatives.
Watch out for urgency
At their core, successful scams require us to throw caution to the wind and do things we wouldn’t normally do.
Scammers will try to create a sense of panic, urgency or need to act quickly.
Video deepfakes might use a clone of a celebrity urging you to get involved in a limited investment opportunity or a loved one to play on your emotions with pleas for money or sensitive information.
As mentioned above, a successful celebrity is unlikely to spend their time making personal appeals to everyday people.
Get set up for safety
In its survey on AI-generated faces, the Commonwealth Bank found that while 74% of respondents thought they should set up a codeword with loved ones, only 20% had actually taken this step.
Agreeing with friends and family on a word you’ll say to each other or question-answer exchange you’ll recite is a strong way to verify any urgent requests for help that might sound eerily similar to a scammer’s opening pitch.
*Last name withheld on request
Get the inside story on our investigations into consumer rip-offs and bad business practices.
Liam Kennedy is a Journalist with the Editorial and investigations team. He answers consumers' most burning questions, from which scams to be aware of and how to save money, to whether new services and products are worth using and how the latest developments in consumer news could affect them.
Prior to CHOICE, Liam worked in production in daily news radio and podcasting.
Liam has a Bachelor of Communication (Journalism) and a Bachelor of Arts in International Studies from the University of Technology Sydney. LinkedIn
Liam Kennedy is a Journalist with the Editorial and investigations team. He answers consumers' most burning questions, from which scams to be aware of and how to save money, to whether new services and products are worth using and how the latest developments in consumer news could affect them.
Prior to CHOICE, Liam worked in production in daily news radio and podcasting.
Liam has a Bachelor of Communication (Journalism) and a Bachelor of Arts in International Studies from the University of Technology Sydney. LinkedIn
For more than 60 years, CHOICE has been fighting the good fight for Australian consumers.
In the past year alone we've uncovered systemic issues with sunscreens, investigated shonky supermarket pricing, fought for stronger scam protections and helped make complex energy pricing fairer and clearer.
CHOICE is here to provide unbiased advice and independent testing in our world-class labs. We buy the products we test, just like you do, and our expert reviews are influence free. We’re here to help you choose smarter. Hopefully you’ll also save some money along the way.
Thanks to CHOICE, you’ll never be alone when a business treats you unfairly. You can support our work by joining or donating to our cause.
