more_vert
Late last year, an AI-generated video faked the image of a well known endocrinologist and spread dangerous medical advice on Facebook. InSight+ spoke to industry professionals about the effect these fakes have on health and trust.
Last year, diabetes patients clicked on a Facebook ad that read: “Managing type 2 diabetes doesn’t have to be hard or expensive.”
Behind the link, it appeared that Professor Jonathan Shaw, a well-respected endocrinologist, told journalist Karl Stefanovic that metformin — a standard and established drug used to treat type 2 diabetes — was “not enough to manage diabetes effectively,” and that medical professionals still prescribing it were “idiots.”
The video was a fake. An AI-generated video spreading misinformation, and promoting a product not approved by the Therapeutic Goods Administration (TGA). It was, however, a convincing deepfake, and led to confusion in the diabetes community.
Noticing the fake
Professor Jonathan Shaw is an endocrinologist working almost entirely in diabetes both as a clinician and a researcher. He is the deputy director of The Baker Institute, a legitimate and well established research centre focused on heart disease and diabetes.
Professor Shaw was on holiday when his patients started calling the Institute.
“The Baker was receiving phone calls from patients asking what was the treatment that I had recently been talking about. I was trying to remember what interviews I might have done, or podcasts, to try and figure out what it might be,” he said.
“Eventually my son sent me the link. That was when I realised that this wasn’t something that I had done. So, I tried to understand the extent of it, what it meant, and what I and the Baker needed to do about it,” he said.
“My biggest concern was that people with diabetes would be fooled by this and would take unsafe actions,” said Prof. Shaw.
“There were two components: one was that I was made to say that metformin is a bad drug and that anybody prescribing it was an ‘idiot’,” said Prof. Shaw.
“Of course, metformin is a good drug and for many people is very helpful. We’ve used it for many decades, it’s effective, cheap, and has a side effect profile that is well understood,” said Prof. Shaw.
“The second aspect was, patients might try and access this other product, about which I know nothing. I have no idea if it’s harmful,” he said.
“But I do know that it was costing somewhere between three and four hundred dollars. So, there would certainly be a financial impact on people,” said Prof. Shaw.
Complicated consequences
There were other issues created by the deepfake, which Professor Shaw and the Baker Institute had to tackle.
“I was made to say that this [fake] product had TGA approval. It doesn’t. There are significant penalties for doctors claiming that something has TGA approval when it doesn’t,” he said.
“That opened legal exposure for me. It is possible that a person makes a complaint to the TGA or to AHPRA [the Australian Health Practitioner Regulation Agency] that I was promoting clinical falsehoods. I would then have to prove that this wasn’t me. And that may not be easy,” said Prof. Shaw.
The Baker Institute wrote to both the TGA and to AHPRA to inform them of the deepfake.
“There are other concerns, which are reputational. It makes me look a little bit like a second-hand car salesman. Not that I have anything against them — but it makes me look as if an important part of my work is to sell products that I might get some benefit from,” he said.
“Most people don’t understand how the health care industry works. Many people, understandably, suspect that doctors may be influenced by pharmaceutical companies and get direct reimbursement for them. And this [deepfake] supports that idea,” he said.
“So, it doesn’t just affect me. It tars all health care professionals.”
The Institute contacted the platform, Facebook.
“We did engage with Facebook to try and have the posts taken down. That was not easy. Multiple people posted multiple complaints and that seemed to do nothing,” he said.
“We realised … we could make a complaint based on copyright and intellectual property. Remarkably, when we did that, it was taken down in about two or three hours,” he said.
“I’ve certainly devoted many hours to this. Multiple people at the Baker have had input. Once you start adding up people’s time, it’s in the $10 000 plus arena,” he said.
“I have mentioned [the deepfakes] to government. But unfortunately, very little interest was expressed.”
The invisible arms of misinformation
Professor Shaw doesn’t know how far the deepfake reached, but he knows it was further than he could see.
“[The deepfake] was fairly successful in targeting people with diabetes. The ad was on Facebook. They don’t have a list of people with diabetes, but their algorithm seemed to find them,” said Prof. Shaw.
“We had dozens of calls at The Baker Institute, and a good number of my own patients have told me that they have seen it,” he said.
“I had one patient whom I’ve looked after for many years say, partway through a consultation, ‘I’ve just seen a video of you on Facebook saying how bad metformin is. So, why do you keep prescribing it to me?’” said Prof. Shaw.
He said that, once people see a faked video, they can find information that seems to support it.
“If people have never heard of me — which most people haven’t — and they Google me, they would see my expertise in diabetes,” he said.
Trust issues
As the global medical and scientific community sees the scientific neutrality under threat within United States government institutions such as the US Centre for Disease Control and Prevention (CDC), trust in unbiased medical and scientific information is under pressure.
Professor Shaw says that patients have the right to research without being targeted by misinformation.
“This is a population of people who are looking for answers. Anybody with a chronic health condition is vulnerable, and many of them are looking for the fix,” he said.
“As people find out what has happened [with the deepfake], they become less trusting of this kind of information,” he said.
Professor Shaw said that, even if patients made an appointment to clarify with their doctor, it was a waste of time and money.
“It’s an entirely wasteful episode. And we see a lot of this in health care.”
A community affected
Karen Crawford is a Diabetes Victoria Advocacy Officer, registered nurse and credentialled diabetes educator.
Ms Crawford said the deepfake affected patients, and there can be serious consequences.
“There was a lot of confusion for people living with diabetes who take metformin and/or other medications to manage their diabetes,” said Ms Crawford.
“People may stop taking their prescribed medications and risk possible adverse medical outcomes. This might look like glucose levels above or out of target range, and any unknown side effects of a substance that isn’t TGA approved, or adverse interactions,” said Ms Crawford.
“In the long term, above-target glucose levels can increase the risk of diabetes-related complications.”
“I always encourage people to check the evidence! Look it up on the TGA website and ask your local pharmacist.”
Ms Crawford said that this scam is not the only one that diabetes professionals are aware of on social media.
“We have noticed lately an increase in glucose monitoring product scams, which include the logos of reputable organisations while advertising a product that is far from legitimate,” she said.
“They are advertised as ‘non-invasive’, ‘pain-free’ and ‘no finger pricks required’. At this point in time, there are no devices that can accurately measure glucose without piercing the skin. Any ads for watches or gadgets that claim to measure glucose levels from the skin surface, are a scam,” said Ms Crawford.
According to Diabetes Victoria, the only TGA approved devices for monitoring glucose levels are:
- Blood glucose meters that use a lancing device with a lancet to prick the finger.
- Continuous glucose monitors (CGM) with a cannula/sensor that is inserted through the skin to sense the glucose level in the interstitial fluid around the cells.
Tackling the deepfakes
Kathy Sundstrom is the National Manager of Outreach and Engagement for IDCARE, an organisation offering scam, identity and cyber support services for individuals and businesses.
“Last year, it was estimated the world lost $9 trillion (US) to cybercrime. It is by far the largest criminal industry, with profits greater than the drugs and sex trade combined,” she said.
Ms Sundstrom said that of the over 91 000 cases and over $580 million in losses reported to IDCARE in 2024, people from every demographic and sector were included.
“Despite massive attempts from government and industry to increase awareness, the losses reported show no sign of declining,” she wrote.
“As more people have become aware of celebrity-endorsed scams and are more wary of videos and posts of celebrities endorsing products, it appears to be shifting to impersonating everyday people, in particular respected people, like doctors,” she said.
“I think we now live in a world where we can no longer trust what we see or hear. Criminals can make deepfakes, spoof numbers, write emails and create websites impersonating legitimate organisations. Governments and industry are always one step behind. It’s why education is so important,” she said.
“In my view, we need to stop seeing cybercrime as an if, but see it as a when, said Ms Sundstrom.
Professor Shaw thinks that deepfakes will affect the way professionals communicate.
“We feel it’s a public duty to communicate our knowledge and findings,” said Prof. Shaw.
“I don’t think it’ll make a difference for me. I think I’ve already got too many things out there in the public domain. But colleagues have told me they refuse to [make videos] because they can be used to train AI. This will impact public communication and professional education.”
Subscribe to the free InSight+ weekly newsletter here. It is available to all readers, not just registered medical practitioners.
Fascinating and chilling story. I wonder how much the deepfake was ingested by LLMs before it was taken down. Garbage in: garbage out. Perpetuating the whole AI circus. For clarification, I’m not AI averse, but of the opinion we have already passed the singularity where they became out of our control.