A new report by NewsGuard has revealed that a network of 17 TikTok accounts is exploiting hyper-realistic AI voice technology to disseminate conspiracy content that appears convincingly human.
According to the report, these accounts employ AI text-to-speech software to instantly create narrations and voices for their videos on a large scale showing a novel way bad actors can leverage AI software to quickly and cheaply produce false content that reaches vast audiences.
The report revealed that these accounts, active since June 2023, have generated approximately 5,000 videos, many featuring AI-generated voiceovers. These videos have amassed 336 million views and 14.5 million likes as of September 25, 2023.
“Baseless narratives pushed by the AI narrators in these videos include the claim that former U.S. President Barack Obama was connected to the death of his personal chef Tafari Campbell; that TV show host Oprah Winfrey is a “sex trader”; that actor Jamie Foxx was left paralyzed and blind by the COVID-19 vaccine; and that comedian Joan Rivers was murdered for claiming that Obama was gay,” the report read.
“For example, a July 2023 video posted by an account that goes by @e.news.tv featured a female narrator with a polished tone delivering a three-minute “news” report that baselessly linked Obama to the death of his personal chef. The video began with an AI narrator stating, “Obama has reacted to Sherise Campbell blaming him for the death of her husband Tafari Campbell. Here’s what he had to say about it.”
“This introduction, seemingly read by an AI narrator, was followed by audio of a robotic, clearly AI-generated “Obama” reading a statement on Campbell’s death.
“While I cannot comprehend the basis of the allegations made against me, I urge everyone to remember the importance of unity, understanding, and not rushing to judgements,” the Obama voice stated. As of Sept. 27, 2023, the video had garnered more than 50,700 views on TikTok.”
NewsGuard reconstructed the video’s audio introduction using ElevenLabs’ AI text-to-speech software. An AI speech classifier by ElevenLabs confirmed a high likelihood that the audio was generated using their software.
Deceptively, 13 of the 17 accounts pretended to be mainstream news outlets, TV shows, or famous TikTok profiles to appear credible. Comments on these videos indicate that AI-generated content is influencing users and aids in spreading baseless accusations.
According to the report, TikTok introduced a disclosure label feature for AI-assisted content in September 2023, but none of the identified AI-narrated conspiracy videos had such labels as of September 26, 2023.
The report noted that TikTok took action by removing three accounts for impersonation and one video for manipulated media and restricted four videos from algorithmic promotion for spreading misinformation.
Hafiz Malik, a computer science professor at the University of Michigan-Dearborn, noted that AI text-to-speech technology has drastically lowered the barriers to producing fabricated audio, making it easier and faster to create deepfakes with a mere click of a mouse.
Nurudeen Akewushola is a fact-checker with FactCheckHub. He has authored several fact checks which have contributed to the fight against information disorder. You can reach him via [email protected] and @NurudeenAkewus1 via Twitter.
