A new report by NewsGuard has uncovered how 41 TikTok accounts are using AI-generated narration to disseminate political misinformation at scale.
The platform described it as “the rise of the TikTok AI content farm.”
“Content farms are entities that generate large volumes of low-quality content, typically to attract views and ad revenue. AI has essentially democratized the content farm, allowing anyone to generate slick narration in videos instantly at scale,” the report stated.
It noted that in September 2023, the platform identified a network of 17 TikTok accounts using AI text-to-speech software to create videos advancing conspiracy theories about celebrities, garnering hundreds of millions of views.
READ: TikTok begins auto-labelling of AI-generated content
The new discovery now shows that the phenomenon appears to be growing and shifting its focus to politics. This may be due to elections slated to be held in various countries in 2024.
“These 41 AI content farm accounts, which NewsGuard detected with the help of AI detection tool TrueMedia.org, published 9,784 videos in just 458 days, from March 2023 to June 2024, cumulatively garnering over 380 million views,” part of the report reads.
“Most of these accounts were created in 2024. On average, each account posted at least one video daily, with some publishing up to four AI-narrated videos a day. The videos spread false narratives on topics including U.S. and European politics and the Russia-Ukraine war,” it noted.
False narratives published by these accounts include claims that former President Donald Trump was arrested and “cuffed” after attacking Maine’s Secretary of State; that Rep. Marjorie Taylor Greene is pregnant with Trump’s child, that NATO deployed troops in Ukraine, among others.
According to the report, seventeen of the 41 accounts regularly post false, pro-Kremlin narratives about the Russia-Ukraine war. Fourteen of these 17 accounts are in French.
NewsGuard identified identical scripts for different videos across several of these accounts, each read by different AI voices, suggesting a coordinated effort between at least some of the accounts.
ALSO READ: How war in Ukraine breeds propaganda and disinformation ― Report
The report revealed that eighteen of the accounts appeared to qualify for TikTok’s “Creator Fund,” which pays creators based on their views and engagement. This means these accounts have over 10,000 authentic followers and more than 100,000 authentic views in the last 30 days.
When contacted by NewsGuard, a TikTok spokesperson referred to the platform’s policies on misinformation and AI-generated content.
“TikTok has industry leading AI-generated content policies and tools to label content, as well as policies that prohibit the spread of harmful misinformation,” the spokesperson wrote in a July 2024 email. “We will continue to remove content that violates those policies.”
TikTok removed some of the accounts before NewsGuard inquired about them. As of July 8, 2024, 14 of the 41 accounts identified by NewsGuard had already been removed. After NewsGuard contacted TikTok for comment, another eight accounts were taken offline. Nineteen of 41 accounts remained active as of July 10, 2024.
Nurudeen Akewushola is a fact-checker with FactCheckHub. He has authored several fact checks which have contributed to the fight against information disorder. You can reach him via [email protected] and @NurudeenAkewus1 via Twitter.
