🔗 Share this article

Tech firms and child safety organizations will be granted authority to assess whether AI systems can generate child exploitation images under recently introduced UK legislation.

Significant Increase in AI-Generated Illegal Content

The declaration came as revelations from a protection monitoring body showing that reports of AI-generated CSAM have increased dramatically in the past year, growing from 199 in 2024 to 426 in 2025.

Updated Legal Structure

Under the amendments, the government will allow designated AI companies and child safety groups to examine AI models – the foundational systems for chatbots and visual AI tools – and ensure they have sufficient safeguards to prevent them from producing depictions of child exploitation.

"Fundamentally about stopping exploitation before it happens," stated the minister for AI and online safety, noting: "Specialists, under strict protocols, can now identify the danger in AI models promptly."

Tackling Legal Challenges

The amendments have been implemented because it is illegal to create and possess CSAM, meaning that AI developers and other parties cannot generate such images as part of a testing regime. Until now, authorities had to delay action until AI-generated CSAM was uploaded online before dealing with it.

This legislation is designed to preventing that problem by enabling to halt the production of those images at their origin.

Legal Structure

The amendments are being added by the authorities as revisions to the crime and policing bill, which is also establishing a ban on possessing, producing or sharing AI systems developed to generate exploitative content.

Practical Consequences

This recently, the minister visited the London base of Childline and listened to a simulated conversation to counsellors featuring a report of AI-based abuse. The interaction portrayed a teenager requesting help after being blackmailed using a explicit AI-generated image of himself, constructed using AI.

"When I learn about children facing blackmail online, it is a cause of intense frustration in me and justified anger amongst parents," he said.

Concerning Statistics

A prominent online safety organization reported that instances of AI-generated abuse content – such as online pages that may contain numerous images – had significantly increased so far this year.

Instances of the most severe content – the gravest form of abuse – increased from 2,621 visual files to 3,086.

• Girls were overwhelmingly victimized, accounting for 94% of prohibited AI depictions in 2025
• Depictions of newborns to toddlers rose from five in 2024 to 92 in 2025

Industry Reaction

The law change could "represent a vital step to ensure AI tools are safe before they are launched," commented the head of the online safety foundation.

"Artificial intelligence systems have made it so victims can be victimised repeatedly with just a simple actions, giving offenders the capability to make potentially limitless quantities of advanced, lifelike child sexual abuse material," she added. "Content which further exploits survivors' trauma, and makes young people, particularly girls, less safe both online and offline."

Support Interaction Information

The children's helpline also released details of support sessions where AI has been mentioned. AI-related harms mentioned in the conversations comprise:

• Using AI to rate weight, body and appearance
• AI assistants discouraging children from talking to trusted adults about harm
• Being bullied online with AI-generated material
• Digital blackmail using AI-faked images

During April and September this year, the helpline delivered 367 support interactions where AI, chatbots and associated terms were discussed, significantly more as many as in the equivalent timeframe last year.

Half of the references of AI in the 2025 interactions were connected with mental health and wellness, encompassing utilizing chatbots for support and AI therapy apps.