🔗 Share this article

Technology companies and child safety organizations will receive authority to assess whether artificial intelligence systems can generate child abuse material under new UK legislation.

Significant Increase in AI-Generated Harmful Material

The declaration coincided with findings from a safety watchdog showing that cases of AI-generated CSAM have more than doubled in the past year, rising from 199 in 2024 to 426 in 2025.

Updated Regulatory Framework

Under the changes, the authorities will permit designated AI developers and child safety organizations to inspect AI models – the underlying technology for chatbots and visual AI tools – and ensure they have adequate protective measures to stop them from creating images of child sexual abuse.

"Ultimately about preventing abuse before it happens," declared Kanishka Narayan, noting: "Experts, under rigorous conditions, can now detect the risk in AI systems promptly."

Addressing Regulatory Challenges

The amendments have been introduced because it is against the law to produce and own CSAM, meaning that AI creators and others cannot generate such content as part of a evaluation process. Until now, authorities had to delay action until AI-generated CSAM was uploaded online before dealing with it.

This law is aimed at preventing that problem by enabling to halt the creation of those images at their origin.

Legal Structure

The amendments are being introduced by the authorities as revisions to the criminal justice legislation, which is also establishing a prohibition on owning, creating or sharing AI systems designed to generate exploitative content.

Practical Impact

This week, the official toured the London base of a children's helpline and heard a mock-up call to counsellors involving a account of AI-based abuse. The interaction portrayed a adolescent seeking help after facing extortion using a explicit AI-generated image of themselves, constructed using AI.

"When I learn about children experiencing blackmail online, it is a cause of extreme frustration in me and justified concern amongst parents," he said.

Alarming Statistics

A prominent internet monitoring foundation reported that instances of AI-generated exploitation material – such as online pages that may include multiple images – had more than doubled so far this year.

Instances of category A content – the gravest form of abuse – increased from 2,621 visual files to 3,086.

• Female children were predominantly victimized, making up 94% of illegal AI depictions in 2025
• Depictions of infants to toddlers rose from five in 2024 to 92 in 2025

Industry Reaction

The law change could "represent a crucial step to ensure AI products are safe before they are released," commented the chief executive of the internet monitoring organization.

"AI tools have made it so victims can be targeted repeatedly with just a few clicks, providing criminals the capability to create potentially limitless amounts of advanced, photorealistic exploitative content," she added. "Material which additionally exploits victims' trauma, and renders young people, especially girls, less safe both online and offline."

Counseling Interaction Data

The children's helpline also published details of counselling interactions where AI has been referenced. AI-related risks mentioned in the sessions include:

• Using AI to rate body size, body and appearance
• Chatbots discouraging young people from talking to trusted guardians about harm
• Facing harassment online with AI-generated content
• Digital blackmail using AI-faked pictures

During April and September this year, Childline delivered 367 support sessions where AI, chatbots and related terms were discussed, significantly more as many as in the equivalent timeframe last year.

Fifty percent of the references of AI in the 2025 sessions were connected with psychological wellbeing and wellness, including using AI assistants for support and AI therapy apps.