🔗 Share this article

Tech firms and child safety agencies will receive authority to assess whether artificial intelligence systems can generate child abuse material under new British laws.

Substantial Rise in AI-Generated Illegal Material

The declaration came as revelations from a safety monitoring body showing that reports of AI-generated CSAM have more than doubled in the last twelve months, rising from 199 in 2024 to 426 in 2025.

Updated Legal Framework

Under the changes, the government will allow designated AI developers and child protection groups to inspect AI systems – the underlying systems for chatbots and image generators – and verify they have sufficient safeguards to stop them from creating depictions of child sexual abuse.

"Fundamentally about preventing exploitation before it occurs," declared Kanishka Narayan, adding: "Specialists, under rigorous protocols, can now identify the risk in AI systems early."

Tackling Legal Challenges

The amendments have been implemented because it is against the law to produce and possess CSAM, meaning that AI developers and others cannot generate such content as part of a testing process. Until now, officials had to wait until AI-generated CSAM was uploaded online before addressing it.

This law is aimed at preventing that problem by enabling to halt the production of those materials at their origin.

Legal Framework

The changes are being added by the government as revisions to the criminal justice legislation, which is also implementing a ban on owning, producing or sharing AI models designed to create child sexual abuse material.

Practical Consequences

This recently, the official visited the London headquarters of Childline and heard a simulated conversation to counsellors featuring a report of AI-based abuse. The interaction portrayed a teenager requesting help after being blackmailed using a sexualised AI-generated image of themselves, constructed using AI.

"When I learn about young people experiencing blackmail online, it is a source of intense anger in me and rightful concern amongst parents," he stated.

Alarming Statistics

A prominent internet monitoring foundation reported that cases of AI-generated abuse material – such as webpages that may contain multiple images – had more than doubled so far this year.

Instances of category A content – the gravest form of exploitation – increased from 2,621 visual files to 3,086.

• Female children were predominantly targeted, accounting for 94% of prohibited AI images in 2025
• Depictions of infants to two-year-olds rose from five in 2024 to 92 in 2025

Industry Response

The law change could "constitute a crucial step to guarantee AI products are secure before they are released," stated the chief executive of the online safety organization.

"Artificial intelligence systems have made it so survivors can be victimised all over again with just a simple actions, giving offenders the ability to create potentially limitless quantities of advanced, lifelike child sexual abuse material," she continued. "Content which further commodifies survivors' trauma, and renders young people, especially female children, more vulnerable on and off line."

Support Session Information

Childline also released information of support interactions where AI has been referenced. AI-related harms discussed in the sessions include:

• Using AI to evaluate body size, body and appearance
• AI assistants dissuading young people from talking to safe adults about harm
• Being bullied online with AI-generated content
• Digital extortion using AI-manipulated images

During April and September this year, Childline delivered 367 counselling sessions where AI, conversational AI and associated terms were mentioned, significantly more as many as in the equivalent timeframe last year.

Half of the references of AI in the 2025 sessions were connected with psychological wellbeing and wellness, encompassing using AI assistants for support and AI therapy apps.