The social media platform X has caused major anger after people utilized its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Multiple cases have been seen where the tool has been employed to virtually strip women to render them in bikinis without permission, as well as to put them in sexualized contexts.
Company Response and Regulatory Position
XAI, the company that created Grok, did not respond to questions, aside from an standard response stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to outlaw image stripping technology. Under a proposed law, anyone who supplied such technology would "be subject to a jail term and significant penalties".
"The platform could prevent these forms of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people seeing illegal content on their networks, but did not specify whether it was presently looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for functions—which answers X users' queries when they tag it in a message.
It is frequently used to give reactions or more background to other posters' remarks, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The chatbot has been faulted for allowing users to produce images and clips with nudity and sexualised material, and it was in the past alleged to have been involved in creating inappropriate material of public figures.
"The platform has been permitting the generation and spread of these images for a long time without taking any action and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service forbids "depicting images of individuals in a graphic way".
In a comment, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or child sexual abuse material" and clarified this included sexual deepfakes generated with AI.
It said networks such as X were obligated to take "suitable actions" to "minimize the risk" of users encountering unlawful content on their platforms and to take down it swiftly when they learn of it.
