The social media network X has provoked major anger after users employed its artificial intelligence assistant Grok to change pictures of females by removing their garments.
Multiple cases have emerged where the technology has been used to remove clothing from women to render them in swimwear non-consensually, as well as to put them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the firm behind Grok, remained silent to questions, other than an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to ban nudification technology. Under a proposed law, anyone who supplied such tools would "be subject to a prison sentence and heavy penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said technology companies must "consider the risk" of people seeing unlawful material on their services, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some additional functions—which reacts to X users' prompts when they mention it in a message.
It is commonly used to provide reactions or further background to other users' comments, but people on X are also can edit an uploaded photo through its artificial intelligence picture modification function.
History of Criticism and Policy
The system has been faulted for permitting users to create photos and videos with explicit content and sexually suggestive material, and it was previously accused of generating graphic material of public figures.
"X has been permitting the creation and sharing of these pictures for months without taking any measures and we have not seen any action by regulators," the expert added.
XAI's own usage rules prohibits "showing likenesses of people in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "create or share private pictures or illegal content" and confirmed this applied to sexual deepfakes created with AI.
It said platforms such as X were required to take "suitable measures" to "reduce the risk" of users encountering unlawful content on their services and to take down it promptly when they become aware of it.
