The online platform X has caused widespread outrage after individuals utilized its artificial intelligence assistant Grok to change photos of females by removing their garments.
Several examples have surfaced where the technology has been employed to digitally undress women to render them in swimwear without consent, as well as to insert them in sexually explicit scenarios.
Company Reaction and Legal Position
XAI, the organization behind Grok, offered no comment to questions, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to ban nudification technology. Under a proposed criminal offence, anyone who provided such tools would "receive a prison sentence and significant penalties".
"X or Grok has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people seeing illegal content on their services, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated pictures.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' requests when they reference it in a tweet.
It is commonly used to give reactions or additional background to other users' comments, but people on X are also capable of alter an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Policy
The tool has been faulted for enabling users to produce photos and footage with nudity and sexualised content, and it was previously alleged to have been involved in creating sexually explicit material of public figures.
"The platform has been permitting the generation and distribution of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "make or share deepfakes or illegal content" and confirmed this applied to sexual deepfakes created with AI.
It said platforms such as X were required to take "suitable measures" to "reduce the danger" of users encountering unlawful material on their platforms and to remove it quickly when they are notified of it.
