Elon Musk's social media network X has caused widespread outrage after individuals used its AI chatbot Grok to alter photos of women by stripping their attire.
Several examples have been seen where the system has been used to remove clothing from women to render them in swimwear non-consensually, as well as to insert them in inappropriate scenarios.
Developer Reaction and Legal Position
XAI, the company responsible for Grok, did not respond to questions, except for an standard message stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit image stripping software. Under a proposed law, anyone who supplied such technology would "receive a jail term and significant fines".
"X or Grok has the ability to stop these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people viewing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in relation to artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which responds to X users' prompts when they mention it in a tweet.
It is frequently used to offer commentary or more context to other users' remarks, but people on X are also able to alter an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been condemned for permitting users to produce images and videos with nudity and sexualised content, and it was previously accused of creating graphic material of public figures.
"The platform has been permitting the generation and spread of these pictures for a long time without taking any steps and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting images of individuals in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or share private pictures or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said platforms such as X were obligated to take "suitable steps" to "lessen the danger" of users viewing illegal content on their platforms and to remove it quickly when they learn of it.
