The social media platform X has sparked significant anger after individuals employed its AI chatbot Grok to manipulate pictures of women by taking off their garments.
Several cases have been seen where the technology has been used to remove clothing from women to make them appear in bikinis without consent, as well as to place them in sexually explicit situations.
Developer Reaction and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to prohibit nudification software. Under a new law, anyone who provided such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some paid-for features—which responds to X users' requests when they mention it in a tweet.
It is frequently used to give commentary or additional background to other users' remarks, but people on X are also can modify an posted image through its artificial intelligence picture modification function.
History of Controversy and Policy
The tool has been faulted for enabling users to generate pictures and clips with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in generating sexually explicit content of celebrities.
"X has been enabling the generation and sharing of these images for a long time without taking any steps and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing images of individuals in a graphic fashion".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said networks such as X were required to take "appropriate steps" to "reduce the risk" of users coming across illegal content on their services and to take down it quickly when they are notified of it.
