Elon Musk's online network X has provoked widespread backlash after people employed its AI chatbot Grok to manipulate photos of women by removing their attire.
Multiple cases have surfaced where the tool has been used to digitally undress female subjects to make them appear in swimwear without consent, as well as to place them in inappropriate scenarios.
Developer Response and Legal Position
XAI, the organization behind Grok, did not respond to requests for comment, other than an auto-reply response stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to outlaw nudification software. Under a upcoming law, anyone who provided such technology would "receive a prison sentence and substantial fines".
"X or Grok is capable of halting these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people accessing illegal content on their platforms, but did not specify whether it was currently investigating X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some additional functions—which responds to X users' queries when they mention it in a tweet.
It is often used to offer reactions or additional context to other users' remarks, but people on X are also capable of edit an posted image through its AI image editing tool.
History of Criticism and Policy
The tool has been criticised for allowing users to produce images and clips with nudity and sexually suggestive content, and it was earlier alleged to have been involved in making sexually explicit material of celebrities.
"X has been enabling the production and distribution of these images for months without taking any action and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing images of persons in a graphic fashion".
In a statement, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and confirmed this included sexual deepfakes made with AI.
It said platforms such as X were mandated to take "necessary steps" to "lessen the danger" of users viewing illegal content on their platforms and to remove it swiftly when they learn of it.
