The social media platform X has caused major anger after users used its AI chatbot Grok to manipulate pictures of women by stripping their garments.
Numerous instances have surfaced where the technology has been used to digitally undress women to make them appear in bikinis non-consensually, as well as to insert them in sexualized scenarios.
Developer Response and Legal Position
XAI, the firm that created Grok, did not respond to requests for comment, aside from an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to ban nudification software. Under a upcoming law, anyone who provided such tools would "face a jail term and substantial penalties".
"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people seeing unlawful material on their platforms, but did not specify whether it was presently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' queries when they reference it in a message.
It is commonly used to offer commentary or additional context to other users' comments, but people on X are also capable of alter an uploaded image through its artificial intelligence picture modification function.
History of Criticism and Rules
The chatbot has been criticised for enabling users to produce pictures and footage with explicit content and sexually suggestive content, and it was in the past accused of making graphic material of public figures.
"The platform has been enabling the creation and distribution of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting likenesses of people in a graphic way".
In a statement, Ofcom said it was illegal to "make or circulate deepfakes or illegal content" and confirmed this applied to sexual deepfakes created with AI.
It said platforms such as X were obligated to take "appropriate measures" to "minimize the danger" of users encountering illegal content on their services and to remove it quickly when they learn of it.
