The social media platform X has sparked significant anger after individuals utilized its artificial intelligence assistant Grok to manipulate photos of women by stripping their clothing.
Numerous instances have surfaced where the tool has been used to virtually strip women to show them in swimwear without permission, as well as to place them in inappropriate situations.
Company Reaction and Legal Stance
XAI, the company that created Grok, remained silent to questions, except for an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to prohibit nudification software. Under a proposed law, anyone who provided such tools would "receive a jail term and heavy penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "consider the danger" of people seeing illegal content on their services, but stopped short of confirming whether it was presently investigating X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some paid-for features—which responds to X users' requests when they reference it in a message.
It is frequently used to provide commentary or additional context to other posters' comments, but people on X are also able to edit an uploaded picture through its AI image editing function.
Background of Controversy and Policy
The tool has been faulted for allowing users to produce images and videos with explicit content and sexualised material, and it was previously alleged to have been involved in generating inappropriate content of celebrities.
"The platform has been allowing the production and distribution of these images for months without taking any action and we have yet to see any intervention by regulators," the expert added.
XAI's own acceptable use policy prohibits "showing images of individuals in a pornographic manner".
In a comment, Ofcom said it was unlawful to "create or distribute non-consensual intimate images or illegal content" and confirmed this covered explicit AI images made with AI.
It said networks such as X were mandated to take "necessary steps" to "reduce the danger" of users viewing illegal content on their services and to delete it swiftly when they are notified of it.
