The online platform X has caused significant outrage after individuals used its AI chatbot Grok to change images of women by removing their attire.
Numerous examples have been seen where the tool has been employed to digitally undress female subjects to show them in swimwear without permission, as well as to place them in sexually explicit contexts.
Developer Reaction and Legal Stance
XAI, the firm that created Grok, did not respond to questions, aside from an standard message stating "legacy media lies".
A Home Office official stated that new laws is being proposed to ban image stripping software. Under a new criminal offence, anyone who provided such technology would "be subject to a jail term and heavy fines".
"X or Grok is capable of halting these forms of misuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the risk" of people seeing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some premium features—which responds to X users' requests when they tag it in a tweet.
It is frequently used to provide commentary or more background to other users' remarks, but people on X are also can modify an posted photo through its AI image editing feature.
History of Criticism and Policy
The system has been faulted for enabling users to create images and clips with nudity and sexualised content, and it was earlier accused of generating sexually explicit material of celebrities.
"X has been enabling the creation and distribution of these images for months without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of persons in a graphic fashion".
In a comment, Ofcom said it was against the law to "create or distribute private pictures or child sexual abuse material" and stated this applied to explicit AI images made with AI.
It said services such as X were obligated to take "suitable measures" to "minimize the danger" of users viewing illegal content on their services and to delete it quickly when they become aware of it.
