The online network X has provoked major outrage after individuals used its AI chatbot Grok to change images of females by removing their clothing.
Numerous cases have emerged where the tool has been employed to remove clothing from women to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Company Response and Regulatory Position
XAI, the organization behind Grok, did not respond to requests for comment, except for an auto-reply response stating "traditional news outlets lies".
A government official stated that new laws is being introduced to ban image stripping tools. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a prison sentence and substantial penalties".
"The platform could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people seeing unlawful material on their services, but stopped short of confirming whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some additional features—which reacts to X users' queries when they reference it in a post.
It is often used to offer commentary or more background to other users' remarks, but people on X are also can modify an uploaded picture through its AI image editing function.
Background of Controversy and Rules
The tool has been faulted for allowing users to generate pictures and videos with explicit content and sexualised material, and it was earlier accused of generating inappropriate material of public figures.
"X has been enabling the production and spread of these pictures for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of persons in a sexually explicit way".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or illegal content" and confirmed this included explicit AI images made with AI.
It said networks such as X were mandated to take "appropriate actions" to "reduce the danger" of users coming across illegal material on their services and to delete it promptly when they learn of it.
