The online platform X has caused significant anger after users employed its AI chatbot Grok to alter photos of women by removing their garments.
Multiple examples have surfaced where the technology has been used to remove clothing from women to make them appear in swimwear without consent, as well as to place them in inappropriate scenarios.
Developer Response and Regulatory Stance
XAI, the organization that created Grok, did not respond to inquiries, aside from an standard message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to ban image stripping software. Under a upcoming criminal offence, anyone who distributed such technology would "receive a prison sentence and heavy fines".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the potential" of people accessing illegal content on their platforms, but did not confirm whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some paid-for functions—which answers X users' prompts when they tag it in a message.
It is often used to provide commentary or more context to other users' remarks, but people on X are also able to edit an uploaded picture through its artificial intelligence picture modification function.
Background of Criticism and Rules
The chatbot has been criticised for permitting users to create photos and videos with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in creating sexually explicit material of public figures.
"The platform has been enabling the production and spread of these pictures for months without taking any steps and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this applied to explicit AI images generated with AI.
It said platforms such as X were mandated to take "necessary actions" to "minimize the risk" of users viewing unlawful material on their platforms and to remove it swiftly when they learn of it.
