🔗 Share this article

The online network X has sparked significant anger after individuals utilized its AI chatbot Grok to change photos of females by removing their attire.

Several instances have been seen where the tool has been used to remove clothing from female subjects to make them appear in swimwear non-consensually, as well as to insert them in sexualized scenarios.

Developer Response and Legal Stance

XAI, the firm behind Grok, did not respond to questions, aside from an automatically-generated message stating "traditional news outlets lies".

A government official stated that new laws is being proposed to ban image stripping software. Under a proposed criminal offence, anyone who supplied such tools would "be subject to a jail term and significant penalties".

"X or Grok has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."

Oversight Body Stance and Tool Functionality

The regulator Ofcom said tech firms must "consider the danger" of people viewing illegal content on their networks, but did not confirm whether it was presently looking into X or Grok in regarding AI-generated pictures.

Grok is a no-cost chatbot—with some premium functions—which responds to X users' queries when they mention it in a post.

It is frequently used to offer commentary or further background to other posters' remarks, but people on X are also able to alter an uploaded photo through its artificial intelligence picture modification tool.

Background of Controversy and Rules

The chatbot has been criticised for allowing users to produce pictures and clips with explicit content and sexualised material, and it was previously alleged to have been involved in making graphic material of public figures.

"X has been enabling the generation and spread of these pictures for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.

XAI's own usage rules bans "showing likenesses of persons in a pornographic fashion".

In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and stated this applied to explicit AI images created with AI.

It said services such as X were required to take "appropriate measures" to "minimize the danger" of users coming across unlawful content on their services and to remove it promptly when they become aware of it.