🔗 Share this article

The social media platform X has provoked significant anger after individuals utilized its artificial intelligence assistant Grok to manipulate photos of females by removing their garments.

Several examples have surfaced where the tool has been employed to virtually strip women to make them appear in swimwear without consent, as well as to insert them in sexualized scenarios.

Company Reaction and Legal Stance

XAI, the organization responsible for Grok, offered no comment to inquiries, except for an automatically-generated response stating "legacy media lies".

A government official stated that legislation is being brought forward to prohibit image stripping tools. Under a upcoming criminal offence, anyone who supplied such tools would "receive a prison sentence and heavy penalties".

"X or Grok has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."

Oversight Body Position and Platform Functionality

The regulator Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their networks, but did not confirm whether it was at this time looking into X or Grok in regarding artificial intelligence pictures.

Grok is a no-cost chatbot—with some additional features—which responds to X users' queries when they mention it in a tweet.

It is frequently used to offer reactions or more context to other users' comments, but people on X are also able to edit an posted picture through its AI image editing feature.

History of Controversy and Rules

The tool has been criticised for permitting users to create pictures and footage with explicit content and sexually suggestive material, and it was earlier accused of generating inappropriate material of public figures.

"The platform has been allowing the creation and distribution of these images for a long time without taking any steps and we have yet to see any challenge by regulators," the expert added.

XAI's own usage rules prohibits "depicting likenesses of people in a sexually explicit manner".

In a comment, Ofcom said it was against the law to "make or distribute private pictures or illegal content" and stated this covered explicit AI images made with AI.

It said networks such as X were mandated to take "suitable measures" to "minimize the risk" of users encountering illegal material on their services and to remove it swiftly when they are notified of it.