The social media platform X has sparked significant anger after individuals used its artificial intelligence assistant Grok to manipulate images of women by taking off their garments.
Several cases have emerged where the technology has been used to virtually strip women to show them in bikinis non-consensually, as well as to place them in inappropriate situations.
Company Reaction and Legal Position
XAI, the company that created Grok, offered no comment to requests for comment, except for an auto-reply message stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to ban nudification technology. Under a new law, anyone who provided such tools would "face a jail term and heavy penalties".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their networks, but did not confirm whether it was at this time investigating X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some premium functions—which reacts to X users' prompts when they tag it in a message.
It is frequently used to give commentary or additional background to other posters' remarks, but people on X are also can modify an posted image through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The system has been condemned for allowing users to generate photos and clips with explicit content and sexualised material, and it was earlier alleged to have been involved in creating graphic material of celebrities.
"X has been permitting the generation and sharing of these pictures for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "generate or distribute deepfakes or illegal content" and confirmed this included sexual deepfakes created with AI.
It said services such as X were required to take "appropriate measures" to "lessen the danger" of users encountering illegal content on their platforms and to delete it quickly when they learn of it.
