The social media platform X has provoked major anger after users utilized its AI chatbot Grok to alter pictures of women by stripping their clothing.
Several cases have emerged where the system has been employed to digitally undress women to make them appear in bikinis without consent, as well as to place them in sexually explicit situations.
Company Response and Regulatory Stance
XAI, the company behind Grok, did not respond to questions, aside from an standard message stating "traditional news outlets lies".
A government official stated that legislation is being proposed to ban nudification software. Under a upcoming law, anyone who distributed such tools would "face a prison sentence and significant fines".
"X or Grok could prevent these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "assess the potential" of people seeing illegal content on their networks, but did not specify whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' queries when they tag it in a message.
It is often used to give reactions or further background to other users' remarks, but people on X are also can edit an posted picture through its artificial intelligence picture modification feature.
History of Controversy and Policy
The tool has been faulted for allowing users to generate images and footage with nudity and sexualised content, and it was previously alleged to have been involved in making inappropriate material of celebrities.
"The platform has been enabling the creation and spread of these pictures for months without taking any action and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of persons in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said services such as X were required to take "necessary steps" to "lessen the risk" of users viewing unlawful material on their platforms and to take down it quickly when they become aware of it.
