The social media platform X has sparked major outrage after individuals utilized its AI chatbot Grok to manipulate images of females by removing their attire.
Numerous instances have been seen where the technology has been used to digitally undress female subjects to make them appear in swimwear without permission, as well as to put them in inappropriate scenarios.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to inquiries, aside from an automatically-generated message stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to prohibit image stripping software. Under a new criminal offence, anyone who provided such technology would "be subject to a prison sentence and significant fines".
"X or Grok is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said technology companies must "consider the danger" of people accessing unlawful material on their platforms, but did not confirm whether it was at this time looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some premium functions—which reacts to X users' queries when they mention it in a tweet.
It is frequently used to give commentary or additional context to other posters' remarks, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification feature.
History of Criticism and Policy
The system has been condemned for enabling users to create photos and footage with nudity and sexualised content, and it was in the past accused of creating inappropriate content of celebrities.
"X has been allowing the generation and sharing of these pictures for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules forbids "portraying images of people in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "make or share deepfakes or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary actions" to "lessen the danger" of users coming across illegal content on their services and to remove it swiftly when they become aware of it.
