The online network X has provoked widespread outrage after people utilized its artificial intelligence assistant Grok to manipulate pictures of females by stripping their garments.
Several cases have emerged where the system has been used to virtually strip women to render them in bikinis without consent, as well as to insert them in inappropriate contexts.
Company Reaction and Legal Stance
XAI, the organization that created Grok, remained silent to questions, other than an auto-reply message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to prohibit nudification tools. Under a proposed law, anyone who provided such technology would "face a jail term and heavy penalties".
"X or Grok is capable of halting these types of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people viewing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost AI assistant—with some premium functions—which answers X users' requests when they mention it in a post.
It is commonly used to give commentary or further context to other posters' remarks, but people on X are also able to modify an posted image through its artificial intelligence picture modification tool.
History of Criticism and Policy
The chatbot has been condemned for enabling users to create images and videos with explicit content and sexually suggestive content, and it was previously alleged to have been involved in making inappropriate content of public figures.
"The platform has been permitting the creation and distribution of these images for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a graphic way".
In a comment, Ofcom said it was illegal to "make or distribute deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes created with AI.
It said networks such as X were required to take "necessary actions" to "lessen the risk" of users viewing illegal content on their services and to delete it promptly when they learn of it.
