This decision arises as the company confronts threats of fines, regulatory actions, and even a potential ban of the social media platform X in the UK.
“Image generation and editing are currently limited to paying subscribers,” Grok stated in an automated message.
This action follows incidents where users prompted Grok’s integrated AI chatbot to undress women and children in photos or alter them into sexually explicit images, which were subsequently shared online without consent. Grok was also reportedly used to create disturbing imagery portraying women being shot and killed.
Paid subscribers with access must input full personal and credit card information, making it easier to identify any misuse of the feature.
Grok’s image-editing capability was introduced in December.
Despite ongoing public demands for restrictions or removal of the feature, no steps have been taken thus far.
After the Indian government held the social media company accountable for “unlawful” content last week, X dismissed claims of systemic flaws, attributing the issues to user behavior.
As reported by The Wall Street Journal, Genevieve Oh, a social media and deepfake researcher, noted that Grok generated approximately 7,750 sexually suggestive or nudifying images per hour in an analysis of content on X.
UK Prime Minister Keir Starmer indicated that severe actions could be taken against X if it does not curb the spread of harmful AI-generated content. He urged X to “get a grip” on what he termed a surge of AI-generated images of partially clothed women and children, labeling the content “disgraceful” and “disgusting.”
The government is poised to utilize the full extent of the Online Safety Act, as reported by The Telegraph. The law grants regulators the power to impose substantial penalties, potentially amounting to billions of pounds, on technology companies that fail to adequately prevent the distribution of illegal and harmful material.
Concerns intensified after the Internet Watch Foundation (IWF), a charity focused on eliminating child sexual abuse material from the internet, reported the discovery of “criminal imagery” of girls aged 11 to 13 that “appears to have been created” using Grok, according to the BBC.
The IWF indicated it found “sexualized and topless imagery of girls” on a dark web forum where users claimed to have utilized Grok to generate the images.
