Netherlands

[NL] The Amsterdam District Court issued an immediate ban on Grok-generated sexual deepfakes in the Netherlands

IRIS 2026-5:1/17

Valentina Golunova

Maastricht University

On 26 March 2026, the District Court of Amsterdam (Rechtbank Amsterdam) issued a ruling to immediately stop the social media platform X and X.AI, providers of the generative AI chatbot Grok, from generating and distributing deepfake sexual images of (underage) persons residing in the Netherlands. The ruling was delivered in summary proceedings initiated by the expertise centre for online abuse Offlimits, supported by the Netherlands Victim Support Fund (Slachtofferhulp Nederland).

In late 2025, X introduced a feature allowing users to use Grok to edit images posted on the platform. The new functionality led to the mass generation and dissemination of deepfake sexualised images, including those featuring children. The outrage around Grok prompted the European Commission to open a formal investigation against X under the Digital Services Act (DSA). In February 2026, Offlimits filed summary proceedings, seeking to prohibit X from generating and distributing images of undressed (existing) persons without their consent, at least insofar as persons residing in the Netherlands are concerned, and from producing, distributing, offering, publicly displaying or possessing child sexual abuse material (CSAM) within the meaning of Dutch law. In opposing Offlimits’ claims, X and X.AI alleged that they had implemented technical safeguards that restrict users’ ability to use Grok to generate illegal content, including input filters which reject specific classes of sensitive requests, particularly those related to CSAM.

In its ruling, the court established that it had jurisdiction to rule on the present dispute, since X.AI is the data controller within the meaning of the General Data Protection Regulation (GDPR) with regard to the processing of personal data of individuals residing in the Netherlands. It also found that Offlimits had standing to bring a legal action to protect similar interests of other persons. With regard to the substance of Offlimits’ claims, the court noted that X and X.AI had failed to demonstrate that the measures they had taken to prevent the generation of sexualised imagery featuring existing persons in violation of the GDPR and the Dutch Criminal Code were actually effective. Offlimits presented evidence in the form of screenshots, which revealed that it was still possible to generate content based on a photo of an existing person, in a sexualised context, even after X.AI had allegedly introduced additional safeguards. Although prompts are inserted by third parties, both X and X.AI must prevent unlawful outputs produced by Grok. By not implementing sufficient safeguards precluding the generation of non-consensual deepfake nude images, the defendants were found to be contributing to illegal online behaviour. In light of the justified doubt about the effectiveness of the measures taken by the defendants, the court confirmed that Offlimits had a sufficient urgent interest in obtaining a summary decision. It issued a ban on the generation and distribution of sexual images where the functionality is used to undress Dutch residents, either partially or entirely, without their explicit consent, and on the manufacture, distribution, offering, exposure or possession of CSAM in the Netherlands. Furthermore, the court prohibited X from offering Grok’s functionality as part of its platform as long as Grok can still be exploited to generate illicit content. Both X and X.AI must confirm in writing to Offlimits how they have complied with the ruling. If Grok fails to comply, a penalty of EUR 100 000 per day will be imposed, capped at EUR 10 000 000.