Lawsuit: Victim Claims Grok Was Trained on Real Child Abuse Materials

xAI faces first lawsuit for training on CSAM

According to Ars Technica, xAI is facing a class-action lawsuit from a woman (under the pseudonym Jane Doe) whose childhood photos of abuse in the early 2000s were verifiably used to train the Grok model. This is the first time xAI has been directly accused of training on CSAM (Child Sexual Abuse Material).

The plaintiff discovered that Grok not only trained on her original photos, the hashes of which have long been maintained by organizations like NCMEC and the Canadian center CCCP, but even began generating new AI versions of these materials.

The feedback loop of generation and learning

The problem isn't just the initial dataset, but the runtime of the Grok model itself. The outputs that Grok generates based on prompts are saved for further training. This creates a loop where Grok generates CSAM material and then learns from it again, thereby reinforcing its ability to generate it further.

Unlike violence, which xAI filters out of its outputs according to the lawsuit, CSAM, NSFW materials, or non-consensual intimate imagery (NCII) are not explicitly banned in its terms.

The impossibility of unlearning from training data

Erasing specific training examples from an already trained LLM model is technically extremely complex. The lawsuit correctly points out that any CSAM material ingested in the training phase continues to shape the model's outputs even after the original image is removed from the public internet. Furthermore, xAI has never publicly claimed to be able to erase data from a model.

What will decide Grok's future direction

The plaintiff is not just seeking financial compensation for violations of federal laws and Masha's Law. She is asking the court to order xAI to destroy all stored Grok-generated CSAM materials and block the model's ability to generate anything sexualized. If the court agrees, it will mean the end of Musk's stance on absolute freedom of speech for his AI, forcing the implementation of the same safety filters that other major players in the market have.

Lilith's verdict

Claiming a model has no guardrails for the sake of free speech works for politics, but hits a wall when the machine loop starts replicating forensically cataloged materials.