How Big Tech’s AI Models Are Built on Mass Privacy Invasion, According to Amnesty

Amnesty equates this automated data harvesting with state-level mass surveillance systems, arguing it violates the right to privacy from the very first step of development . The briefing names several major publicly available models and tools in its research, including GPT-3, Gemini, Llama, and others like DeepSeek, Midjourney, and Stable Diffusion .

Three Categories of Harm

To support its call for a prohibition rather than a risk-based regulatory approach, the briefing highlights three distinct but interconnected categories of harm.

1. Mass Invasions of Privacy by Design

The first and foundational harm is the privacy violation itself. The report details how datasets built from billions of public posts, photos, and conversations are compiled without permission. This isn't a leak or a breach; it is the intentional, structural method for creating the AI. Amnesty argues this "abuse of privacy rights" is the starting point, making the entire development pipeline illegal under human rights standards .

2. Perpetuating Stereotypes, Biases, and Discrimination

The second category of harm flows directly from the data. Because the training material is scraped indiscriminately from the open web, it is "polluted with real-world biases." Amnesty warns that these biases are not neutralized by the AI; instead, they are "amplified in model outputs along racial, gender, and cultural lines" . The report expresses particular concern for historically marginalized communities, who face the brunt of this systemic discrimination as toxic stereotypes and prejudiced associations are automated and scaled .

3. Adverse Environmental Impact

The final category of harm shifts focus from the digital to the physical. The briefing documents the significant ecological cost of building and running these systems, pointing to the "enormous data centres and energy consumption required" . This environmental impact, the report argues, is an additional and often overlooked human rights consequence of the unchecked AI race.

A Call for Prohibition, Not Just Regulation

Amnesty International's conclusion is stark. It finds that standalone generative AI systems, dependent on these unlawful web-scraping pipelines, are "incompatible with IHRL" . The organization argues that existing regulatory models, like the risk-based framework of the EU's AI Act, are insufficient . Instead, it calls on governments to intervene with a "full prohibition" on systems built through these methods, framing the entire practice as a human rights crisis that cannot be merely managed .