OpenAI compensates hundreds of contractors to examine genuine ChatGPT exchanges and assess the quality of responses through an initiative internally called Project Lily, according to 404 Media. When pressed to identify where the company notifies users about this human review process, OpenAI declined to respond directly, only later directing to a help page after the story appeared.
The Court of Justice of the European Union addressed this exact scenario last September in its EDPS v SRB ruling. The court determined that organizations collecting data must inform individuals before transferring that information elsewhere, and this obligation is assessed based on the collector's own perspective rather than whether the recipient could identify specific people.
Contractors performing the reviews do not have access to user identities. However, they can view a summary of user memories positioned above the prompt, which may reveal previous uses of the chatbot and geographic location. OpenAI states that a Privacy Filter model removes sensitive details beforehand, though the company's own documentation acknowledges the model can fail to catch unusual identifiers and may leave information exposed when context is sparse.
Other AI companies handle human review differently. Anthropic disclosed it employs human reviewers for users who enable this feature and strips account identifiers before review. Google's Gemini includes a notice stating that humans may examine certain saved conversations.
Italy's data protection authority has already taken enforcement action, imposing a EUR 15M penalty on OpenAI—EUR 9M specifically for processing without sufficient legal justification—and mandating six months of public awareness campaigns on Italian television and radio.
The default settings reveal a split approach. The "Improve the model for everyone" option, which enables human review, activates automatically for free, Plus, and Pro subscribers but remains disabled for Enterprise, Business, and Edu accounts. Disabling the setting only affects future conversations, not past ones.
OpenAI has positioned privacy as a premium feature throughout the year, unveiling zero data retention options for enterprise users in August. Standard consumers receive the setting that contributes to model improvement.
I don't think they would imagine some contractor somewhere is analyzing the conversations
one reviewer
The same company has simultaneously requested that these users authorize access to their bank accounts.
Source: The Next Web
