• Horizon AI
  • Posts
  • Humans may be reading your ChatGPT conversations 🚨

Humans may be reading your ChatGPT conversations 🚨

"The Last AI Built by Humans": Chinese Tech Firms Race Toward AI That Can Improve Itself 👀

Author

Gina Acosta
September 16, 2026

In partnership with

Welcome to another edition of Horizon AI,

In today's issue, we take a look at a report from 404 Media that reveals how OpenAI employs hundreds of human workers to read, summarize, and critique real user conversations. We'll also show you how to opt out, so you can keep your future chats private.

Let’s jump into it!

Read Time: 4.5 min

Here's what's new today in the Horizon AI

  • OpenAI Pays Contractors to Read Real ChatGPT Conversations

  • Chinese Researchers Map Out 5-Stage Path to 'Last AI Built by Humans'

  • AI Tutorial: How to Stop OpenAI from Using Your Chats

  • AI Tools to check out

  • AI Findings/Resources

  • The Latest in AI and Tech 💡

AI News

OPENAI

OpenAI Pays Contractors to Read Real ChatGPT Conversations

According to an investigation by 404 Media, OpenAI is using human contractors to review real ChatGPT conversations as part of its model training and improvement process.

Details:

  • The internal initiative, reportedly called "Project Lily," involves contractors reading, summarizing, and evaluating ChatGPT conversations.

  • Contractors are reportedly paid through third-party staffing firms and assess multiple AI responses with the goal of improving the responses ChatGPT gives its users.

  • Reviewers are instructed to flag overly flattering responses (sycophancy), unnecessary AI-style language ("AI-speak" and "emoji misuse"), and claims that suggest the model has human experiences (such as "As a chef, I like to…" or "I know what that's like.")

  • OpenAI says usernames are not provided to reviewers, and conversations pass through an automated privacy filter designed to remove identifying information.

  • However, OpenAI acknowledges that its privacy filter can make mistakes, including missing uncommon identifiers or ambiguous personal details.

  • 404 Media reports that reviewers can also see a "user memories summary" in some cases, potentially providing additional context about a user's previous interactions, which can inadvertently reveal their general location, profession, or personal life context.

OpenAI isn't the only AI lab doing this. Anthropic confirmed that it uses human reviewers when users have turned on the "Help improve Claude" setting in their privacy settings, while Google's Gemini clearly posts a disclaimer that says, "Humans review some saved chats to improve Google AI."

TOGETHER WITH ATLASSIAN

AI made PMs faster. Multiplayer mode is still broken.

A PM can summarize research, draft a PRD, and mock up a prototype before lunch. The hard part starts when the team has to decide what actually gets built.

Jira Product Discovery gives product teams one place to capture insights, prioritize ideas with consistent frameworks, and build living roadmaps stakeholders can rally around.

And because it’s connected to Jira, the context behind every decision stays with the work—so developers and their agents know not just what to build, but why.

AI helps PMs move faster. Jira Product Discovery helps the whole team build with confidence.

SELF IMPROVING AI

Chinese Researchers Map Out 5-Stage Path to 'Last AI Built by Humans'

Researchers from ByteDance, Tsinghua University, and the Shanghai Artificial Intelligence Laboratory have proposed a five-stage roadmap for AI systems that could improve themselves with increasingly less human intervention, and eventually, almost none at all.

Details:

  • The study, titled "The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement," focuses on recursive self-improvement (RSI), where improvements persist across tasks and can be inherited by successor AI systems.

  • The first stage involves AI carrying out improvement procedures designed by humans, while later stages give models more control over how they improve themselves.

  • More advanced stages would allow AI to determine what information or experiences it needs and adapt its learning after deployment.

  • The final stage envisions systems that can continuously improve the very methods used to improve AI itself.

  • The researchers said progression through the five stages would vary by field, with software engineering presenting a relatively clear path forward and robotics and scientific discovery facing steeper hurdles.

  • They argue that automating AI research could reduce development time, labor requirements and computing costs for foundation models.

Safety remains a major requirement. The researchers said that genuine RSI required strict safeguards, including verified testing environments to ensure updates were safe and beneficial before being deployed.

AI Tutorial

How to Stop OpenAI from Using Your Chats

According to 404 Media, OpenAI told the outlet that users' chats won't be used to improve the company's models if they turn off the "Improve the model for everyone" setting.

This is turned on by default for Free, Plus, and Pro plans, so users need to proactively turn it off if they wish to opt out. Enterprise, Business, and Edu customers have the model-improvement setting off by default.

To keep your future conversations from reaching human reviewers:

  1. Go to ChatGPT and sign in.

  2. Click your profile icon in the bottom corner and select Settings.

  3. Select Data Controls.

  4. Toggle the "Improve the model for everyone" switch to Off.

Note: The opt-out isn't retroactive. It only applies to new conversations going forward; prompts that have already been de-identified and queued in the system stay part of the training cycle. Also, turning this off doesn't stop automated scans for Terms of Service or safety violations.

AI Tools to check out

🌙 Dreambeans: A Google Labs experiment that uses the company's AI to create illustrated daily stories based on your interests, drawing on the data Google holds on you.

📚 myAIcademy: Learn AI skills for your specific role and team.

🚀 Mastra: Build, observe, and improve agents that run for days.

🔍 Consensus: An AI academic search engine for peer-reviewed literature—your research OS for finding, organizing, and analyzing science 10x faster.

✨ Figma Weave: AI powered design workflows built for creative pros.

AI Findings/Resources

🏡 Now you can scan an entire house with your phone, and anyone on Earth can walk through it in a browser tab

🔨 This Reddit user says he managed to build his first ever PCB, entirely designed by Claude Fable 5

The latest in AI and Tech

The company has introduced Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking, two new audio models designed to more effectively handle interruptions, understand visual context, and support conversations across 97 languages.

Gemini 3.8 Live focuses on speed and efficiency, while the Extended Thinking version is designed for more complex, multi step tasks.

Meta has introduced Meta One, a subscription service offering expanded access to AI tools and premium features across Facebook, Instagram, and WhatsApp.

Two new AI hotlines have launched to give AI agents a way to report suspicious behavior by other agents to humans.

Meta CEO Mark Zuckerberg has distanced the company from calls by leaders at Anthropic, OpenAI, Google, and SpaceXAI for a coordinated slowdown in AI development. He said each AI lab should individually determine the pace needed to train its models safely.

That’s a wrap!

Thanks for sticking with us to the end! Let’s stay connected on LinkedIn and Twitter.

We'd love to hear your thoughts on today's email!

Your feedback helps us improve our content

Login or Subscribe to participate in polls.

Not subscribed yet? Sign up here and send it to a colleague or friend!

See you in our next edition!

Gina 👩🏻‍💻