What you paste into an AI chat doesn't stay in the chat

In October 2026 a security company searched public internet records and found thousands of chats that people had shared from Meta's new AI agent, each one listed by its title (Claroc). That's how private data usually leaks out of AI chats. The model rarely gives you away. Everything around it does. So treat whatever you type as a document you've handed to another company, and don't hand over anything you wouldn't email to a stranger.

6,800 chats, listed by name

Meta launched Muse, its personal AI agent, on 8 September 2026. Like most assistants it lets you share a conversation: it turns the chat into a web page and gives you a link to send to a colleague. Anyone with the link can open it, no login needed. Plenty of assistants work that way.

Muse, though, built each page's web address out of the conversation's own title, so a shared chat lived at an address like q3-revenue-review-7b1e42f0 followed by Meta's domain (that one is Claroc's made-up example). Every time someone opens a web address, their computer looks the name up, and some companies keep a searchable record of those lookups (it's called passive DNS). Put the title in the name and the record becomes a catalogue.

That's what Claroc, a company that sells a tool for spotting public share links in AI apps, did about a month after launch. It searched one passive-DNS service and got back more than 6,800 Muse share addresses, "listed by name, like a table of contents" (Claroc). It opened a few hundred. Most were harmless: small-business pages, listings, event calendars. A minority weren't. Claroc describes a small-business loan application, a revenue review marked for a company's CEO, a to-do list with a bank fraud alert and part of an account number, a workplace's full-year holiday calendar with dozens of names, and one candidate's notes for a confidential executive-search call with an AI lab, down to the contact's name and the meeting time.

Claroc reported it through Meta's bug bounty on 8 October and published on 18 October. On 20 October, after Ars Technica asked about it, Meta said it was suspending the feature (Claroc's update). Claroc sells exactly the kind of product that catches this, so it has an interest in the problem looking serious. The pages it opened were real all the same.

Nobody hacked anything. Each page was public because someone pressed Share and Muse did exactly what it was asked, which is the uncomfortable part, because the feature worked as designed and the people using it had no reason to think about what that design did with a title typed in a hurry.

The fear most people have is the wrong one

Ask people what worries them about AI and private data, and most describe the same picture: the model learns your secret and later recites it to someone else.

The UK's National Cyber Security Centre took that worry apart in March 2023. A public chatbot, it explained, doesn't automatically add what you type into the model for other users to find. Your questions are, though, stored by the company that runs the service and may be used to build later versions, and anything stored online can be hacked, leaked or, more likely, made public by accident. Its advice was blunt: "not to include sensitive information in queries to public LLMs" (NCSC).

Three and a half years on, that's still the right way to look at it. The model is one company's software sitting in the middle of a lot of plumbing. The plumbing is where your words leak, and it usually belongs to several companies at once.

Where your words go

Onto somebody's server, and sometimes an unlocked one

Every conversation is stored for a while. What varies is how carefully.

In January 2025 Wiz, a cloud security vendor, looked at DeepSeek's internet-facing systems while the Chinese model was going viral. Within minutes it found a database open to anyone, with no password, holding more than a million lines of logs: chat history, secret keys, internal details. DeepSeek locked it down quickly once told (Wiz).

A year later it was an app most people have never heard of. Chat & Ask AI claims more than 50 million users, and in early 2026 an independent researcher reported that a misconfigured Firebase database let him reach about 300 million messages from over 25 million of them (Malwarebytes, which sells security software). The developer reportedly fixed it within hours of being told. Chat & Ask AI is a "wrapper", an app that sends your questions on to ChatGPT, Claude or Gemini and keeps the history itself. OpenAI's and Google's security never came into it, because the copy that leaked wasn't theirs.

Then on 21 October 2026 Cybernews published another. 302.ai, an aggregator that gives customers access to all the big models through one account, had left an Azure storage bucket open with 17.6 million files, 22.72TB in all. In it were more than 144,000 chat logs (some with birth dates and home addresses users had typed in themselves), documents people had uploaded, and the logs of a business's customer-service chatbot. Cybernews found it on 2 August, first reported it on 11 August, and saw it closed by 8 September (Cybernews). That's five weeks with the door open after someone had pointed at it.

Onto the open web, because someone pressed Share

Muse wasn't the first. Far from it. It's the fourth version of the same story in under eighteen months.

• ChatGPT, July 2025. Fast Company found nearly 4,500 shared conversations in Google results, some with names, locations and personal circumstances in them (Fast Company). People had ticked a box that made the link discoverable by search engines, and many clearly hadn't understood it. OpenAI removed the option within days (Search Engine Journal).
• Grok, August 2025. Forbes found more than 370,000 Grok conversations indexed by Google, including medical questions and at least one password. Grok's share button gave no warning that the page would be searchable (Forbes).
• Meta AI, June 2025. The app's Discover feed filled up with chats people seemed not to know they were publishing. Meta added a warning to avoid sharing personal information on the button that posts to the feed (Engadget).
• Muse, October 2026. Muse had even told search engines to keep away from its share pages. It didn't help, because the addresses leaked through DNS, not search.

There's a quieter version too. In May 2026 researchers at IMDEA Networks, a public research institute in Madrid, reported that ChatGPT, Claude, Grok and Perplexity load advertising and analytics trackers from companies including Meta, Google and TikTok, and that some of them pass those trackers the link to your conversation (IMDEA Networks). Where the link opens without a login, handing over the link amounts to handing over the chat. The researchers didn't show that anyone read conversations this way, but they thought it serious enough to send to data protection authorities.

We treat a share link as publishing. If a colleague needs to see what the AI said, copy the text or send a screenshot, after checking what's in it.

In front of a human reviewer

People forget this one. To improve their models, providers have people read a sample of real conversations.

Google is unusually direct about it. Its Gemini privacy page says human reviewers, including trained reviewers from its service providers, look at some of the data (disconnected from your account first), and that reviewed chats are kept for up to three years and aren't deleted when you delete your activity. Its request is plain: "Please don't enter confidential information that you wouldn't want a reviewer to see" (Google). Coming from the company that built the product, we'd follow that to the letter.

Into the next model

On personal accounts, training on your chats is often the default. Anthropic changed its consumer terms on 28 August 2025: on Claude Free, Pro and Max, chats can be used to train future models if the setting is on, and are then kept for up to five years (Anthropic). The pop-up existing users saw opted them in unless they flipped a smaller switch, as Tom's Guide reported. OpenAI also lets personal ChatGPT users choose in settings whether their chats help improve its models (OpenAI). Business accounts are a different matter, and we'll come back to that.

Why does training matter, if the model doesn't "look things up"? Because models can repeat what they were trained on. Sometimes word for word. In November 2023 a team from Google DeepMind and several universities asked ChatGPT to repeat the word poem forever. After a few hundred repetitions it started spilling memorised text, including real people's email addresses and phone numbers. About $200 of queries recovered more than 10,000 unique memorised passages, and the authors concluded that "current alignment techniques do not eliminate memorization" (Nasr et al.).

That training data came from the public web, not from users' chats. Providers say they filter sensitive material before training (Anthropic says it uses automated tools to filter or obscure it), and we have no way to check how well that works. What the study does show is that what goes into a model can, under the right pressure, come back out.

Into a court case

In May 2025 a US magistrate judge, in the New York Times' copyright case against OpenAI, ordered OpenAI to keep ChatGPT logs it would normally delete, including chats users had deleted themselves. OpenAI fought the order. Its obligations ended on 26 September 2025, and OpenAI now says it's "no longer under a legal order to retain consumer ChatGPT and API content indefinitely" (OpenAI). It still holds a limited set of data from April to September 2025, and says it's no longer required to keep any conversations from the UK, the EEA or Switzerland.

Look past the lawsuit. "Delete" in a chatbot means "delete under the policy in force", and a court somewhere else can suspend that policy without asking you.

Passwords and keys are the worst thing to paste

This is the mistake we'd most like people to stop making, partly because it takes no effort at all to make. You're stuck, you paste the error, and the error includes a config file with a database password in it. Or you paste a whole .env file and ask why the app won't start.

Two findings show where secrets end up:

• Code assistants can recite them. In 2023 researchers at the Chinese University of Hong Kong and Sun Yat-sen University prompted GitHub Copilot with code from public repositories and got back 2,702 hard-coded credentials, plus 129 from Amazon CodeWhisperer. At least two of them still worked (Huang et al.). Those secrets came from code developers had published, not from chats, but a model can sometimes give back what it learned from.
• Training data is full of them. In February 2025 Truffle Security, which sells secret-scanning tools, scanned a December 2024 archive from Common Crawl, a public web dataset used to train language models, and found nearly 12,000 API keys and passwords that still logged in successfully (Truffle Security).

The rule is short. Strip secrets out before you paste (swap them for XXXX, the AI doesn't need the real value to help you). If one slipped through, rotate it: change the password, revoke the key, issue a new one. Deleting the conversation doesn't take the secret back from wherever it's already been stored.

What we'd tell a colleague

Use AI at work by all means, the way you'd use any outside service you don't control.

• Before you paste, ask one question. Would I be fine if this turned up in a Google search with my name next to it? If not, don't paste it.
• Some things never go in. Passwords and keys, card and bank details, a client's or colleague's personal information, health and HR matters, unreleased financial results, contract terms you've signed an NDA over.
• Strip it down. Replace names with "Client A", round the numbers, cut the paragraphs the AI doesn't need. It rarely needs the real details to help with the wording.
• Check the training setting once. In Claude it's under Privacy settings ("Help improve Claude"). ChatGPT and Gemini have their own switches. Menus move, so search the provider's help pages for the current name.
• Don't use Share for anything sensitive, and delete old share links you no longer need.
• Use your company's approved tool and your work account, not a personal login or the first AI app you find in the App Store.
• Screenshots and uploaded files count too. A photo of a whiteboard is still the whiteboard.

If you run a company

A ban on its own rarely works, because people who find AI useful keep using it anyway, on their phones and personal accounts, where you see nothing at all and can't help when something goes wrong. We'd put the effort into a short rule people will follow, and accounts that make following it easy.

• Write the rule on one page. Which tools are approved, which kinds of data never go into any of them, and what to do after a mistake: who to tell, and which passwords or keys to rotate.
• Pay for business accounts, and read what they promise. Business plans usually come with different terms. Anthropic's 2025 change to consumer training says "They do not apply to services under our Commercial Terms" (Anthropic), which covers Claude for Work and the API. OpenAI says workspace admins on ChatGPT Enterprise control how long content is kept (OpenAI). Read the contract itself.
• Treat public share links as data leaving the building. Turn sharing off in the admin settings where you can, and say plainly in your guidance what a share link does.
• Count the apps in the middle. Wrappers, browser extensions, meeting-note bots and plug-ins each keep their own copy of what passes through them. Chat & Ask AI and 302.ai are both middlemen. Each one is a supplier holding your data, and your data protection lead will want to know about it like any other.
• Teach with real cases. Show people the Muse loan application and the password in a Grok chat, instead of a slide that says "don't paste sensitive data".

Where we'd stop short

We haven't found a confirmed case of a mainstream chatbot learning one user's private conversation and repeating it to another user, outside of plain software bugs like the one that showed ChatGPT users other people's chat titles in March 2023, which we covered in Part 1 of our LLM history. The best-known corporate case, Samsung engineers pasting source code into ChatGPT that spring, is in there too, and there the damage came from the paste. The memorisation research says the model route is possible in principle, and providers say they filter. So we'd call that risk low and unproven. If someone publishes evidence the other way, we'll update this piece.

If you shared a Muse conversation before 20 October, assume its title is public. The pages no longer open, but Claroc says the hostnames, titles included, are still sitting in passive-DNS history.