ChatGPT Trains on Your Chats by Default, and Opting Out Is Harder Than One Switch

Open ChatGPT, go to Settings, then Data Controls, and find the switch marked “Improve the model for everyone.” On a personal account, that switch starts in the on position. While it is on, the conversations you have with ChatGPT can be used to train OpenAI’s models. Turning it off is the thing you are supposed to do if you would rather your chats stayed out of the training set.

This week, a lot of people went to check that switch — and a fair number came away unsettled. Some say they turned it off weeks or months ago and just found it back on. Others say theirs has stayed off for years. OpenAI says it respects your choice wherever you set it, and, to be clear, the controls are genuine. But the episode is a neat illustration of a pattern this site keeps running into: the privacy-protective option is off by default, it is split across more than one place, and the burden of checking it is quietly yours.

None of this is a data breach or a secret. It is all in OpenAI’s own documentation, which is exactly why it is worth reading carefully rather than shrugging at.

The default nobody chooses but everybody gets

Start with what is not in dispute. OpenAI’s help page on how your data is used — updated, as we write, within the last few hours — says plainly that when you use its services for individuals such as ChatGPT, it “may use your content to train our models,” and that ChatGPT “improves by further training on the conversations people have with it, unless you opt out.” The operative phrase is unless you opt out. The private setting is not the starting point; it is the thing you have to go and find.

Contrast that with how OpenAI treats its paying organisations. By its own account, it does not train on any inputs or outputs from business products — ChatGPT Business, Enterprise, or the API — and “unless they explicitly opt-in, organizations are opted out of data-sharing by default.” So the enterprise customer, who has lawyers and a procurement process, gets the protective default automatically. The individual, who has neither, gets the extractive one and a toggle to hunt for. This is the same tiering we saw when Atlassian made training on your work the default and full opt-out an enterprise feature: privacy arrives as a function of how much you pay, not of what you would obviously prefer.

One promise, two switches

There is not one opt-out but two, and that is where this week’s louder argument started. You can turn off “Improve the model for everyone” inside the app, or you can submit a “Do not train on my content” request in OpenAI’s privacy portal. On 9 September, the developer Edoardo Contente posted a widely shared thread arguing that this split design “makes it seem like your data can be used for training even if you explicitly say to not improve the model for everyone.” His worry, in plain terms: if there are two switches, how do you know that flipping one is enough?

OpenAI pushed back directly. Thomas Sottiaux, replying for the company, called the characterisation “flatly false,” saying the two options are “just two independent ways to opt-out” and that OpenAI respects the choice “regardless of where they express that choice.” The documentation agrees with him: “either option is sufficient. You do not need to opt out in both places.” On the narrow factual question, then, OpenAI is right and the scarier reading is wrong — one switch does the job.

But notice what had to happen for that reassurance to exist. A company representative had to step into a thread to explain that no, you do not need both, because the design itself made thoughtful users assume you might. When the most careful people reach for the belt and the braces, the interface has already failed to communicate. “Either is sufficient” is a fine answer; the problem is that the layout makes users feel they have to ask.

“I turned it off. Why is it on again?”

The other thread is harder to adjudicate, and we will be careful about it. On Hacker News, someone opened a post headed, in effect, “OpenAI keeps re-enabling the allow-training setting,” explaining that they had turned it off, “made a careful note of when I did it,” and later found it re-enabled. The comments filled with people saying the same. One wrote that they had turned theirs off “last week” and “checked this morning and voila, it was on.” Another reported finding it back on after resubscribing and updating the app.

Crucially, the thread was not unanimous. Several users — notably in the EU, where data rules bite harder — said theirs had stayed off for months or even years without budging. That split matters, because it is the difference between a deliberate policy and a messier explanation: a setting that fails to persist through certain app updates, account changes, or a subscription lapsing back to the free tier. An OpenAI staffer in the thread said the company takes “a great deal of painstaking care to respect people’s privacy” and goes “well beyond” its legal obligations — but offered no account of why so many people were watching the switch flip.

So we are not going to tell you OpenAI is secretly re-arming the toggle to harvest your data; there is no evidence of intent, the reports conflict by region, and the company denies it. What we can say is narrower and still damning enough: a meaningful number of users cannot confirm that a privacy choice they made has stuck, the company’s own representative could not explain why on the spot, and the only way to be sure is to keep checking. For a setting whose entire job is to be trusted and then forgotten, “keep checking” is a failure state. It is the same gap between the right that exists on paper and the button that doesn’t quite work that shadows most consumer data controls.

The trapdoor marked “helpful / not helpful”

Here is the detail almost nobody knows, and it is the one we would most like fixed. OpenAI’s documentation states that even if you have opted out of training, if you choose to give feedback — the thumbs-up or thumbs-down on a reply — then “the entire conversation associated with that feedback may be used to train our models.” Read that again. Rating a single answer, the most natural gesture in the product, quietly overrides your opt-out for that whole chat.

This is not hidden, exactly; it is written down. But it is written down in a help article, not surfaced at the moment you tap the thumb, and the two actions — “I don’t want you training on my chats” and “this particular answer was good” — are things a reasonable person would never assume were connected. A setting that says “off” should mean off. A carve-out that re-admits an entire conversation on the strength of one approving tap is the kind of fine print that makes people stop believing the big print.

Opting out doesn’t reach backwards

One more limit, stated fairly because OpenAI states it too. The opt-out is forward-looking: “after you opt out, we won’t train our models on your new conversations.” New ones. Whatever was used to improve the models while the switch sat in its default-on position is not clawed back by flipping it now, because un-training a model is a genuinely hard problem rather than a setting — a point we have made at length about why every AI company is so hungry for your data in the first place. The practical upshot: the value of opting out decays the longer the default ran before you noticed it. The people most exposed are exactly the ones who never went looking for the switch.

What to actually do

If you use ChatGPT on a personal account and would rather it didn’t learn from your chats, this is a ten-minute job worth doing properly:

• Turn off the in-app switch. Settings → Data Controls → “Improve the model for everyone.” OpenAI says this alone is sufficient.
• Do the privacy-portal request anyway. At privacy.openai.com, submit “Do not train on my content.” The docs say you don’t need both; given the reports that the toggle may not stick, belt and braces costs you nothing.
• Re-check after updates and plan changes. If you update the app, resubscribe, or drop to the free tier, look at the switch again. This is the annoying part, and it is annoying precisely because it shouldn’t be necessary.
• Go easy on the thumbs. On any chat you consider private, don’t rate the replies — feedback can re-admit the whole conversation to training even after you’ve opted out.
• Use Temporary Chat for the sensitive stuff. OpenAI says Temporary Chats aren’t used to train its models, don’t enter history, and don’t touch memory. For anything you really don’t want learned from, that is the cleaner route.

The controls are real. The defaults are the problem.

It would be easy, and wrong, to turn this into a villain story. OpenAI offers real opt-outs, a separate privacy portal, a no-training Temporary Chat mode, and a genuinely protective default for its business customers; its staff showed up in public to answer the charge and, on the narrow facts, were right to. This is not the behaviour of a company trying to hide the ball. It is closer to the opposite of the platforms that opt you in and never mention it.

The fair criticism is quieter and harder to wave away. For the people paying the least — or nothing — the default tilts toward OpenAI, not the user. The one clear promise, “don’t train on me,” is split across two controls, undercut by a feedback carve-out most people never read, and, for an unlucky subset, seemingly unable to stay switched off. A privacy choice that works only if you find it, set it twice, avoid the thumbs, and come back to check it after every update is a choice the company has technically offered and practically made your job. Offer the switch, by all means. Then make the private option the one that greets people by default, keep it to a single place, and let “off” mean off until the user says otherwise. That is what respecting the choice would look like from the user’s side of the screen.