AL+

RO EN

Email us

Does your SaaS vendor train AI on your customer data?

The Slack AI debate showed why SaaS buyers must read agreements and referenced policies together. Six checks for model training, opt-outs and data exports.

. By . AI and agentic engineering

Citiți articolul în română

In May 2024 a lot of companies found out, at the same time, that the chat tool holding their internal conversations had been analysing those conversations to build models, and that the way to stop it was to send an email. Slack had changed nothing before the thread appeared. The terms had been on a policy page for months, and few people had read them.

Two and a half years on, the useful lesson is about where the answer lives rather than whether Slack did something sinister. Slack and Zoom show why buyers must read the agreement and its referenced policies together. These two cases do not establish how frequently training is enabled by default across SaaS. A company should treat AI data-use terms as part of choosing a vendor, get the answer in writing, and know how it would leave. An opt-out email is a request you cannot observe being carried out, which makes it a poor substitute for a control.

What the page said

On 16 May 2024 someone posted a bare link to Slack's privacy principles on Hacker News under the title "Slack AI Training with Customer Data". The page, as archived that day, said: "To develop AI/ML models, our systems analyze Customer Data (e.g. messages, content, and files) submitted to Slack as well as Other Information (including usage information)." To be excluded from Slack's global models, a customer had to "have your Org or Workspace Owners or Primary Owner contact our Customer Experience team at feedback@slack.com" with a set subject line.

TechCrunch reported the next day that, according to the Internet Archive, these terms had applied since at least September 2023. Adam Engst at TidBITS later traced the document back further, to an October 2020 version titled "Privacy principles: search, learning and intelligence", written before the generative AI boom.

Slack answered on 17 May with a blog post saying the wording "could create misunderstandings". Its traditional models, used since 2017 for search ranking, channel and emoji recommendations and autocomplete, rely on "de-identified, aggregate data" and "do not access message content". No customer data was used to train the large language models behind its paid Slack AI add-on. And the sentence that matters most for this article: "workspaces are not opted out by default."

The counterargument deserves its full weight

The strongest case against the outrage was made at the time, and it is a good one. Engst's TidBITS piece argued that people reacted to a link and a screenshot without reading the document. In a correction he added to the comments, he conceded he had first read the version Slack revised on 18 May. Even so, the version that set off the row described channel suggestions, search ranking, autocomplete and emoji suggestions, none of them generative, and it already said that global models were not trained "in such a way that they could learn, memorize, or be able to reproduce some part of Customer Data". His reading was that nobody had built a setting because nobody expected anyone to want out of emoji suggestions.

That is probably right about Slack's intent. It does not answer the business question, for three reasons.

First, the protection was a description written by the vendor. "De-identified, aggregate" and "can't access the underlying content" are claims about engineering you cannot inspect. They may be true, and you still have to take them on trust.

Second, the scope moved without the customer doing anything. The same page that covered emoji suggestions in 2020 was, by 2024, the page people turned to when they wanted to know what an AI product did with their messages. TechCrunch quoted a Slack engineer conceding that the page needed updating to show how the principles applied to Slack AI. The terms were stable. What they were being asked to cover had grown.

Third, the exit was weak. Only certain owners could request it, by email, and the customer learned it had worked when Slack replied. One Hacker News commenter posted the reply he received: confirmation, followed by the same paragraph Slack gave TechCrunch.

A second example: Zoom

Nine months earlier Zoom went through the same cycle. In August 2023 The Record reported that Zoom's terms, apparently changed in March, gave it broad rights over "service generated data", including for training AI. Zoom then added a bolded line saying it would not use audio, video or chat content to train its models "without your consent". A lawyer quoted in the piece, David Brody of the Lawyers' Committee for Civil Rights Under Law, noted that the terms did not say what consent would look like.

The pattern in both cases: the terms were set on a page the vendor controls, the public read them only after someone posted them, and the fix came as a revised sentence on the same page.

Where things stand now

Slack's privacy principles page, as read on 8 November 2026, shows two things changed and one not. The page now says Slack does not develop generative AI models using customer data and "will not use Customer Data to train generative AI models unless Customer provides affirmative opt-in consent". Generative training has moved to opt-in. The predictive models are still opt-out, and the documented route out is still an email from a workspace owner to feedback@slack.com.

That split, opt-in for generative training and opt-out for the predictive global models described here, is a reasonable place for a vendor to land. It is also the distinction most buyers do not know to ask about.

A review you can run on published terms

Most advice on this subject is written for companies that negotiate enterprise contracts. A company with twelve people and a Slack Pro plan cannot negotiate anything. It can only choose between vendors on what they publish. For each tool that holds client or employee data, write down the answers to six questions.

  1. Where do the AI terms live? Read the signed agreement, data processing agreement and referenced policies together. Check which policies are incorporated, how changes are notified and what rights follow. Slack’s current principles expressly refer to customer agreements and confidentiality obligations; a policy page need not sit outside the contract.
  2. Which kinds of model? Generative models that produce text, or predictive models for ranking and recommendations? Get an answer for each. A promise about one says nothing about the other.
  3. What is the aggregation exception? Most terms let the vendor use "aggregated" or "de-identified" data. Find out what that covers and whether it reaches message content or only usage counts.
  4. Who has to opt out, and how? A setting you can check, or an email from a named role? If it is an email, file the reply, and repeat the request for every new workspace.
  5. What does your processing agreement say? Under Article 28 of the GDPR, a processor handles personal data on documented controller instructions, except where Union or Member State law requires otherwise, and a processor that decides the purposes and means of processing in infringement of the GDPR is considered a controller for that processing under Article 28(10). Whether training a vendor-wide model counts as your instruction is a question for your agreement and your adviser, not for a policy page. Ask the vendor to say in writing which legal role it claims for that use.
  6. How would you leave? Find out what you can export on your plan before you need it. On Slack's Free and Pro plans, owners and admins can export messages from public channels only. A full export of private channels and DMs requires Business+ with an approved application, or an Enterprise plan.

The answers rarely decide the choice on their own. They tell you what you have accepted, and they show which vendor will tell you in writing.

Where this does not hold

This review is worth the time for tools that hold client conversations, contracts, health or financial details, or source code. It is not worth it for a tool that only ever sees your own public website. If you already have a negotiated enterprise agreement, the contract governs and the policy page matters less, though it is still worth checking that the two agree. Sometimes every vendor in a category has similar terms and the review only confirms you have no better option. Knowing that is still useful: it tells you what not to put into the tool.

The opt-out email should still be sent. It is better than nothing, and in Slack's case it is the only documented route. Just do not record it as a control. A control is something you can check is still working. For the one or two systems where you need to be able to check that, the honest options are a contract with real terms or software you own and run yourself, which can give you more control over data flows. Ownership or self-hosting alone does not prevent training by external providers: verify model-provider terms, integrations, telemetry and deployed controls.

Sources