- Zen IT Technologies
- Technical notes
- Where data actually leaves through an AI platform
Where data actually leaves through an AI platform
Jonny Flaks, Founder & Principal Architect
Technical note in AI Platform Governance
The concern that reaches leadership first is an engineer pasting proprietary source into a chat window. It is a reasonable concern, but it is only one of the paths that matters.
In a governed SaaS AI platform, there are three primary paths to think about: what the user sends, what connected systems make available, and what the assistant is allowed to do on the way back out. They have different owners, different controls and different failure modes. Only the first one is obvious to the person worrying about data leaving the company.
Three paths through an AI platform
Control plane
- AI platform administration
- Identity / OAuth
- Connected SaaS
Adjacent surfaces: MCP / tool servers · Coding assistants · API access
The paste path
Text typed, pasted or attached directly to the interface is the path everyone pictures. Much of the control for it is decided before anyone types anything. The product and tenant type, contract, retention settings, training policy, administrative visibility and audit configuration determine what happens to the data after it is submitted. The same AI product used through a personal account and through a governed corporate tenant may therefore represent two very different risk positions.
Consolidating company use onto an approved tenant behind the identity provider closes more of this path than trying to control every individual prompt. What remains is a written boundary for what users may submit and an audit trail that allows the organization to verify how the platform is being used. The companion note, Six questions to ask about an AI platform, covers both.
What a connector actually grants
The second path is larger and much less examined. A connector is commonly a delegated authorization, often OAuth, that allows the AI platform to act against another system. Depending on the integration, access may be delegated from a user or granted through an application, service account or administrative authorization.
Two properties matter more than the description shown in the connection dialog.
Scope granularity is often coarser than people assume. Provider scopes are written for applications, not for the particular intention behind one prompt. A mail permission may expose most or all of a mailbox. A drive permission may cover everything the account is allowed to open, including content shared into it from elsewhere in the company. The authorization is granted at connection time. It does not normally become narrower because the user later asks the assistant a narrower question.
User delegated access is inherited, not created. A user delegated connector generally cannot reach data that the user themselves cannot access. That is reassuring, but it also exposes an existing problem. Over permissive sharing that has sat quietly in a drive for years because nobody knew where to look can suddenly become retrievable by a system whose purpose is to find and assemble information. The AI platform did not create the underlying access. It made the existing access significantly easier to exercise.
Application level permissions and service accounts need separate review because they may not be constrained by an individual user's own access boundary. Enumerate the connectors. Read the actual scopes granted rather than relying on the description in the connection dialog. Some integrations request broader scopes than a particular workflow needs.
Read and write fail differently
The third path is the one most likely to produce an unexpected external action, and it is not egress in the sense most policies traditionally define. Within a single connector there may be separate capabilities for reading, drafting, sending, replying, forwarding, sharing, modifying and deleting. They arrive through one connection and are often thought about as one capability. They are not.
A read tool exposes information to the model.
A write tool exposes the outside world to the model's decisions.
The failure mode is straightforward. A user asks for a draft. If the platform has both a draft tool and a send tool available, and sending does not require an additional approval, the system may be capable of completing the task by transmitting the message rather than only preparing it. No security control necessarily had to be bypassed. The problem is that the available capability exceeded what the organization intended the assistant to do without confirmation. The message still leaves under the user's identity.
Three mechanisms address this, and they should be considered in this order:
1. Set the ceiling at the organization, not the user. A user permission decision made during a task is being made by somebody who wants the task to finish. An organization level restriction defines the maximum capability any user can grant and applies consistently to new users and new connections.
2. Distinguish blocked from approval gated. Sending, replying, forwarding, sharing or deleting can often be blocked completely where the workflow does not require them. Where an outbound action is genuinely needed, require explicit approval for the action rather than relying on a permissive default.
3. Recheck the effective permissions after change. Product migrations, tier changes, new connector versions and newly introduced tools can alter the effective capability available through an integration. The resulting configuration needs to be checked against the organization's intended risk position rather than assumed to remain unchanged.
The adjacent paths
Several important surfaces sit outside the normal connector inventory and can be missed by a review that focuses only on SaaS integrations.
Internally built tool servers. Engineering teams can connect assistants directly to internal systems using the Model Context Protocol or a bespoke equivalent. These can become integrations into production systems without looking like traditional SaaS procurement. If the organization does not deliberately include them in its security review process, they can sit outside the normal connector inventory.
Coding assistants and repository context. An assistant embedded in an editor may send more context than the file currently visible on screen. Depending on the product and its settings, context can include neighboring files, dependency manifests, configuration, terminal output and other material from the working tree. The effective boundary is therefore defined by the product's context strategy and configuration, not only by what the developer believes they selected.
API access. API keys and service credentials may be associated with users, projects, service accounts or workloads. The governance problem is that they can sit outside normal joiner, mover and leaver processes unless somebody deliberately inventories ownership, purpose and lifecycle. A key created for an evaluation can otherwise become part of a script, pipeline or production workflow long after the evaluation is forgotten.
Where the boundary is enforced
Endpoint, browser, network and data protection controls can still restrict uploads, block destinations, inspect some traffic and enforce access policy. They do not normally have the complete picture. A browser session to an approved AI domain does not by itself reveal the tenant type, connector inventory, scopes already granted, or whether an assistant has permission to send, share, modify or delete data in another system.
Those authoritative controls are distributed across the AI platform's administrative console, the identity and OAuth layer, and the systems being connected. That is also why this work is frequently nobody's job. It sits between security tooling, identity administration and SaaS administration, and the gap is often discovered only after the first surprise.
Verification worth doing
Open the administrative controls for every AI platform in use and produce four lists:
Accounts. The tenant or product tier each account belongs to, its owner, and whether authentication is controlled through the identity provider.
Connectors. What is connected, which scopes were granted, how the authorization was granted, and who authorized it.
Action capability. Send, reply, forward, share, modify, delete and other write tools, with the organization level policy and approval requirement for each.
API access. Keys, projects, service accounts and other credentials, with an owner, purpose and known place of use.
The lists are the deliverable. Most of the remediation becomes obvious once they exist, and the questions on a customer security questionnaire become answerable from evidence rather than recollection.
Companion technical note: Six questions to ask about an AI platform
Explore this expertise: AI Platform Governance