AI gateway model routing without giving agents keys

Tailscale built a product called Aperture that grants access to language models by network identity instead of by API key. An agent starts in a sandbox, authenticates as itself, does its work and terminates — having never held a credential. For anyone weighing AI gateway model routing against calling providers directly, that is the argument that deserves attention, and it is not the one usually made.

The usual case for a gateway is convenience: one endpoint, failover, unified billing. The interesting case is that it removes a secret from a place you cannot fully control.

The problem this actually solves

Every agent architecture eventually faces the same question: how does the agent authenticate to the model?

The default answer is an API key in the environment. It works immediately and it is the reason a whole class of incidents exists, because as OpenAI's own sandbox guidance states plainly, agent-generated code can access the files, credentials and network available to its environment. A key placed where an agent runs is a key the agent can read, print, log, or pass to a tool you did not anticipate.

The usual mitigations are all partial. Rotate often, and you have shortened a window rather than closed it. Scope the key per agent, and you now operate a key management system. Inject through a broker, which is better, and you have built infrastructure.

Tailscale's approach sidesteps the question. Access is a property of network membership, so an organisation grants or revokes model access by adding or removing a machine from its tailnet. There is no key to leak because no key is issued.

Worth being precise about what is and is not solved. Identity-based access controls who may call the model. It does not control what the agent does with the answer, which tools it invokes, or what it sends outbound to a third party — those remain sandbox and allowlist problems. What it removes is one specific, common and consequential failure: a long-lived secret sitting in a filesystem that an improvising process can read.

How the flow works

The sequence is short enough to hold in your head. A sandbox spins up and connects to Aperture. Aperture connects to the gateway. Tailscale validates the identity. The agent executes its work. The sandbox terminates. At no point is an API key issued to the agent.

Two consequences follow that are worth more than they first appear.

Revocation becomes immediate and complete. Removing a machine from the network ends its access in the same action, with no key to rotate afterwards and no forgotten copy in a configuration file, a CI secret or someone's local environment. Anyone who has tried to establish with certainty that a leaked credential is now dead will recognise the difference.

And the audit question changes shape. Instead of asking which key made a request, you ask which machine did, with identity established by the network layer rather than by a bearer token that anyone holding it can present. That is a materially stronger claim to make to an auditor.

There is a limit worth naming early. This works because Tailscale already runs a network with strong machine identity, and because the agents in question run on machines that can join it. If your agents execute inside a managed sandbox you do not control the networking of, or on a provider's hosted infrastructure, the same pattern may not be available to you — the credential broker approach is then the realistic version of the same idea. The principle transfers even where the implementation does not.

Why they did not build it

Tailscale considered building their own routing layer and decided the complexity was not worth it. David Carney's assessment is the useful part: there are a lot of things the big providers do not do that blow his mind that the gateway does, including cost tracking in responses.

Cost tracking in the response is a small detail with large operational consequences. Without it, attributing spend to a tenant, a feature or a runaway agent means correlating your own request logs against a provider invoice that arrives later and aggregates differently. With it, every response carries what it cost, and per-tenant economics become a property of the data rather than a monthly reconciliation exercise.

The other line worth quoting is Remy Guercio's, on why the abstraction is harder than it looks: you would think all of the endpoints are the same, they are not. This matches our experience precisely. Providers differ on streaming semantics, tool-calling formats, token accounting, error taxonomies, retry behaviour and how they express refusals. A routing layer is not a switch statement over base URLs. It is a compatibility surface that has to be maintained as five vendors ship breaking changes on their own schedules, and it is exactly the kind of undifferentiated work that quietly consumes a team.

One commercial detail removes the usual objection: the gateway does not mark up token costs on any provider or model, so customers pay the same rates as calling directly. That matters, because the standard reason to avoid an intermediary is a margin on every token.

It is also the piece of the stack most likely to be a false economy to own. A routing layer produces no differentiated value for your customers, it is invisible when it works, and it demands attention every time a provider changes something. Teams build it anyway because the first version is a weekend of work — and that first version is the one that encodes assumptions about streaming and tool calls that break six months later.

Read it with the source in mind

This account is published by the gateway vendor, about a customer using the gateway. That does not make the engineering wrong, and the named quotes are specific enough to be checkable, but it does shape what is present and absent.

What is absent is worth listing. There are no latency figures, so the cost of an additional network hop in front of every model call is unquantified. There is no failover or provider-outage data. There is no discussion of what happens when the gateway itself is unavailable, which is the obvious question about any component that sits in front of everything. If you are evaluating this, those are the three things to measure in your own pilot rather than infer from the post.

AI gateway model routing: what we would take from it

Three things transfer regardless of which vendor you end up with.

Decide how agents authenticate before you decide anything else. It is an architectural decision that is cheap at design time and expensive to retrofit, because by then keys are distributed across sandboxes, CI and local environments. If you are doing nothing else from this article, write down where your agents' credentials live and who can read them.

Prefer identity to secrets wherever the platform allows it. This is not new thinking — workload identity federation, mutual TLS and short-lived tokens all express it — but agents make the argument urgent, because an agent is a workload that reads its own environment and improvises. The reasoning is the same one behind starting sandbox design with the allowlist: constrain what is reachable, because you cannot constrain what will be attempted.

Treat routing as infrastructure to buy, not build, unless routing is your product. The compatibility surface across providers is real work with no end date, and the same logic applies as when we looked at whether to buy the agent harness or keep your own. Keep your prompts, tools and evaluation suite portable and in version control; let someone else maintain the adapter layer.

None of the three requires adopting any particular vendor, which is the point. They are decisions about your own architecture that stay correct whichever gateway, model or sandbox provider you land on, and each one is cheaper to make now than to unpick later.

The wider pattern is that the AI stack is separating into layers and the security-relevant one is turning out to be access rather than the model. Which model you call is increasingly a routing decision made at runtime. What that call is permitted to reach, and what it proves about who is asking, is the part that belongs to you. That is where we would concentrate effort on any AI engineering project, and it is the question we ask before the model comparison, not after it.