Self-hosted AI coding assistant: what GitLab Duo requires

A self-hosted AI coding assistant is not one decision. It is five: which GitLab tier you hold, where the model runs, who carries the GPUs, which features quietly degrade, and what still leaves your network anyway. GitLab's reported expansion of Duo Self-Hosted to models served through Microsoft Foundry moves two of those. The other three are where most teams get the answer wrong.

On 22 September 2026 InfoQ reported that GitLab broadened Duo Self-Hosted to models deployed through Microsoft Foundry, so organisations can run GitLab's AI features against models hosted in their own Azure environments. The architecture has three parts: a self-managed GitLab instance, a self-hosted GitLab AI Gateway, and one or more model endpoints. The gateway sits between Duo and the model so features are not coupled to a single vendor.

That is the headline. Below it sit the details that decide whether this is a project or a slide.

What GitLab Duo Self-Hosted supports, and the tier it actually needs

Start with entitlement, because it is the cheapest thing to get wrong. GitLab's own Duo Self-Hosted documentation lists the feature as Premium and Ultimate tier, on GitLab Self-Managed and GitLab Dedicated for Government. Self-hosted models went generally available in GitLab 17.9 and were extended to Premium in 18.0.

Then the add-on. To use GitLab Duo features with Duo Self-Hosted you need the GitLab Duo Enterprise add-on — and GitLab states this applies even for features you could otherwise use on Duo Core or Duo Pro when GitLab hosts the connection through its cloud AI Gateway. Self-hosting is not a way to avoid the licence. It is a more expensive licence plus infrastructure.

The Agent Platform is a separate line again. GitLab Duo Agent Platform reached general availability in GitLab 18.8 for self-hosted deployments. Customers on an offline licence need the GitLab Duo Agent Platform Self-Hosted add-on and are billed under an Enterprise Licence Agreement; customers on an online licence are billed by usage, and GitLab Duo Self-Hosted itself uses seat-based pricing.

Feature maturity is uneven and the docs say so. Code Suggestions, non-agentic Chat, Code Explanation, Test Generation, Refactor Code and Fix Code are generally available from 17.9, Code Review from 18.3. Root Cause Analysis, Vulnerability Explanation, Vulnerability Resolution, merge request summaries and GitLab Duo for the CLI are still marked beta. If your business case rests on a beta feature running on your own model, say that out loud in the proposal.

Microsoft Foundry is the Azure OpenAI path, not a Claude-on-Azure path

This is where the coverage and the primary source pull apart, and it is worth a paragraph of its own. InfoQ describes the integration as providing access to model families "including OpenAI GPT, Anthropic Claude, Meta Llama, and Mistral". GitLab's configure LLM platforms page does not name Microsoft Foundry at all. It lists the validated cloud-hosted providers as AWS Bedrock, Amazon Bedrock Mantle, Gemini Enterprise Agent Platform, Azure OpenAI, Anthropic and OpenAI — and the Azure OpenAI entry links to Microsoft's catalogue of models sold directly by Azure, on the Azure OpenAI pivot.

Read together, the honest statement is narrower than the headline: GitLab validates the Foundry path as Azure OpenAI. Claude, Llama and Mistral are genuinely supported models, but by way of Bedrock, the Anthropic API, or your own vLLM deployment. InfoQ makes the same point in its own caution — Foundry's catalogue evolves faster than GitLab's compatibility matrix, so availability in Foundry does not imply Duo compatibility. Check each model against GitLab's supported models table before anyone signs a capacity commitment.

That table also carries the finding that changes architectures. In GitLab's supported models matrix, Claude Sonnet 4.6, Claude Opus 4.8, GPT-5.1 and GPT-5.2 are rated full functionality across code completion, code generation, Chat and the Agent Platform. The open-weight models you would realistically run on your own GPUs are not. Llama 3.3 70B, GPT-oss-120B, Codestral 22B and Mistral Small 24B are all rated full or partial for code completion and generation, and limited for the Agent Platform — and GitLab says a model with limited functionality for a feature will not receive support for that feature.

There is an escape hatch, in beta since 18.3. You can bring your own compatible model served through vLLM or LiteLLM, provided it is exposed through an OpenAI-compatible /v1 endpoint. GitLab does not provide technical support for issues specific to your chosen model or platform, and warns that response quality and speed may vary significantly.

What a self-hosted AI coding assistant costs you, qualitatively

We will not quote you a GPU price, because the number that matters is not a sticker price — it is utilisation, and that is yours. What we can quote is GitLab's published hardware floor, which sets the shape of the bill.

Model size	Minimum GPU configuration	Minimum VRAM
7B (for example Mistral 7B)	1x NVIDIA A100 (40 GB)	35 GB
22B (for example Codestral 22B)	2x NVIDIA A100 (80 GB)	110 GB
Mixtral 8x7B	2x NVIDIA A100 (80 GB)	220 GB
Mixtral 8x22B	8x NVIDIA A100 (80 GB)	526 GB

Alongside that, GitLab's base system requirement is a minimum of 8 CPU cores and 32 GB of RAM, with 16 or more cores and 64 GB recommended for production. Those are minimums for the model server. The AI Gateway is a separate deployment with its own scaling guidance.

The second cost is latency under concurrency, and GitLab publishes its own benchmarks. On a four-GPU A100 machine, Mistral-7B-Instruct-v0.3 averaged 3.80 seconds for a single request and 11.71 seconds per request at 100 concurrent requests. On an eight-GPU A100 80 GB machine, Mixtral-8x22B-Instruct-v0.1 averaged 14.39 seconds for one request and 27.58 seconds per request at 100. Code completion that takes fourteen seconds is not code completion; it is a distraction with a spinner. Capacity planning for an inference fleet is its own discipline, closer to the work in GPU utilization on Kubernetes for AI inference than to running another web service.

The third cost is the one InfoQ names and most business cases omit: responsibility transfer. Your platform team now owns model deployments, capacity management, networking, credential administration, availability and model lifecycle, on top of the GitLab estate. That is a standing rota, not a project. Before committing, it is worth walking the arithmetic the way we did for self hosted LLM cost generally — the compute is rarely the line that decides it.

The middle paths: private endpoints, regional processing, zero data retention

Most teams asking for self-hosting are not asking for GPUs. They are asking for three guarantees: our code does not train anyone's model, our code is not stored where we cannot see it, and our traffic does not cross a border it should not. Each has a documented control that costs far less than a GPU fleet.

Keep the network path private. Microsoft Foundry supports private endpoints through Private Link, with public network access set to Disabled, or to selected IP addresses and virtual networks. The model runs in a Microsoft service; the route to it does not touch the public internet.

Pin where processing happens. Microsoft's data privacy documentation for models sold by Azure distinguishes deployment types. A Global deployment may process prompts and responses in any geography where the model is deployed. A DataZone deployment processes within the declared zone — a resource in an EU member nation stays within EU member nations. For both types, data stored at rest remains in the customer-designated geography.

Turn off the retention you did not want. Microsoft states the models are stateless, that no prompts or completions are stored in the model, and that prompts and completions are not used to train or improve the base models without your permission. Abuse monitoring may store a sample of prompts and completions for human review, in the geography where your Foundry resource is deployed; for deployments in the European Economic Area, authorised reviewers are located in the EEA. Customers approved for modified abuse monitoring do not get that storage or human review at all, and Microsoft documents how to verify it by checking that the ContentLogging capability reads false.

The equivalent control at OpenAI is documented too: abuse monitoring logs are retained for up to 30 days by default, and eligible customers may be approved for Zero Data Retention or Modified Abuse Monitoring, set for the whole organisation or per project. Zero Data Retention also forces the store parameter to false on the responses and chat completions endpoints. If residency is the actual requirement rather than retention, that is a different control again, and we covered the regional options in OpenAI data residency in India and the UAE.

GitLab has built the middle path into the product. Its hybrid AI Gateway configuration went generally available in 18.9 and lets you choose, per feature, whether requests go to your self-hosted gateway and models or to GitLab-managed models. The trade is explicit in the docs: any feature pointed at a GitLab-managed model calls the GitLab-hosted gateway, requires internet connectivity, and is not a fully self-hosted or isolated configuration.

When self-hosting is genuinely the right answer

Give the case its strongest form. A contractual promise is not a network control. Retention policies change with a product update; a subnet with public access disabled changes when you change it. For a defence supplier, a government-adjacent programme or anyone operating behind a physical air gap, there is no vendor endpoint that satisfies the requirement, and GitLab's fully self-hosted configuration is the only configuration that can operate in a completely isolated network.

Even then, read the connectivity table. GitLab documents that an online licence needs outbound access on port 443 to customers.gitlab.com, cloud.gitlab.com and the Duo Workflow service to run usage billing for Agent Platform features — and if your firewall blocks any of them, billing fails and the features stop. Only billing metadata leaves: instance ID, a de-identified user ID, a call count and a timestamp. GitLab states that inference data, including code inputs, prompts and model responses, does not leave the customer network, and that it does not capture which model or provider you use. An offline licence removes even that, billed under an Enterprise Licence Agreement instead.

This is the wrong choice if your constraint is really procurement anxiety rather than a control requirement. It is also the wrong choice if agentic workflows are the point, because the models rated full functionality for the Agent Platform are overwhelmingly the hosted frontier ones. And it is the wrong choice if the actual exposure is a tool you have not audited — the risk we wrote about in AI coding tool data privacy was never about where the GPU sat.

How to decide this in a week

Write down the requirement as a sentence someone can test. "Source code must not be stored outside our tenant" is testable. "We want AI on-prem" is not. Then map it: storage is answered by modified abuse monitoring or zero data retention, location is answered by a DataZone deployment, network exposure is answered by a private endpoint, and only isolation from the public internet entirely requires your own GPUs.

Next, price the licence before the hardware. Premium or Ultimate, plus Duo Enterprise, plus the Agent Platform Self-Hosted add-on if you are offline, is the floor before a single GPU is racked. Then pick two models from GitLab's supported table that are rated full functionality for the features your developers will actually use, and check both are servable on the platform you have.

Run the comparison as a one-week spike, not a quarter. In the systems we build, the deciding evidence is nearly always latency at realistic concurrency and the honest support position on the model you chose — both of which you can establish in days. If you want that spike run with someone who has done the residency and gateway work before, that is the shape of our AI engineering services engagements.

Decide the control first, the model second, and the hardware last. Most teams that start at the hardware end up with a self-hosted AI coding assistant nobody uses, and a rota nobody wanted.