Feature flag cleanup with AI agents: DoorDash's $4.79 per flag

Every team that uses feature flags accumulates dead ones: flags whose feature shipped months ago, still wrapped around code nobody dares delete. DoorDash has published the most detailed account yet of automating feature flag cleanup with AI agents — and the numbers make a strong case that stale flags are one of the best first jobs to give a coding agent.

The size of the problem

According to DoorDash's engineering write-up, summarised in detail by ZenML's LLMOps database and InfoQ, the company's experimentation platform manages more than 60,000 flags — DoorDash calls them dynamic values — across roughly 623 repositories. More than 1,000 were stale, and about 2,300 new flags are created every month. Removing one by hand takes an engineer one to two hours, because it means tracing every call site, collapsing the branching logic and updating tests.

DoorDash defines a flag as stale when it has not been modified for 90 days, is still referenced in code, is not archived and is not on an exclusion list. A daily job files a Jira ticket for each one and assigns it to the flag's creator. That gave the automation a queue to work from.

How DoorDash automated stale feature flag removal

The system runs in two phases, with a person between them.

• Phase 1 — plan. An orchestrator agent, reported as running on Claude Sonnet, reads the Jira ticket, queries the experimentation platform through the Model Context Protocol for the flag's rollout percentage and target value, finds the repository and searches for references. It produces a cleanup plan.
• Human confirmation. An engineer confirms the value the flag should be replaced with. This is the one judgement that matters most, and it stays with a person.
• Phase 2 — remove. One removal agent per confirmed flag, running on Claude Opus, edits the code in its own git worktree — up to four at once per repository — replacing flag checks with the confirmed value, deleting dead branches and updating tests. It then runs the build, tests, coverage and lint checks, and opens a pull request only if they pass. Each agent has a one-hour hard timeout.

Isolation turned out to be essential. DoorDash reports that early experiments without separate worktrees produced race conditions and corrupted diffs.

What it cost

DoorDash evaluated the system on its 50 most recent stale flags across several Kotlin repositories. It produced usable pull requests for 45 of them. Time and cost grew with complexity:

Complexity	Flags	Average time	Average cost
Simple (one check, 1–2 files)	6	7.5 min	$2.69
Medium (3–5 files, some tests)	18	10.4 min	$3.46
Complex (nested logic, cross-file)	26	17.7 min	$6.20
All	50	13.8 min	$4.79

Some complex outliers approached 40 minutes and $19. Against one to two hours of engineer time per flag, that is a large saving even before counting the context switching a manual cleanup costs. ZenML rightly cautions that the figures come from Kotlin repositories and may not transfer directly to other languages or codebases.

Why stale feature flags are a good first job for agents

Most agent failures come from ambiguity. On private codebases, the best agents solve well under half of realistic tasks, and missed requirements and integration errors are the leading causes. Flag removal avoids most of that:

• The target state is known. Once a person confirms the value, the correct code is determined.
• Success is checkable. The build compiles, the tests pass, and behaviour should be unchanged.
• The blast radius is small. Each change is one flag, in one worktree, in one pull request.
• Failure is cheap. DoorDash frames its reliability boundary as completeness rather than correctness: a failed attempt should leave an unmerged pull request, not a broken build.

That profile — known outcome, mechanical verification, small reversible changes — is the same one that let Duolingo auto-approve a slice of its pull requests and let GitHub run an agent-assisted migration one slice at a time.

Feature flag cleanup best practices, with or without agents

1. Define stale mechanically. A time threshold plus "fully rolled out" and "not archived" is enough to generate a queue.
2. Give every flag an owner and an expiry. Most flag debt exists because nobody is responsible for removing it.
3. Keep the value decision human. Whether the flag resolves to on or off is a product decision. Automate everything after it.
4. Isolate each change. One flag, one branch, one pull request. For agents, one worktree each.
5. Gate on the build, not the model. Build, test, coverage and lint checks must pass before a pull request is opened.
6. Measure cost per flag. DoorDash's $4.79 average is a useful benchmark for deciding whether automation is paying off in your own codebase.

DoorDash's next step is a confidence score that could auto-approve the lowest-risk cases, such as a fully rolled-out boolean whose production value already matches the code default. That is a sensible direction, but only once the plain version has a track record.

If your codebase carries years of dead flags and your team never has time to remove them, our AI engineering services team can set up the same plan-confirm-remove pipeline against your repositories and CI.