Vulnerability alert fatigue: what to automate, what to keep

Vulnerability alert fatigue is not a volume problem you can scan your way out of. On 23 September 2026 CISA's Known Exploited Vulnerabilities catalogue held 1,721 entries, which FIRST puts at roughly 0.5% of published CVEs. Almost everything your scanner escalates will never be exploited anywhere. Triage is a routing problem, and exactly one step in it needs a person.

This post sets out a triage design for a team that does not have a dedicated security operations function: what to automate, what to route, how to rank by likely exploitation rather than severity alone, and the specific point at which automation stops being leverage and starts being a rubber stamp.

Why vulnerability alert fatigue is a routing problem

The numbers make the case better than any argument about process. FIRST, which runs the Exploit Prediction Scoring System, states that in any 30-day window its data partners observe exploitation activity for roughly 2.5-3% of published CVEs. The base rate of exploitation is low, and it is low for a reason: most vulnerabilities are never weaponised, never packaged, and never reach anything you run.

That means the default posture of most scanners — escalate everything above a severity threshold — guarantees a queue in which the overwhelming majority of items are correctly ignored. Engineers learn that quickly. Once a queue is 97% noise, the rational response is to stop reading it, and the alert that mattered goes past with the rest. The fatigue is not weakness; it is a correct read of a badly calibrated signal.

So the goal is not fewer scanners. It is a pipeline that silently disposes of what is provably irrelevant, ranks the remainder by likelihood of exploitation, delivers each survivor to the person who owns the code, and asks a human one question about each: fix now, schedule, or accept.

What WHOOP automated, and what it did not

The New Stack described the health and fitness company WHOOP as having relied on multi-day, all-hands triage sessions to keep up with the alert deluge, and building an automated vulnerability-response workflow in response. Worth stating plainly: that piece is a Datadog-sponsored announcement of a webinar on 7 October 2026 with WHOOP staff engineer Vinay Raghu and Datadog senior product engineer Amber Tunnell, not an independent technical write-up. Take the shape of the workflow from it, not the results.

Four things it says the workflow does, which is a fair skeleton for anyone:

• Focus on real exposure rather than scanner noise. WHOOP used Datadog's Software Composition Analysis to analyse runtime code execution and prioritise active threats.
• Route the right vulnerability to the right engineer. Vulnerability mapping to microservice owners was automated, so developers received tickets with full context attached.
• Build automated guardrails so developers self-resolve. This let security engineers move from gatekeeping and ticket-pushing to systemic work.
• Keep a human in the loop. Handling sensitive user data, the article says WHOOP "isn't ready to automate the engineer out".

The second bullet is the one most teams skip and the one that pays. An alert sent to a security channel is a task without an owner. The same alert filed against the service that owns the dependency, with the call path and the fix version attached, is a ticket someone can close before lunch. Ownership routing is unglamorous plumbing and it removes more fatigue than any scoring model.

Rank by exploitation likelihood, not severity alone

Two free, public signals do most of the ranking work, and they answer different questions.

Signal	What it tells you	How to use it
CISA KEV	This vulnerability is known to have been exploited. 1,721 entries as of 23 September 2026, of which 361 are flagged as known ransomware campaign use.	Treat a KEV match in a running service as a decision already made. Evidence beats prediction.
EPSS	A daily 0-1 probability that a published CVE will be exploited in the wild in the next 30 days, with a percentile rank.	Rank the enormous middle of your queue, where no exploitation evidence exists yet.
CVSS	How bad it would be if exploited, in the abstract, independent of your deployment.	A tiebreaker and a blast-radius input, not a queue order.

Three details from FIRST's own documentation change how you set thresholds. First, the score distribution is heavily skewed: a probability of 0.10 currently sits around the 95th percentile, so a number that reads as small is in fact an unusually strong signal. Second, there is no universal threshold — FIRST says 0.10 "is commonly cited in the field but carries no special authority from EPSS", and points instead at a coverage, effort and efficiency framework calibrated to the remediation capacity you actually have.

Third, and least known: EPSS deliberately does not look up confirmed exploitation. Exploitation activity observed today does not directly change tomorrow's score; exploitation data trains the model rather than driving daily scoring. FIRST is explicit that where direct evidence of active exploitation exists, that evidence should supersede EPSS. So KEV and EPSS are not competing rankings to blend — KEV is the evidence lane and EPSS orders the queue where evidence is absent. We worked through the same split in more depth in why a CVSS 9.8 can wait.

Automate the dismissals your tooling already supports

Before building anything, turn on what the platform gives you. GitHub's Dependabot auto-triage rules dismiss or snooze alerts automatically, and critically, "Rules are applied before alert notifications are sent" — so an auto-dismissed alert never reaches anyone's inbox.

• The GitHub-curated preset Dismiss low impact issues for development-scoped dependencies auto-dismisses npm development-dependency vulnerabilities unlikely to be exploitable in a non-production environment. It is enabled by default on public repositories and worth enabling on private ones.
• A second preset, Dismiss package malware alerts, is disabled by default. Leave it that way unless you have a specific reason.
• Custom rules match on metadata including severity, package name and CWE.
• Dismissal is reversible, and alerts auto-reopen when the metadata changes — for example when a dependency moves from development to production scope. That reversibility is what makes aggressive dismissal safe.

Then wire the remainder to owners. Dependabot alerts can be assigned to collaborators, teams or AI agents; an agent assigned an alert opens a session and a draft pull request with a proposed fix, and stays the assignee if it cannot produce one. That is a good use of an agent: it drafts, a person merges. Note one operational wrinkle in GitHub's docs — assignment is visible in the repository-level alert view, not in the organisation-wide security overview — so do not build your programme reporting on it yet.

What your scanner will not tell you

Automating triage on a signal you have not characterised is how a quiet queue becomes a false sense of safety. GitHub publishes the limits of Dependabot alerts directly, and they are worth reading as a list of the gaps you must cover another way:

• Alerts cannot catch every security issue; detection depends on manifest and lock files being current.
• New vulnerabilities may take time to appear in the GitHub Advisory Database before any alert fires.
• Only advisories reviewed by GitHub trigger alerts.
• Archived repositories are not scanned.
• For GitHub Actions, alerts are only generated for actions using semantic versioning, not SHA pinning.

That last one catches teams who did the right thing. Pinning actions to a commit SHA is standard supply-chain advice, and it removes you from Dependabot's alerting for those actions. The dependency you pinned hardest is the one nobody is watching. Registry-level compromises, which we covered in the docs-builder attack on a package registry, land squarely in that blind spot.

The step where a human decides

Here is the pipeline we would run for a team of 20 to 200 engineers. Five of the six steps are machine work.

1. Ingest. Every scanner writes into one queue with a stable identifier per finding, so the same CVE in four services is four routed items and not four independent investigations.
2. Auto-dismiss. Development-scoped and non-deployed findings are dismissed by rule, reversibly, before notification.
3. Rank. KEV match first, then EPSS order, with CVSS and asset exposure as tiebreakers.
4. Route. Each survivor is filed against the service that owns the dependency, with the fix version and the call path attached.
5. Decide. A named human answers one question per item: fix now, schedule, or accept with an expiry date.
6. Execute. An agent or a bot opens the upgrade pull request; CI and a reviewer do the rest.

Step five is not sentiment. It is where the three inputs a scanner cannot see get applied: whether the vulnerable path is reachable in your deployment, what the fix costs in regression risk, and whether a compensating control already exists. A machine can rank probability of exploitation across the world. It cannot tell you that the affected code path sits behind an internal-only load balancer, or that the upgrade drags in a breaking change to a payments integration you ship on Friday. Those are the facts that flip a decision, and they live with the engineer who owns the service.

The accept branch matters as much as the fix branch. An acceptance without an expiry is a silent permanent exception, and a year later nobody remembers why. Every accept gets a date and returns to the queue on it.

Where automated triage quietly becomes rubber-stamping

Give the counter-argument its due: a human decision step is exactly what created the multi-day, all-hands triage sessions in the first place, and adding a person to a 97%-noise queue does not improve anything. That is true, and it is why the human step comes fifth. The person is not reading the queue; they are answering one question about items that already survived dismissal, ranking and routing. If your engineers still face a wall, steps two to four are not doing their job.

Four signals that automation has stopped helping, all of them observable in your own tracker:

• Decision latency collapses to seconds. If the median time between an item appearing and being marked "accept" is under a minute, nobody is deciding anything.
• The accept rate climbs above the auto-dismiss rate. Your dismissal rules are too timid and the human is absorbing the overflow.
• Bulk actions dominate. Decisions taken 40 items at a time are a rule that should have been written as a rule, where it would be reviewable.
• Nothing ever reopens. Auto-reopening on metadata change is a designed behaviour; if it never fires, check that it is wired at all.

Instrument those four and you have a triage programme that reports on itself. That is the same discipline we apply in QA and automation work generally: automate the repeatable judgement, measure the automation, and keep the irreversible call with a named person. The pipeline plumbing usually lands in CI/CD implementation, and the asset inventory that makes routing possible in DevOps and cloud engineering.

If you do one thing this week, do not buy a tool. Export a month of alerts, mark which ones were a KEV match and which had an EPSS score above the 95th percentile, and count how many of those reached the owning engineer inside a working day. That number is your actual triage capability, and it is usually a surprise.