When to build your own database: Perplexity dropped DynamoDB

Perplexity moved its core search serving tier off Amazon DynamoDB onto CobbleDB, a key-value store it wrote in Rust, and median batch-read latency went from 31.4ms to 5.60ms. That is a real result on a real workload, and it is still the wrong reason to build your own database. The question worth copying is not what Perplexity built but what made the managed service a bad fit — and whether any of it is true of you.

The figures below are as InfoQ reported them on 25 September 2026. Perplexity has published its own engineering write-up at perplexity.ai/hub/blog/cobbledb; that URL returned HTTP 403 to every request we made from here, so everything attributed to Perplexity in this post comes through InfoQ's account rather than from the original.

What Perplexity replaced, and what it measured

CobbleDB is a distributed key-value store optimised for one thing: batched lookups. Each partition keeps three replicas on independent compute nodes, the daemon uses RocksDB as its embedded engine with memory-mapped caching over local NVMe, and a stateless router hashes page identifiers to partitions, prefers a replica in the same availability zone, and speculatively hedges a second read to another replica when the first one is slow. Within a node, keys are fetched through RocksDB's batched MultiGet.

Measure	DynamoDB	CobbleDB
Median batch-read latency	31.4ms	5.60ms
p90	56.7ms	9.77ms
p99	123ms	24.2ms
Storage cost	baseline	at least 20% lower

Production traffic exceeds 200,000 requests per second, and synthetic benchmarks with payloads up to 100KB held throughput to 500,000 requests per second. Perplexity's CEO Aravind Srinivas said CobbleDB is about 40,000 lines of Rust, written in two months by two systems engineers working alongside an autonomous swarm of AI coding agents that handled integration testing, build monitoring and operational runbooks. The company has said it plans to open-source the code.

The workload shape explains the rest. A query to Perplexity resolves to between 100 and 120 target page keys, which the retrieval service splits into parallel batches of 10 to 20. Unlike a conventional search engine returning short metadata snippets, retrieval for a language model pulls full chunked passages and dense vector embeddings, so the average record is roughly 50KB.

Why the managed service stopped fitting

Two reasons were given, and they are different in kind. The first is pricing: DynamoDB meters usage, and at the volumes above that metering became financially unsustainable. The second is opacity — DynamoDB conceals partition placement, memory caching policy and replica routing, so Perplexity's engineers could not prevent tail-latency spikes caused by uncached reads, cross-zone network hops or a lagging replica. A third problem sits between them: reprocessing jobs triggered by a new chunking algorithm or a newer embedding model pushed high-volume writes straight into DynamoDB, contending with live user reads.

The pricing half is arithmetic you can do yourself, and it is worth doing before you conclude anything about your own bill. DynamoDB's on-demand pricing charges eventually consistent reads at 0.5 read request units per 4KB of item size, rounded up. A 50KB record is 13 four-kilobyte increments, so 6.5 RRUs per item. At roughly 110 keys per query, that is about 715 RRUs for a single user question before any retry, any miss and any re-read. Writes are metered harder still: 1 write request unit per 1KB, so re-embedding one 50KB record costs 50 WRUs, and re-embedding a corpus multiplies that by the corpus.

Run those two numbers against your own record size and fan-out. If your records are 2KB and a request touches four of them, you are paying 0.5 RRUs per read and the entire pricing argument evaporates. The metering only bites when large records meet high fan-out meets high request volume, and most products have at most one of the three.

Build your own database only when four things are true

Here is the test we apply when a client asks this, in the order that kills the idea fastest.

1. The access pattern is one shape, and you can write it in a sentence. CobbleDB serves batched gets of hashed keys. No secondary indexes, no range scans, no transactions. If your sentence needs an "and also", you are describing a general-purpose database and you should buy one.
2. You can tolerate what the design drops. CobbleDB discards distributed transaction protocols and synchronous consensus; replicas apply updates asynchronously at their own rate because search serving tolerates slight replication lag. That is only free if reading a stale record is genuinely harmless in your product. For an order book or a ledger it is not.
3. You have measured that the managed service's opacity is the cause, not a suspect. Perplexity could name the mechanisms — uncached reads, cross-zone hops, lagging replicas. If your p99 investigation ends at "DynamoDB is slow sometimes", you have not finished it, and the same investigation usually turns up an index or an access pattern you can fix for a fortnight's work.
4. You can staff the operations for as long as the product lives. This is the one that decides it.

Three of CobbleDB's ideas work on a store you already rent

Most of what makes CobbleDB fast is not the storage engine. Three of its choices are client-side patterns that apply to any key-value store, including the managed one you are already paying for, and they are worth trying before a rewrite is on the table.

Zone-affinity routing. CobbleDB's router prefers a replica in the same availability zone. Cross-zone hops were named as one of the causes of Perplexity's tail latency, and in most stacks that is a client and placement decision rather than a database feature. Check where your readers actually sit relative to the data they read.

Hedged reads. When a replica responds slowly, CobbleDB issues a concurrent read to an alternate replica on another node and takes whichever returns first. You can implement the same pattern in your data access layer against almost any backend: a short timer, a second request, first response wins. It converts a tail-latency problem into a modest increase in read volume, which is usually the cheaper trade.

Batching at the engine boundary. CobbleDB fetches keys through RocksDB's batched MultiGet so a batch costs one round trip rather than twenty. If your service loops over keys issuing one request each, you are paying per-key network cost that batching removes, and you do not need a new database to stop doing it.

None of these three needs a migration, and if they close most of your gap then the build-versus-buy conversation is over before it starts. That is the outcome we want when a client raises this, because the alternative is a year of engineering time spent reaching a latency number that a fortnight of routing and batching work would have reached.

The costs that do not appear in the latency table

Replacing a fully managed database moves node lifecycle management, backup verification and partition rebalancing onto your own site reliability engineers, permanently. Applications must also withstand eventual consistency, which is not a one-off migration cost but a constraint every future feature inherits.

The build itself is the cheap part, and the 40,000-lines-in-two-months figure is the most misleading number in the story — not because it is wrong, but because writing the code was never what stopped anyone. What stops teams is the fourth year: the on-call rotation that now owns a storage engine, the engineer who wrote it having left, the rebalance that goes wrong at 3am against a system with no vendor to call and no Stack Overflow answers. The same asymmetry shows up in every rewrite we look at, which is why incremental migrations usually beat big-bang ones even when agents are doing the typing.

There is a cheaper move hiding inside Perplexity's architecture, and it is the part most teams should copy. The system was split into three: Pillar for durable state on YTsaurus over high-capacity mechanical drives, Lorry as a stateless consumer that groups Pillar exports into partition-aligned batch files in S3 and posts metadata notices, and CobbleDB pulling those batches to serve reads. That split isolates hot serving nodes from the write-heavy crawl pipeline entirely — which is the fix for the noisy-neighbour contention described above, and it does not require writing a database. Our reading is that a meaningful share of the tail-latency improvement comes from that decoupling rather than from RocksDB. You can do the same separation on a managed store this quarter.

Should we build our own database? Usually no, and here is why

Give the other side its strongest form first. Perplexity's cost curve was genuinely adversarial: usage-based pricing on a workload where every user question moves several megabytes, growing with usage rather than with revenue per query. At 200,000 requests per second the engineering salary line is small next to the infrastructure line, and owning the store buys a latency budget the vendor will not sell. That argument is correct at Perplexity's scale.

It stops being correct surprisingly quickly on the way down. Below roughly the point where your storage bill exceeds the fully loaded annual cost of the two or three engineers who would own the replacement — and that comparison has to include the years after the launch — the arithmetic does not close. Before that point, the cheaper interventions are the ones to exhaust: a read-through cache in front of the hot keys, the DynamoDB Standard-Infrequent Access table class at $0.10 per GB-month against Standard's $0.25, provisioned rather than on-demand capacity for predictable traffic, smaller records by moving embeddings out of the primary item, and separating batch write traffic from the serving path.

The same logic governs when to shard rather than replace, and our position there is the same one: the right moment is later than most teams think. The instinct to build reappears at every layer of the stack, and the honest version of the custom versus off-the-shelf decision is that infrastructure is where buying wins most often, while the product logic on top of it is where building wins.

What to do this week instead

If you are seriously asking this question, write down three things before anyone opens an editor: your p99 target as a number, your access pattern in one sentence, and the annual spend you expect to save. If any of the three is a guess, the answer is no for now, and the work that follows is measurement rather than engineering.

If all three are firm and the arithmetic still closes, the next decision is scope. Perplexity did not replace its durable store — Pillar still holds the authoritative data, and CobbleDB is a serving cache with a strong opinion about batch reads. A narrow component with one access pattern and a rebuildable dataset behind it is a defensible thing to own. A system of record is not, and the distinction is the whole game. That is how we handle it in the custom software work we do: build the piece where your workload is genuinely unusual, and let the boring, well-served layers stay boring, which is also the cheapest route to architecture that scales with the business.