Creuto is now an OpenAI Select Partner Read More

Software Architecture & Technical

AWS Emulator vs LocalStack: which tests go where

AWS Emulator vs LocalStack: AWS says its emulator can't test IAM, and LocalStack's free plan bars commercial use. Here is the test-by-test split.

AWS Emulator vs LocalStack: which tests go where

AWS Emulator vs LocalStack is a licensing question and a test-design question. AWS's emulator has no additional service fee and is dedicated to the public domain, but AWS's own documentation says not to use it to test IAM or any other security control. LocalStack's free Hobby plan prohibits commercial use, so commercial pipelines need a paid licence.

We have not run AWS Emulator in a client pipeline. This comparison rests on AWS's user guide, its repository, LocalStack's licensing and pricing pages, and the fidelity scores AWS published with release 1.0.471. We read all of them on 17 November 2026.

AWS Emulator vs LocalStack: what each one permits and costs

AWS released the emulator on 11 November 2026. LocalStack changed its own terms earlier in the year. The two now differ on price, licence, sign-in and where state lives.

QuestionAWS EmulatorLocalStack
Price"No additional service fee", and no AWS account needed (user guide)Hobby is free. Base is $39 per licence per month billed annually, or $45 billed monthly. Ultimate is $89 billed annually (pricing page)
Commercial useCC0 licence, except code in vendor/ under Apache 2.0 (README)Base, Ultimate and Enterprise carry commercial-use licences; Hobby is non-commercial (plans docs)
Sign-inNone. It generates its own credentials on each run (security page)A mandatory auth token starts the container (auth token docs)
SourceOn GitHub; the repository does not take pull requestsThe open-source repository has been archived and read-only since 23 March 2026
StateHeld in memory and deleted when the emulator stopsLocal state persistence is listed from the Base plan upward

The LocalStack row that matters most is Hobby. LocalStack's pricing page says the plan's terms "prohibit use for commercial purposes", and that anyone running LocalStack for commercial CI/CD or shared team pipelines should choose a paid plan or talk to sales. Licences are assigned to individual users, and LocalStack asks buyers who will use it mainly in automated environments or on shared infrastructure to contact its sales team. A CI-only setup therefore doesn't map neatly onto a per-seat price. For scale, ten Base licences billed annually come to $390 a month at the listed price (our arithmetic).

What AWS says AWS Emulator cannot test

AWS's security page is blunt. It tells you not to use the emulator to test IAM policies and permissions boundaries, resource-based policies such as bucket policies, AWS KMS key policies, access across accounts or Regions, or any other security control. It also says "The emulator never denies a request signed with its own key", so a request that AWS denies might succeed in the emulator, and one that AWS allows might fail.

The user guide lists the other limits:

  • One account and one Region per emulator. It accepts any Region and account ID but doesn't keep resources separate; to test more than one Region you run a separate emulator for each, on its own port.
  • Behaviour is checked against AWS only in US West (Oregon), except global services, which are checked in US East (N. Virginia).
  • All resources live in memory and are deleted when the emulator stops.
  • AWS hasn't tested it for long-running use or load testing.

InfoQ's coverage calls it "not a drop-in replacement for every workflow" the older tools cover, "especially permission testing and infrastructure-as-code pipelines". That is InfoQ's framing, and AWS's pages never mention LocalStack. The permission point matches AWS's own wording. The infrastructure-as-code point matches AWS's statement that support for CloudFormation, Terraform and the AWS CDK varies by service.

What the published fidelity scores show for S3, SQS, DynamoDB and CloudFormation

AWS defines a fidelity test as a program that runs a workflow against a service through an AWS SDK and asserts the result, the resulting state and, for invalid calls, the exact error. A test passes only if every call matches AWS, including status codes and side effects. An operation the emulator hasn't implemented returns HTTP 501 with the code NotImplemented, so a test that hits one fails immediately. AWS says the match is closest for control plane APIs, while data plane behaviour and infrastructure-as-code support vary by service.

AWS publishes the scores as a scores.json.gz file on each GitHub release. We tallied the file for release 1.0.471, scored on 16 November 2026. Of 43,548 scored units, 30,237 pass, or 69%. The services most test suites touch look like this:

ServicePassingShare
S32,172 of 2,84576%
DynamoDB1,657 of 2,14277%
SQS1,173 of 1,53676%
Lambda2,103 of 3,02170%
EC21,346 of 2,49954%
CloudFormation1,022 of 4,28224%

Treat those numbers as a snapshot with a stated method. We counted each unit once for every service it touches, so a CloudFormation test that creates an S3 bucket counts toward both. AWS's own per-service scores may be counted differently, and they change with every release. A developer quoted by InfoQ counted S3, DynamoDB and SQS at about two in three and CloudFormation at under one in five. Our tally is higher for both groups, which is a reason to run the check on your own operations instead of trusting any single count.

AWS describes that check. List the operations your workflow calls, find the matching tests in the repository's manifest, compare the scores for the release you use, and for anything no test covers, run your own tests against both the emulator and AWS and compare the results. A green run proves only that your code agrees with the emulator; our post on what coverage hides in AI-generated tests covers a related trap.

Which tests belong on which environment

Decide per test, not per tool. This split is our judgement, built from the evidence above and as of 17 November 2026.

What you are testingWhere to run itWhy
IAM, bucket and KMS key policies, cross-account accessA real AWS sandbox accountAWS lists all of these as things not to test on the emulator
CloudFormation, Terraform or CDK deploymentsReal AWS, or your current setup until you have checked your own templatesSupport varies by service, and 24% of CloudFormation-touching units pass in 1.0.471
Application code calling S3, SQS or DynamoDBAWS Emulator, after checking your operations against the manifest76% to 77% of scored units pass, and invalid calls return AWS's own errors in the documented cases
Lambda execution and event wiringRun it on the emulator and real AWS and compare70% of Lambda-touching units pass, so don't assume parity
State that must survive a restartLocalStack Base or higher, or real AWSThe emulator deletes all resources when it stops
Load, soak and multi-Region testsReal AWSOne account and Region per emulator, and no load testing by AWS

None of this is a verdict on LocalStack's fidelity, which we haven't measured. Its plans docs list IAM policy enforcement from Base upward, but AWS's advice to test security on real AWS is the safer default for any emulator, in our view.

The case for staying on LocalStack, or skipping emulators altogether

The strongest argument for LocalStack is maturity. Your tests already run on it, it persists state on paid plans, and its plans docs list standard support on Base and priority support on Ultimate. AWS Emulator is days old. Amazon says autonomous agents wrote every line and that its "team hasn't written or reviewed any of the code". The guardrails are real, including tens of thousands of fidelity tests verified against real AWS and a merge gate for each change. Still, GitHub's releases page shows the version number going from 1.0.385 on 14 November to 1.0.471 on 16 November, and AWS warns that a release can fix a response so it matches AWS: "If your tests relied on the earlier response, they can start to fail."

The strongest argument for real AWS sandbox accounts is that nothing else evaluates your actual policies. AWS itself says to test security settings and the whole app on real AWS before production. The cost is account structure, per-test cleanup and real charges, which is cloud-platform work as much as test work. Going this way means designing sandbox accounts and guardrails before the first test is written.

Don't switch if your pipeline tests CloudFormation or Terraform deployments, if you rely on policy enforcement, if you need state to persist between runs, or if you can't tolerate a dependency that changes daily. Do consider it if your commercial tests mostly make S3, SQS or DynamoDB calls from application code and a seat-based licence is the obstacle.

Pin the version, keep it patched, and keep CI disposable

AWS gives two pieces of advice that pull against each other. It recommends installing a specific version if a new release changes your results, and it publishes version packages for that. It also says it publishes "security fixes only in the latest release" and that a fix might need a breaking change, even within a major version. Pinning buys stable tests and costs you security fixes until you upgrade, so schedule the upgrade.

The security page adds a CI constraint. When the emulator runs function code for you, it doesn't isolate that code from your machine; the code runs with your user's permissions. AWS also tells you not to run it on a machine other people can sign in to. Use disposable runners. Telemetry is on by default and can be switched off with --telemetry off or EMULATOR_TELEMETRY=off, which a commercial pipeline's data-handling review will want to know. On macOS and Windows you build the container image yourself from AWS's Dockerfile.

A one-sprint pilot looks like this:

  1. List the AWS operations your integration tests call.
  2. Check them against the manifest and the scores for one pinned release.
  3. Run the same tests against AWS Emulator and your current setup in parallel.
  4. Add a scheduled job that tries the latest release before you move the pin.
  5. Keep one stage on real AWS for permissions and infrastructure-as-code tests.

Wiring a pinned emulator, a version-bump job and a real-AWS stage into one pipeline is CI/CD implementation work, not just test work.

Split your integration tests by environment before you pick a tool

The choice between AWS Emulator and LocalStack matters less than the sorting that comes first. Put API-level tests of S3, SQS and DynamoDB in one bucket, policy and infrastructure tests in a second, and stateful or load tests in a third. Then price each bucket: an emulator with no service fee, a per-seat LocalStack licence, or real AWS accounts. Re-run the sort when either tool's terms or scores change.

If you want help sorting an integration suite into those buckets and wiring the result into CI, our QA and automation team can work through it with you.

Frequently asked questions

AWS Emulator can replace LocalStack for some application-level tests, such as calls to S3, SQS and DynamoDB, but not for IAM or other security testing, which AWS's documentation says it cannot do. Release 1.0.471 passes 24% of CloudFormation-touching scored units, so infrastructure-as-code pipelines need checking before any switch.

No. LocalStack's pricing page says the free Hobby plan's terms prohibit commercial use. As of 23 March 2026 its docs list Base, Ultimate and Enterprise as the subscriptions with commercial-use licences, with Base priced at $39 per licence per month billed annually. Teams using it mainly in CI are asked to contact sales.

No. AWS's security guidance says not to use it for IAM policies, permissions boundaries, bucket policies, KMS key policies or cross-account access, because the emulator never denies a request signed with its own key. A request AWS denies might succeed there, so test permissions against real AWS before production.

Amazon says autonomous AI agents wrote every line of AWS Emulator and that its team hasn't written or reviewed any of the code. Agents write fidelity tests verified against real AWS, file issues and fix them behind a merge gate that checks build, standards, security and fidelity regressions. The repository takes no pull requests.

Pin one if a new release changes your test results, which AWS recommends, and AWS publishes version packages for that. The trade-off is that AWS ships security fixes only in the latest release, so schedule regular tests of the newest version and move the pin rather than leaving it fixed indefinitely.

Written by

Akash Mohapatra

Akash Mohapatra

Co Founder & Director

16 Nov 2026

·

9 min read

Share

LET'S CONNECT

Connect with Creuto!

Ready to take the first step towards unlocking opportunities, realizing goals, and embracing innovation? We're here and eager to connect.