Blog
Agentic Infrastructure
15 July 2026

Most teams run pytest serially. That's the whole opportunity.

Blog
Most teams run pytest serially. That's the whole opportunity.

rstest is now open source: a pytest-compatible runner with a Rust orchestration core.

Here's the thing almost every pytest benchmark quietly skips: hardly anyone runs pytest-xdist by default.

Parallelism isn't pytest's out-of-the-box behavior. You install xdist, you opt in with -n, and then you keep the suite parallel-safe by hand. Most teams never finish that work, so the suite they actually run, day to day, in CI and on their laptops, is serial pytest.

That gap is the entire pitch. rstest is parallel by default (-n auto). The speedup is just there on the first run. No flags, no plugin, no migration.

Keep everything. Change nothing.

Your tests, fixtures, conftest, marks, and plugins stay exactly as they are. rstest runs them through a vendored pytest core, so the semantics are preserved rather than re-implemented. At -n 0, outcomes are pytest-exact. In parallel modes, they're preserved for parallel-safe tests.

Under the hood it works differently from xdist. Where xdist distributes by file, rstest dispatches at test granularity, with duration-aware scheduling and a warm duration cache. Crashed workers respawn without losing the run.

The numbers

Measured end-to-end on real suites (Apple Silicon, CPython 3.13, pytest 9.0.3) with per-test outcomes diffed against the pytest baseline:

SuiteSerial pytest (what you run today)rstestParity
SQLAlchemy533s53s99.97%
aiohttp (4,469 tests)197s68s100%
pandas (193,627 tests)182s63s100%
django-allauth (2,050 tests)22s8s100%

SQLAlchemy is the one I'd point at. 10x, and it's not a toy suite; it provisions a separate follower database per worker through xdist's pytest_configure_node hook. rstest isn't xdist, but it impersonates those hooks faithfully enough that SQLAlchemy's own plugin fires, each worker self-assigns its database, and the whole thing runs green.

That 99.97% is honest, not a caveat I'm hiding: 7 IMV/RETURNING tests that the serial baseline skips actually pass under any parallel runner. It's a serial-vs-parallel artifact, verified identical under real xdist, not a bug.

It handles monorepos, too

Point rstest at langgraph's root and it discovers all 8 libs/* packages, each with its own pytest config and rootdir, something a single repo-level pytest invocation can't do at all. It runs them concurrently and parallelizes within each, planning every package's worker share from its duration cache.

Six serial pytest invocations (880s) collapse to 121-133s warm, a 6.6-7.3x speedup, at 100% parity across all 4,284 tests.

Where it doesn't help (because you should know)

I'd rather you trust the wins than be surprised by the non-wins:

  • Small suites barely change. Under ~10s serial? Expect no meaningful gain.
  • CPU-bound suites cap at core count. Physics is physics.
  • Wins grow with suite size, and they're largest for wait-bound suites: the ones spending their time on I/O, network, and database round-trips.

When a suite doesn't speed up, it tells you why

rstest --doctor reuses data the run already owns (per-test wall vs. CPU time, per-fixture setup) to show where the time actually goes:

================== rstest doctor ==================
4442 tests, 185.8s test time (wall 67.7s, 8 workers)

WAIT-BOUND: 95% of test time (176.5s) is waiting, not computing (sleeps / IO / timeouts).
    54.20s waiting of 54.25s  tests/test_proxy_functional.py::test_proxy_https_multi_conn_limit

PARALLEL FLOOR: the longest test (54.2s) exceeds the ideal per-worker share (23.2s at -n 8);
no worker count can finish faster than its longest test.

SLOWEST FILES:
   150.46s (81.0%)  tests/test_proxy_functional.py
===================================================

That's aiohttp: one file is 81% of total test time, almost all of it waiting on 10-second proxy timeouts. No runner can parallelize away a single 54s test, but now you know exactly which one to fix. (--watch also gives you instant reruns on save.)

Onboarding is one command

You don't have to take any of these numbers on faith. Point --try at your own suite: no config, no commitment. It runs your tests through both pytest and rstest, diffs the outcomes for parity, and reports the speedup:

$ rstest --try
================= rstest --try =================
  ✓ parity:  8337 tests — identical outcomes to pytest
  ⚡ speed:   pytest 4.5s  →  rstest 2.9s   (1.5× at -n auto)
================================================
  → drop-in ready: `rstest` is `pytest`, in parallel.

That's the whole pitch in one line of output: same outcomes, less time, nothing to migrate. If the win is there, you'll see it on the first run; if it isn't, --doctor will tell you why.

Proving it's safe before you roll it out

--try answers "is it faster." The next question a team asks is "will parallel break anything," and that is the question that stalls most xdist migrations. rstest --migrate-check answers it for you. It isn't a test run, it's a parallel-readiness report.

It works in two stages. First it collects the suite twice and diffs the id sets, so any test whose id shifts between collections (an address in a repr(), a uuid, a timestamp baked into a parametrize) is caught before it can desync the workers. Then it runs at -n auto and classifies every test that fails only under parallelism, and names the fix for each:

$ rstest --migrate-check
================ rstest migrate-check ================
collection: stable (2 collections, 8337 ids identical)

parallel classification (3 findings):
  ORDER DEPENDENCY   tests/test_cache.py::test_warm_then_read
      passes serial + loadfile, fails under load
      POLLUTED BY: tests/test_cache.py (same-file co-location)
      fix: --dist loadfile, or fix the in-file coupling
  ISOLATION LEAK     tests/test_settings.py::test_override
      fix: reset the leaked global; stopgap @pytest.mark.serial
  WALL-CLOCK         tests/test_ws.py::test_reconnect_deadline
      wait-bound, fails under load; mock the clock

exit 1 - 3 parallel-unsafe tests
======================================================

For the ordering and isolation findings it even bisects the polluting file and tells you which one. And because it exits non-zero, the same command doubles as a CI gate: --migrate-allow tests/legacy/ tolerates a triaged backlog so the build goes red only on new parallel-unsafe tests, not the ones you already know about. That's the difference between flipping it on and hoping, and flipping it on and knowing.

Then wire it into CI

The laptop speedup is the hook. The reason it sticks is that the CI story is built in, not bolted on.

  • Shard one big suite across the runner matrix. rstest -n auto --shard 2/4 runs job 2 of 4. Buckets are balanced by wall time from the same duration cache the scheduler already keeps (longest-processing-time bin-packing, not even counts), and the split is deterministic, so N jobs agree on who runs what with zero coordination. Merge the per-shard JUnit and you have the full run back.
  • On a PR, run only what changed. Bare --changed in a pull-request job diffs against the merge-base with the PR's base branch (it reads GITHUB_BASE_REF), so a clean checkout selects exactly that PR's affected files. It's monorepo-aware too: unaffected packages are skipped, dependents are pulled in.
  • Failures show up where you read them. --output github emits a GitHub Actions ::error annotation per failing test, right on the diff, with modes for Azure, GitLab, Buildkite, TeamCity, and TAP as well. Any --doctor run also appends its report to $GITHUB_STEP_SUMMARY, so "where did the time go" lands on the run page with no extra step.
  • Gate before code lands. rstest ships pre-commit hooks: rstest-changed on pre-commit for the affected tests, rstest on pre-push for the full suite. Pin it in .pre-commit-config.yaml and you're done.

Flaky tests get a lifecycle, not a shrug

Parallel runs surface flakes, that's the honest cost of concurrency. rstest hands you three tools for the whole lifecycle instead of leaving you to it:

  • --reruns 2 rescues a flake within a run: fail then pass on retry, the run stays green and the test is still counted and listed as flaky, not hidden.
  • A flake history (.rstest_cache/flakes.json, recorded automatically, and sparse so a green suite writes nothing) remembers which tests keep doing it across runs, so flaked 7x becomes a ranked fix-list instead of a hunch.
  • --quarantine quarantine.txt ring-fences the offenders a team has decided to tolerate while they get fixed, without hiding them and without letting new failures sneak through. The file is committed, so its diff is the audit trail.

xdist has none of this. It's the piece that turns "parallel made my suite flaky" into "parallel showed me which three tests were always flaky, and here's the fix for each."

Try it

If your suite is large, wait-bound, and currently serial, this is close to a free 5-10x. Keep your tests as they are and let the runner do the work.

rstest is open source now: https://github.com/KovantAI/rstest. The README covers install, the full CLI, and the compatibility details I've glossed over here. Run rstest --try on your slowest suite and tell me what you see. Contributions welcome.