v3.6.0 release notes

One change: the same-host benchmark comparison becomes a gate, at 25% per task. That sentence has been written twice before in this project and both times the number was wrong within days, so this release is mostly the measurement that makes it stick.

It is also the first release in nine whose findings log opens with “the plan was right”. v3.5.0 said the threshold gets set from accumulated data in a later release and not before; the data accumulated, and this is that release.

What was measured

The noise floor

Four same-host runs on v3.5.0’s own pull request, all on code that changes nothing in the benchmarked path:

median +0.5%    worst slower -2.2%    worst faster +6.0%
median -0.1%    worst slower -1.7%    worst faster +3.1%
median +0.1%    worst slower -6.9%    worst faster +9.9%
median -0.1%    worst slower -7.6%    worst faster +2.0%

Plus -1.8% from two runs of identical code on a developer machine. Worst observed: -7.6%.

The detection curve, which no previous attempt ever measured

A noise floor answers “what will not fire” and says nothing about “what will” — which is precisely how the previous two thresholds came to be quoted as though they caught everything.

So To Hex was given a deliberate, tunable slowdown: it redoes a measured fraction of its own work and discards it. Its own work rather than a sleep or a busy loop, so the slowdown carries the operation’s real performance profile. Compared against the same worktree unmodified:

nominal extra work   To Hex 1KB   To Hex 10KB   To Hex 100KB   worst UNTOUCHED task
              0%          0.0%          0.0%           0.0%          0.0%
             10%         -4.5%         -7.4%          -8.2%         -3.7%
             25%        -11.1%        -16.3%         -20.2%         -3.1%
             50%        -19.8%        -27.7%         -32.8%         -2.5%

Two results. It detects roughly linearly — and the regression stays localised: every untouched task stays inside the noise floor at every magnitude. That second property is what makes a per-task threshold meaningful rather than a suite-wide smear, and it had been assumed rather than shown.

The threshold, and what it does not catch

25% per task, which is 3.3x the worst observed same-host noise and matches the 2.5-3x margin this project’s own precedent uses.

Stated in both framings, because conflating them is how a gate’s reach gets overstated:

the gate fires when a task is MORE THAN 25% SLOWER
in the experiment that took ~33% extra work

Verified by running the gate over the experiment’s own fixtures rather than reasoning about the numbers:

identical code                pass
two runner captures           pass
10% extra work  (-8.2%)       pass, reported
25% extra work  (-20.2%)      pass, reported     <- a quarter more work does NOT fail
50% extra work  (-32.8%)      FAIL
a zero-throughput task        FAIL               <- a broken run is not a clean one

A quarter of the work added to an operation does not fail this build. That is a narrower claim than either previous threshold made for itself, and it is the accurate one. The full comparison prints on every run, pass or fail, so the numbers below the threshold stay readable — a pass is not proof there is no regression, and the gate’s own output says so.

That last check matters on its own: a task with no usable measurement now fails rather than being quietly dropped from the comparison. A benchmark that produced nothing on one side is broken, not unchanged, and silently shrinking coverage is the failure this project keeps finding.

Why this is not the same mistake a third time

v3.4.0 set 20% from three captures taken sixteen minutes apart, and CI disproved it within the hour. Four runs from one pull request could be the same sampling error.

The difference is structural, not statistical. v3.4.0’s failure cause was host-class variation — memory-bandwidth-bound tasks collapsing on one host while CPU-bound ones improved on the same run. A same-host comparison is immune to that by construction: both sides run on the same host, in the same job. What remains is within-host noise, and two independent estimates of it agree in magnitude (CI -7.6%, developer machine -1.8%).

It is still four runs, so the gate is built so that being wrong is cheap and visible: it fails per-task, prints the full comparison either way, and the stored-baseline check continues to run on pushes where there is no merge base. A false failure arrives next to the numbers that explain it.

Everything else stayed parked, on purpose

Checked, as the re-measurement ritual requires, and none of it moved:

@modelcontextprotocol/server     2.0.0        unchanged
conformance                      alpha.11     unchanged
server.json schema               2025-12-11   current, validator clean
MCP spec revisions               nothing after 2026-07-28
chainguard/node:latest-slim      v25.9.0      the base-image trigger has NOT fired

So the shell-free base image stays declined, arm64 performance stays unmeasured for want of hardware, and v4.0.0 stays unscheduled — its kill criterion is that a major needs the specification to force one.

Verification

npm run lint               zero errors
npm run check:versions     every location agrees
npm run check:server-json  validates against 2025-12-11
mcp-publisher validate     valid, no deprecation warning
npm test                   241/241 Node API, 2289/2289 operations
npm run test:mcp           the suite, across 69 files
npm run test:coverage      exit 0
npm run conformance        both eras match the baseline

Upgrading

Nothing breaks. No runtime behaviour, tool or surface changes — this release makes a CI gate stricter, which affects contributors rather than deployments. A pull request that genuinely slows an operation by more than a quarter will now fail where it previously printed a number.