v3.1.0 release notes

Release Date: 2026-09-03 Upstream Base: CyberChef v11.4.0 (unchanged) Licence: GPL-3.0-or-later Node: >=24 <27 Image: ghcr.io/doublegate/cyberchef-mcp_v3:3.1.0

Highlights

Measured, not asserted. This release replaces the largest self-assessment left in the project: protocol conformance verified only by tests this repository wrote.

v3.0.0 shipped “MCP 2026-07-28 conformance” on that basis. An official suite — @modelcontextprotocol/conformance — had published scenarios for the exact SEPs it implemented four weeks earlier, and nothing here referenced it. It now runs in CI against both protocol eras, and it found a real defect on its first run.

The planned scope was an evaluation harness. Measuring the charter first — the ritual RE-MEASURE.md makes mandatory — fired the charter’s own first kill criterion, and re-scoped the release. All six findings are in docs/internal/v3.1.0-findings-log.md.

The external oracle

npm run conformance

2026-07-28   96 passed, 31 baselined
2025-11-25   45 passed, 23 baselined

141 checks pass. The ones worth naming are v3.0.0’s work, checked by someone else’s code:

Scenario	Result	Covers
caching	7/7	SEP-2549 ttlMs/cacheScope on all five list methods
server-stateless	26/28	SEP-2575 _meta validation, server/discover, capabilities matching handlers
http-header-validation	14/14	SEP-2243 Mcp-Method/Mcp-Name, -32020 HeaderMismatch
sep-2164-resource-not-found	4/4	Error code, no empty contents, and the data.uri SHOULD fixed below

v3.0.0 chose its cache TTLs by argument. An independent implementation of the SEP now agrees they are conformant — the first time any protocol claim in this project has been confirmed by code this project did not write.

Why the failures are baselined and not filtered. Most are the harness calling the SDK reference server’s own fixtures (test_simple_prompt, test://static-text), which no server with its own surface can pass. Running only the applicable scenarios would be written once and never revisited, and a scenario added by a later release would never run. --expected-failures inverts that: every scenario runs, each non-applicable result carries a written reason, and the build fails when a baselined entry starts passing. See conformance/README.md.

Nothing in that baseline is a deferred defect.

Fixed

A resource error omitted the URI the caller asked for

WARNING  sep-2164-data-uri
         Error data.uri is undefined, expected "test://nonexistent-…". This is a SHOULD requirement.

readResource has two -32602 paths. The missing-recipe path carried data.uri; the unsupported-scheme and empty-id paths did not — deliberately, because v3.0.0 wanted the two distinguishable by exactly that. Both now carry it, and they stay distinguishable by whether data.supported is present. Same information, and the SHOULD is met.

1,426 in-tree tests could not have found this. They assert the shape this server was written to produce; only an independent encoding of the specification asks the question the other way round. That is the argument for the whole release, delivered by the release itself.

cyberchef_describe_operation ignored a required argument

Calling it with the singular operation — the tool’s own name, and the spelling this repository’s own prose uses — returned:

{"operations":[{"operation":"","error":"No such operation. Use cyberchef_search to find the exact name."}]}

The caller is told the operation it never named does not exist, pointed at a search tool it does not need, and never told which argument was missing. isError was unset. v2.2.0 made unknown arguments an error rather than a silent default; this is the same defect from the other side — a missing required argument silently defaulted — and it survived because the two look nothing alike.

It now names the expected argument and echoes the keys the caller sent, matching the neighbouring cyberchef_list_operations. Found by the first run of the new surface benchmark, which recorded a 197-byte operation “schema” that turned out to be this error.

Measured: what each tool surface costs

npm run measure:surfaces

  surface     tools      bytes       KB
  index          28      20297       20
  curated       106      83543       82
  all           531     400701      391

  index + one operation schema: 22,075 bytes  (18.2x cheaper than `all`)

Every figure in tool-catalog.mjs’s header had drifted — the index, the surface the whole design rests on, was documented at ~10 KB and ~24 tools. The argument held by a wide margin throughout, which is exactly why nobody noticed: a claim that stays directionally true is the hardest kind to keep numerically true. The last line quantifies the trade the index makes for the first time.

Bytes, not tokens. No tokenizer has ever been in this repository, and every ~N tokens figure it published was bytes divided by four. Adding one would not fix that: the consumer is Claude, Anthropic does not publish its tokenizer as a package, and counting with a GPT tokenizer would substitute a second unvalidated proxy for the first while looking more rigorous.

Not built, with the reason

The charter also asked for task-level scoring — accuracy against ground truth, tool calls made, redundant-call patterns. That needs a model in the loop: an API key in CI, a per-run cost, and a number that moves when the model changes underneath it, in a repository whose gating philosophy is that a flaky number must not be a gate. The charter’s own second kill criterion anticipates this.

So the split is deterministic-and-gating versus model-dependent-and-deferred, stated in full as F-06. performance-benchmarks.yml already occupies the honest version of the second position, saying in its own output that it cannot fail on a regression.

Upgrading

docker pull ghcr.io/doublegate/cyberchef-mcp_v3:3.1.0

No breaking changes. A client that matched on the absence of data.uri to detect a malformed resource URI should match on the presence of data.supported instead — both are -32602 and both now carry the URI.

Verification

npm run lint            zero errors
npm run check:versions  every version reference and package major agrees
npm test                241 Node-API + 2,289 operation tests
npm run test:mcp        1,428 tests across 54 files
npm run test:coverage   95.47 lines / 89.91 branches / 96.40 functions / 96.20 statements
npm run conformance     both eras match the baseline

The conformance gate was proven in both directions before it was committed: reverting the SEP-2164 fix produces an unexpected failure, and baselining a scenario that passes produces a stale baseline entry. A gate that cannot fail is the defect this project keeps writing up.