AHTL conformance and redaction¶
Audited revision: a996d59f1eb468e7c386747e15f4c7e58d08410e. This is an independent audit; no production fixes were made. Findings below distinguish reachable ingestion failures from public-library codec failures. Run the isolated Rust harness using the procedure.
PDF provenance and inspection¶
docs/ahtl/AHTL_v1_Complete_Technical_Specification_Volume.pdf is 19,921,768 bytes, PDF 1.7, 58 A4 pages, produced by WeasyPrint 68.1. SHA-256: 6219af31092df75c8aef33c99dfeac23ad258fb6c9a7a970e6e30e8288ab4abd. The source PDF was preserved.
All 58 pages were text-extracted with pdftotext -layout; the page ledger records the nonempty page inventory and chapter mapping. This does not mean every statement on every page was independently verified. Detailed comparison covered grammar, test vectors, binary layout, contraction and Hamming proofs, redaction, and reference implementation. Pages 30 and 35 were rendered at 1,500 pixels and visually inspected: the block-majority and contraction equations and the HDC basis code are legible; image/table layout preserves the relevant meaning. Other diagrams have text/source review rather than comprehensive visual inspection.
Root additionally rendered and inspected four contact sheets covering all 58 pages for layout and page continuity. No blank or obviously clipped page was observed at that overview scale. Small labels, individual table cells and equations were not all legible in those thumbnails; this is not a replacement for the two full-size checks or semantic conformance review. Contact sheets remain in /tmp/Infinity-Context/pdf-layout-review/ to avoid duplicating the reference publication in audit deliverables.
The PDF now carries v1.1.0 on its cover and Chapter 08 has the current six-section struct and splitn(6) parser. Prior ICX-AUD-030 is confirmed fixed for its specific stale-struct defect. Residual claims remain inconsistent; this is not evidence that every prior PDF problem was fixed.
Normative conformance matrix¶
| Requirement / source | Implementation / independent check | Result |
|---|---|---|
| Chapter 01 / PDF pp.8–15: outer delimiters, Profile A field widths, 8/64 Braille glyphs | ahtl.rs, braille.rs; Unicode U+2800–U+28FF chart |
Correct fixed-width codepoint/UTF-8 mapping; public fields permit noncanonical construction |
| Chapter 01 §2: LegendHeader and repeated EpochFrames | AhtlStream::parse_stream accepts one optional epoch header and then packets only |
Specification/API mismatch: legend and a second epoch header are rejected |
| Chapter 01 §2: all SafeChar exclusions in subjects and clause values | escape_subject, escape_clause_field |
Raw CR/LF, │, ⟪⟫ can remain in subjects; │, ⟪⟫ remain in clauses. Stream round trip may still work; this is conformance, not demonstrated injection |
| Chapter 01: lowercase canonical field hex; optional SliceMap | Parsers accept and normalize uppercase; hint validated structurally | Uppercase behavior documented; SliceMap range semantics and numeric overflow are deferred to consumers |
| Chapter 01: bounded clauses | Iterative splitn(64,'⇒') |
Prior stack exhaustion fixed for parsed text; API can still manually construct a deeper boxed chain |
| Chapter 01 §6.1 / PDF p.15: BLAKE3 canonical test vector | Python blake3 independent binding hashes raw_payload_bytes |
Digest recomputed in evidence; compare exact output, not the codec's own assertion |
Chapter 01 §6.2, Chapter 03 §4 / PDF pp.15,32: fc is Q1.14 |
FractalCoord::encode_fc_hex64 uses to_q15 |
ICX-CODEX-005, format mismatch |
| Chapter 02 / PDF pp.16–28: byte↔Braille bijection and all 256 glyph table | byte_to_braille, closed-form UTF-8, full byte unit test |
Mathematical bijection confirmed; does not establish token count or embedding orthogonality |
| Chapter 02 §4.2: 512 disjoint 20-bit majority blocks | spectral_bands_64 |
The unnormalized Hamming 1-Lipschitz bound follows because each input bit influences one output bit; not an injective encoding |
| Chapter 02 §4.2: full scan always evaluated, no false-negative candidate pruning | query_associative, Stage 1 candidate cap |
False as a shipped algorithm description; see ICX-CODEX-004 |
| Chapter 04 / PDF pp.34–38: i16 bundler and nonnegative tie threshold | Current i32 counters and coordinate-parity ties | Stale blueprint equations, even though current subsystem spec describes i32 correctly |
| Chapter 05 / PDF pp.39–43: multi-line JSON/YAML secret values | SecretRedactor continuation recognizes only next scalar |
ICX-CODEX-001 |
| Chapter 05: cryptographic replacement marker has zero mutual information | Category, location and occurrence remain observable | Conditional claim needs a threat/distribution model; constant replacement alone proves removal of matched value bytes, not unconditional zero mutual information |
| Chapter 07 / PDF pp.49–52: exact identifier retention, embedding orthogonality, zero hallucination, 18.4–72.1x SNR | No tokenizer/model oracle; capped line scanner and heuristic routing | Unsupported model-effectiveness claims; token spelling is not a proof about contextual keys/queries or model tool selection |
| Chapter 08 / PDF pp.53–58: six-section Profile A struct | Current ahtl.rs |
Struct/parser arity corrected; raw skeleton is not safely framed under all public API inputs |
ICX-CODEX-001 — YAML scalar credentials survive ingestion redaction¶
- Category: security / secret exposure. Severity: Medium. Confidence: High. Revision: audited HEAD above.
- Location:
crates/icx-codec/src/redact.rs,SecretRedactor::redact,ends_with_sensitive_key,redact_value_span;docs/alien-holographic-codec.md§5 and Chapter 05 §3.2. - Prerequisites and boundary: A caller supplies YAML containing a sensitive key with a literal (
|) or folded (>) block value to the normal pre-ingestion redactor. The credential need not use a recognizable vendor prefix or pass an entropy threshold. The trust boundary is untrusted tool/file content → redacted storage/model output. This is local credential exposure, not demonstrated remote exfiltration. - Proof:
SecretRedactor::redact("password: |\n lowentropyfirst\n lowentropysecond\npublic: ok\n")returns the entire string unchanged. Forpassword: >, only>becomes[REDACTED:CONFIG_SECRET]; both secret lines remain. The scalar's bytes survive the very gate that is advertised to remove multi-line YAML secret values. - Second continuation-coverage gap:
password:\n user=correcthorse\npublic: ok\nis valid YAML with password scalaruser=correcthorse(independently parsed using PyYAML).ContinuationLine::classifytreats identifier-shapedkey=valuetext as a nested structure and stops pending sensitive-value redaction, even though YAML does not use=to introduce a nested mapping. Both redaction passes leave this scalar unchanged. The real CLI also stores and recovers it unchanged: harness, results. This lead came from investigating the distinct idempotence failure 006; unlike the fuzzer assertion alone, this fixture demonstrates actual secret-coverage failure. - Independent end-to-end recheck: The root auditor ran real
icx init,encode-ahtl, andderef --rawwith isolated config/state/runtime/workspace paths. Both synthetic secret lines survived CAS and the public field remained. See recheck harness and command results. This establishes current CLI ingestion reachability beyond the library probe. - Impact: Sensitive block values can enter CAS, index-derived output, and downstream context. Model-facing delimiter escaping does not remove them.
- Counterevidence: Inline key/value, a value on the immediately following line, many vendor patterns, and PEM blocks have dedicated protections. Documentation correctly admits unknown patterns can pass. This finding is narrower: common YAML syntax under an explicitly sensitive key is advertised as supported but is not recognized.
- Recommendation: Track YAML block indicators and indentation, scrubbing the complete scalar through dedent, including
|-,|+,>2, comments and blank lines. If syntax-aware parsing is too costly, fail closed for sensitive keys with these indicators while preserving subsequent nonsensitive keys. Keep storage and model-facing paths consistent. - Acceptance: Add fixtures for literal/folded/chomped/indented scalars with CRLF, comments, empty lines and following public fields. Assert no scalar secret bytes survive both
redactand end-to-end ingest/deref. Confirm subsequent public data is retained. - Additional acceptance: Include YAML plain scalars containing
=on the line after sensitive keys and distinguish them from real nestedkey: valuemappings. Avoid a one-language heuristic silently changing another format's value semantics. - Prior: ICX-AUD-015 is partially fixed, not fully fixed for advertised multi-line YAML coverage. ICX-AUD-016's same-line PEM swallowing fix remains independently supported.
ICX-CODEX-002 — Profile A public skeleton field can inject extra stream packets¶
- Category: codec integrity / framing. Severity: Low. Confidence: High. Revision: audited HEAD above.
- Location:
crates/icx-codec/src/ahtl.rs,AhtlWirePacketV1::{render,parse},AhtlStream::parse_stream. - Prerequisites: A library consumer places untrusted text directly in the public
skeleton_bodyfield, or parses and rerenders a standalone Profile A packet. No existing hook path accepting this field directly was established. - Proof: Start with valid Profile A fields, set
skeleton_bodytok:v⟧\n⟦Ω:forged⊢admin:true⋮Ψ:⠀⠀⠀⠀⠀⠀⠀⠀, and render.AhtlStream::parse_streamreturns two packets.renderinserts the skeleton verbatim, whereas stream parsing terminates on the first⟧. - Impact: Library/relay callers cannot assume one typed packet renders to one stream packet. A second packet can assert an invariant it was never separately authorized to create. This does not establish bypass of current MMU Tier-0 sanitization or execution policy.
- Counterevidence: Promotion from Profile B builds the skeleton using escaped clause renderers, and normal skeletonization sanitizes untrusted text. Existing SliceMap forgery remediation is effective at that specific field. These controls limit current reachability and justify Low severity.
- Recommendation: Validate all Profile A components at construction and render, or encode skeleton delimiters bijectively and reject noncanonical standalone input. Prefer a typed skeleton over an unrestricted public String. Describe compatibility for formerly accepted raw skeletons.
- Acceptance: Arbitrary hostile Profile A skeleton property test must yield exactly one packet or an explicit error; standalone parse→render→stream must preserve packet cardinality. Include literal
%and nested delimiter combinations. - Prior: Related to ICX-AUD-025; that slice-specific defect is fixed, but equivalent framing coverage was not generalized.
ICX-CODEX-005 — Fractal wire codec disagrees with normative Q1.14 scale¶
- Category: specification / interoperability defect. Severity: Medium. Confidence: High. Revision: audited HEAD above.
- Location:
crates/icx-hdc/src/fractal.rs,encode_fc_hex64,decode_fc_hex64,to_q15; Chapter 01FractalCoordField, Chapter 03 §4, canonical Profile A test vector. - Prerequisites: An independent consumer follows the normative Q1.14 wire description or exchanges coordinates with
FractalCoordwire methods. - Proof: Coordinate +0.5 encodes as
4000(round(0.5×32767)); normative Q1.14 requires2000(0.5×16384). Decoding canonical2000yields approximately 0.25000763 rather than +0.5. Current Profile A parser validates hex length, not scale; internal same-codec round trips cannot catch this differential. - Impact: Cross-implementation distances and coordinate magnitudes disagree by approximately two. Most signs still agree, so Profile A→B orthant projection alone can conceal the defect.
- Counterevidence: Both Q15 and Q1.14 conversion APIs exist; encode/decode are internally consistent within Q15 quantization error, and the bus separately uses Q15. The problem is specifically the unversioned
fcwire contract, not Q15 arithmetic itself. - Recommendation: Choose one normative scale for Profile A, version or migrate existing archived coordinates, and keep bus serialization explicitly separate. If Q1.14 remains normative, use
to_q1_14/from_q1_14infcwire methods. Do not silently reinterpret existing records. - Acceptance: Independent literal vectors for −1, −0.5, 0, +0.5, +1 must assert exact wire words and decoded magnitudes; run vectors in both implementations and test old-version migration.
- Prior: ICX-AUD-031 numeric consistency is still partial in this area; prior G12 relabeling the example as Q1.14 did not synchronize the encoder.
Prior findings and rejected leads¶
| Prior ID | Independent disposition | Basis |
|---|---|---|
| ICX-AUD-015 | Partially fixed | URL/vendor/next-line fixtures added; YAML block scalars fail (001) |
| ICX-AUD-016 | Confirmed fixed for reported inline-PEM bug | Inline spans replace start..end and continue; runaway bounds remain documented |
| ICX-AUD-024 | Confirmed fixed | Hex epoch formatting, 4–8 digit parser and DAG round-trip test |
| ICX-AUD-025 | Confirmed fixed at slice_hint | Invalid hints not rendered and rejected on parse; Profile A skeleton separate defect 002 |
| ICX-AUD-026 | Confirmed fixed for parser input | Iterative max 64 materialized links; no claim on arbitrary manually built/serde-recursive trees |
| ICX-AUD-030 | Confirmed fixed for stale Chapter 08 struct | PDF now has current fields and six-way split |
| ICX-AUD-031 | Partially fixed | Q1.14/Q15 mismatch 005; current core spec improved but AHTL chapters remain stale |
| ICX-AUD-032 | Partially fixed via supplement | Historical exactness review unchanged; corrections file supplies arithmetic and provenance caveats |
| ICX-AUD-039 | Partially fixed | Properties/fuzz targets exist; Profile A hostile fields and YAML scalar syntax absent from shown tests |
Rejected as findings: uppercase hex is explicitly accepted; invalid SliceMap omission is documented; permute(0) not being identity is explicitly explained in Chapter 04; Braille's 3-byte storage cost is intentional. None is evidence of memory unsafety. Regex/prefix source skeletonization is deliberately lossy; its limits must qualify the stronger 100%-identifier-retention prose.
ICX-CODEX-006 — Redaction changes again when a previously redacted payload is processed¶
- Category: correctness / output stability. Severity: Low. Confidence: High. Revision: audited HEAD above.
- Location:
crates/icx-codec/src/redact.rs,SecretRedactor::redact,ContinuationLine::classify, vendor prefix replacement inredact_line;fuzz/fuzz_targets/redact.rs:10. - Discovery and prerequisites: The redactor's own fuzz target asserts
R(R(s)) == R(s). It found a synthetic 153-byte valid UTF-8 string after about 540,000 executions. A caller only needs to process an already redacted string again, a normal possibility in ingest/sanitize/deref layers. No external credentials or unusual memory behavior are needed. - Reproduction: Minimizer, raw escaped evidence, 22-byte minimized binary, readable fixture. The minimum-by-deletion input is
TOKEN:\nPsk-ssVk-kP-P=S; pass 1 yieldsTOKEN:\nP[REDACTED:API_KEY]=S, pass 2 yieldsTOKEN:\n[REDACTED:SECRET]. A readable equivalent ispassword:\n ask-123456789=x\n. The independent state-debug auditor reproduces the same mechanism withTOKEN:\nPsk-OyyyTOKEN:(source, output). - Artifact integrity: Original SHA-256
565d4de317ac8f92eae0c6f1df205d47ec8a00a6b4a0c4331df645fe59921230;22-byte deletion-minimal SHA-2566881bcdd47c3972c7e4f56380851c51981bb59482bf0c1072e29563026c8fdfa; readable 28-byte fixture SHA-256e59908f4370963a497cb4f6f4634a3cb48d7d3b8252684091ff1e0418f8f377e. The JSON preserves original bytes as hex; original fuzzer artifact remains in isolated scratch storage. Minimality means no further individual-byte deletion preserves this failure, not a proof of a globally shortest synthesized input. - Cause: Pending sensitive-value continuation is classified from the original line. A nested-structure-looking identifier may contain a vendor-token substring; replacing that substring inserts
[REDACTED:...]punctuation. On a later pass, the mutated line no longer looks like an identifier-shaped structure and is treated as the pending secret scalar, so the entire line is removed. Prefix scanning and continuation recognition are not compositionally stable. - Impact: Additional content is removed on subsequent passes. Redaction output, downstream packet hashes and stored/model-facing views can differ depending on how many times the gate runs. The reported fuzz process terminates because the test assertion fails; the production redactor returns normally for these inputs. No production crash, memory-safety violation or secret exfiltration is established by this idempotence fixture. The separately confirmed YAML equals-scalar leakage is merged into 001, not counted twice here.
- Counterevidence: Existing random hostile-text properties and many normal fixtures pass, and the embedded vendor substring is removed on the first pass. The problem is output stability and unintended further erasure, not proof that the matched vendor credential survives.
- Recommendation: Make continuation/structure decisions stable across vendor replacement, using typed format/context state or a canonical redacted representation. Do not blindly loop until convergence: that can hide the original semantic classification error and over-redact legitimate structures. Preserve the arbitrary-input idempotence property as a real gate.
- Acceptance: Add exact original/minimized/readable regression vectors and assert first-pass expected semantics,
R(R(s))==R(s), and stable ingest→model-view behavior. Retain fixtures with real nested mappings, comments, quoted scalars and=-containing YAML values to prevent trading stability for a secret leak. Replay the original fuzz artifact, then resume the campaign with the known failure explicitly reported until a production fix is separately authorized. - Prior: ICX-AUD-039 remains partially fixed: fuzz coverage is valuable and discovered a real violated invariant. ICX-AUD-015 covers adjacent redaction recognition gaps; this idempotence failure is a new finding.