Conformance corpus
This directory contains the fixtures and expected outputs that any conforming implementation of UW Markdown self-certifies against. It is the canonical test corpus — analogous to the CommonMark spec corpus or OpenAPI's compliance suite.
The corpus is tagged by case, rather than by directory. Its authoritative runtime ownership index gives each reference check exactly one target: reader, verifier, one of module, integrity, representations, portfolio, sensitivity, stochastic, pro-forma, document-facts, or starter-products. Fixture directories describe content; they do not confer a tier claim.
| Target | Reference checks | Portable CLI cases |
|---|---|---|
| Reader | 59 | 44 |
| Verifier | 882 | 568 |
| Module | 76 | 1 |
| Integrity | 34 | 16 |
| Representations | 158 | 0 |
| Portfolio | 14 | 0 |
| Sensitivity | 5 | 0 |
| Stochastic | 7 | 0 |
| Pro-forma | 54 | 0 |
| Document facts | 43 | 0 |
| Starter products | 142 | 57 |
| Total | 1,474 | 686 |
How to self-certify
- Implement (or update) your tool against the published
UW_PROTOCOL.md. - Select your tier and explicitly claimed profiles; run their indexed cases through your tool.
- Compare your output to the corresponding
expected/output.
Frozen comparisons retain their existing subset, exact, normalized-byte or shape rules; re-tiering changes ownership, never an assertion. Publish the capability/profile claims and all skipped/unverifiable counts with the result. Starter-product results are separate compatibility claims, not conformance.
Running the reference test runner
The reference implementation ships a runner at scripts/run-conformance.mjs that exercises the corpus against @uwmd/core. CI runs all indexed checks on every PR. The required Build and test (Node 20) Main CI job also runs node scripts/run-conformance.mjs --profile=pro-forma explicitly. The54 pro-forma checks are native reference cases; they are not silently counted as portable driver cases. The support scope records the independent frozen implementation and actual-CLI evidence.
# Default: every indexed standard and starter check
node scripts/run-conformance.mjs
# Reader + Verifier, plus the Module profile
node scripts/run-conformance.mjs --tier=verifier --profile=module
# Bootstrap / refresh expected outputs from the current library
node scripts/run-conformance.mjs --update
# Machine-readable output
node scripts/run-conformance.mjs --jsonVolatile fields stripped before byte comparison: last_modified, _meta.provenance.timestamp, ts= fence attributes, and _meta.integrity.content_hash values (hashes canonicalize over the timestamp, so they vary per run even when content is stable). These change every run and are not normative.
Current counts come from the runtime ownership index and generated CLI cases. Accepted RFC 0088 adds 14 Verifier calc cases to the prior 1,106 default checks, with unchanged original expectations. They pin the selected primitive procedure, public versus internal exceptional values and retained IRR behavior. All 14 also run through actual CLI subprocesses. Portable cases project a subset of the same checks; they are not an additional corpus. The index and generated cases are checked for exact ownership, missing/duplicate results and generation drift.
Accepted RFC 0093 and RFC 0094 each add four authored native Verifier cases. RFC 0094 pins all 60 accepted T1B vectors, present leaf types versus absence, parent suppression, complete pass ordering and retained physical peers. Exact metadata assertions include META-01 through META-06. RFC 0095 adds six authored native controls for optional values, symmetric references, parent/pass ordering, retained peers, lawful unknown data and strict date-only calendars. The portable case count is unchanged; existing source and expectation assertions are not rebaselined.
Starter-product checks belong to the starter guide and remain blocking CI checks. A mixed check with product assertions stays a product check in full; where it also pins a registered validator or provenance code, a separate <scenario> [validate] Verifier case checks that code alone (CUR-01, LOC-01, SRC-01, POL-01, POL-02).
Ownership follows Protocol §II.2. Exact dictionary reproduction (same id, formula and unit as spec/metrics/builtin.json), the IRR procedure, the §VIII.1–VIII.3 expression grammar, variable resolution and built-ins (date-*, period-*, variant-*) are Verifier checks; the expression cases keep requires_capabilities: ["calc-evaluate"], so a Verifier that claims no evaluator skips rather than fails them. Module owns module manifests, the registry and custom declarations (the ADR × occupancy revpar-basic fixture). UW Lite compilation and projection (Lite spec §8) and the deal-package encodings (RFC 0018) belong to Representations.
Fixture source layout
The following tree describes fixture storage. Owners come from the index; directory and suite names describe content:
conformance/
├── documents/ Reader parse, Verifier validation, starter renders
│ ├── fixtures/ Well-formed .uwx.md records an implementer parses
│ ├── malformed/ Files exercising validator / integrity / policy codes
│ │ (CC-NN, FV-NN, DQ-NN, INT-NN, POL-NN, META_*)
│ │ plus optional <id>.policies.json siblings for
│ │ POL-* fixtures
│ │ <id>.validation.json (06, 07): Verifier [validate]
│ │ projection incl. provenance findings
│ └── expected/ Expected JSON parses + frozen validation verdicts
│ (<id>.validation.json: overall_status + every
│ (code, severity) pair) + display strings +
│ chat-format renders
├── starter/editor/ Starter products: round-trip writes, supersede
│ └── fixtures/ <scenario>/{before.uwx.md, operation.json, after.uwx.md}
│ Optional siblings: context.json (EditContext),
│ options.json (ApplyEditOptions, e.g. {integrity: true,
│ maintainGaps: true}), expected-error.json (negative
│ path — assert applyEdit rejects with a specific code)
├── calc/ Verifier calcs (dictionary, IRR, §VIII notation);
│ one Module custom declaration (revpar-basic)
│ ├── fixtures/ <scenario>/{deal.uwx.md, calc.json, expected-result.json}
├── starter/refinement/ <scenario>/{deal.uwx.md, expected-graph.json}
│ Starter dependency-graph checks
├── lite/ UW Lite representation + deal-summary-v1 bridge
│ ├── fixtures/ Well-formed .uw.md Lite documents that must parse
│ │ cleanly AND compile; each freezes five artifacts in
│ │ expected/ (see below)
│ ├── malformed/ Parse-time errors (LITE_*) with <id>.expected.json
│ │ declaring expected_codes; optional "must_parse": false
│ │ asserts parseUWLite throws instead; "detect" mode
│ │ asserts the representation check refuses the source
│ │ (structured content under a Lite name is
│ │ SOURCE_CONTENT_MISMATCH)
│ ├── compile/ Documents that parse cleanly but must FAIL the bridge
│ │ (LITE_COMPILE_*), same <id>.expected.json shape
│ ├── equivalence.json Groups of fixtures that differ only along axes
│ │ UW_LITE_SPEC §6 excludes; all must share one digest
│ └── expected/ Per fixture: <id>.canonical.json (RFC 8785 financial
│ canonical form), <id>.digest.txt (sha256 over its exact
│ UTF-8 bytes), <id>.rendered.uw.md (canonical rendering),
│ <id>.compile.json + <id>.uwx.md (deal-summary-v1
│ compilation), <id>.projection.json + <id>.projected.uw.md
│ (UWX→Lite projection with its omission report)
├── receipts/ Verification receipts (RFC 0016, spec/UW_RECEIPT.md)
│ ├── issue/ <scenario>/{deal.uwx.md|deal.uwx.md, expected-receipt.json}
│ │ Issuance is deterministic apart from issued_at, which
│ │ the runner stubs
│ ├── verify/ <scenario>/{deal.*, receipt.json, expected-verdict.json}
│ │ expected-verdict.json declares one of verified /
│ │ failed / unverifiable plus expected_codes (RCP-NN),
│ │ or refused_code for a receipt a verifier must not
│ │ read at all (receipt_version 1.0 is RCP_MALFORMED)
│ └── refuse/ <scenario>/{deal.*, expected.json} — issuance must throw
│ a typed ReceiptError with expected_code, never emit a
│ caveated receipt
├── market-data/ Market data as an attributable UW document (RFC 0022)
│ ├── valid/ <scenario>/{doc.uwx.md, expected.json} — a market-data
│ │ document parses to the expected identity/observations
│ ├── reject/ <scenario>/{doc.uwx.md, expected.json} — parse must
│ │ refuse with the expected typed code (a market doc
│ │ carrying a deal_id, duplicate field paths, …)
│ ├── resolve/ <scenario>/{case.json, deal.uwx.md, docs/, expected.json}
│ │ — selectCurrentMarketData over several documents:
│ │ most-recent as_of wins, ambiguity refuses, staleness
│ │ is reported rather than silently served
│ └── promote/ <scenario>/{case.json, doc.uwx.md, expected.json} —
│ promoteMarketObservation writes an observation into a
│ deal with provenance and confidence preserved
├── modules/ Declarative module manifests (see modules/README.md)
│ ├── accept/ <id>.module.json — loadModuleManifest accepts the full
│ │ declared surface (calcs, view models, round_to, …)
│ ├── reject/ <id>.module.json + <id>.expected.json — malformed
│ │ manifests refuse with the expected typed code
│ └── asset-classes/ RFC 0003 — module-declared asset classes. Four
│ <scenario>/{deal.uwx.md, module.json, expected.json}.
│ The first THREE share a byte-identical document and
│ differ only in what the host has loaded: module
│ present (resolved), absent but the declaration known
│ (degraded via fallback), neither (unresolved). A
│ cross-scenario invariant asserts the three files stay
│ identical — editing one to fix a failure would void
│ the whole demonstration, which is that the verdict
│ depends on the reader and never on the document
├── packages/ UW Deal Packages, RFC 0018 (see packages/README.md)
│ ├── accept/ <id>.manifest.json + <id>.expected.json — manifest
│ │ validation accepts, including extension link types
│ └── reject/ <id>.manifest.json + <id>.expected.json — dangling
│ links, duplicate member ids, … refuse with the
│ expected code; archive/zip invariants are asserted by
│ the runner without baselines
├── composition/ Composable UWX documents (RFC 0021). The suite exists
│ │ to prove I-1: an externalized record and its inline
│ │ twin share one canonical form and one digest
│ ├── resolve/ <scenario>/{record.uwx.md, parts/, inline.uwx.md,
│ │ expected.json} — resolution matches the inline twin
│ │ on canonical form + digest, never on source bytes
│ ├── unresolved/ A missing fragment leaves the section externalized —
│ │ never a partial rent roll that still totals
│ ├── reject/ One fixture per COMP-* refusal (dup key, count
│ │ mismatch, section mismatch, malformed part)
│ ├── composite/ <scenario>/{case.json, expected.json} — graph shape,
│ │ depth bound, cycle detection, stale-vs-failed
│ ├── inherit/ <scenario>/{case.json, expected.json} — nearest
│ │ ancestor wins; equidistant ancestors refuse
│ ├── rollup/ <scenario>/{case.json, expected.json} — verifyRollup's
│ │ three-state verdict; a failed child short-circuits
│ │ before any arithmetic runs
│ └── lite-projection/ The UWX→Lite projection names externalized sections
│ in its omission report and matches the inline twin
├── lease-up/ Lease-up schedule (RFC 0008, format spec §4.25).
│ Same dispatch style: {case.json, expected.json}
│ exercises verifyLeaseUpSchedule's three-state
│ verdict; {deal.uwx.md, expected.json} pins LU-NN /
│ CC-15 validator codes and an optional end-to-end
│ verdict. See lease-up/README.md.
├── cash-flow/ Calendar-anchored cash flows (RFC 0034, format spec
│ §4.26 / Protocol §VIII.9). Three dispatch kinds:
│ {case.json} → verifyCashFlowSeries; {deal.uwx.md,
│ decl.json} → evaluateCashFlowMetrics with pinned
│ quantized values; {deal.uwx.md} alone → CF-NN codes
│ + an end-to-end verdict. See cash-flow/README.md.
├── waterfall/ Distribution waterfall (RFC 0035, format spec §4.27
│ / Protocol §VIII.10). {case.json} → verifyWaterfall
│ recomputes the full allocation, three-state;
│ {deal.uwx.md} → WF-NN codes. See waterfall/README.md.
├── portfolio-relationships/ Portfolio sidecars (RFC 0015, Protocol §XV).
│ {profile.json} → validatePortfolioProfile with
│ PORT-NN pins, uninterpreted-type reporting, and
│ preservation checks; {package-manifest.json} → the
│ RFC 0018 projection bridge validates end to end.
│ See portfolio-relationships/README.md.
├── locale/ Display-locale negotiation (RFC 0001, Protocol
│ §III.1a). Shared deal + injected locale line: the
│ per-locale rendering pins (NBSP included), the
│ LOC-01 display refusal, calc invariance, and CSV
│ byte identity across locales.
├── currency/ Document currency identity (RFC 0046, Protocol
│ §III.1b): explicit code display and CUR-01 refusal.
├── capability/ Capability-token write authorization (RFC 0011,
│ starter guide §XIV). Each scenario edits the shared
│ deal.uwx.md under a generated token through the
│ @uwmd/signing reference verifier; pins POL-03's
│ typed reasons, the jti note, and the never-escalates
│ rule. Owed only under `capability-verify`.
├── capital-stack/ Typed capital stack (RFC 0026/RFC 0050, format spec §4.24).
Scenario kind is dispatched by the files a directory
carries: {case.json, expected.json} exercises
verifyCapitalStack's three-state verdict (a "variants"
key contrasts pref cash-vs-accrued); {agree.uwx.md,
mismatch.uwx.md, expected.json} exercises the
generalized CC-03 in both directions; {deal.uwx.md,
expected.json} asserts a typed validator refusal
(CS-WATERFALL-UNSUPPORTED); and {deal.uwx.md,
expected-metrics.json} is the no-stack single-loan
regression pin — every pack metric must equal its
pre-RFC value exactly. RFC 0050 adds split-coupon
positive and refusal cases.
├── source/ Source vocabulary (RFC 0031). Five
│ <scenario>/{deal.uwx.md, expected.json}. The split
│ (`provenance.source` actor-only,
│ `provenance.resolution` for the method) round-trips;
│ a resolution tag written in `source` is SRC-02, an
│ error; the retired colon form is neither well-formed
│ nor a human write; and 03-unmatched-supersedes is the
│ data-loss regression — an unrecognized source
│ supersedes instead of replacing, and a caller policy
│ list with no coverage refuses rather than grants
└── size-intensive/ The Protocol §XIII size-intensive registry (RFC 0027).
Seven named scenarios: the registry pins (§XIII.1–3
against the shipped table; primary-in-pack and
pack-paths-in-registry coverage in both directions),
the consumer pins (csv size_basis/size_quantity for
every worked example with the total_units
compatibility column; the report cover's RSF/Keys
facts; the office Lite round-trip), and the CC-13
behavior pair (warns-and-does-not-refuse on a
size-less office; silent for mixed_use, whose
resolveDealSize is null by design)
├── stochastic/ Distributions (RFC 0005, Protocol §VIII.8). Six
│ <scenario>/{deal.uwx.md, stochastic.json,
│ expected.json}. What is pinned is NOT the shape of a
│ distribution - that is a statistics question - but
│ that the same seed produces the same numbers.
│ Reproducibility is asserted IN-PROCESS with no
│ baseline (re-running must return an identical
│ summary), so it binds any implementation and not only
│ one matching our frozen figures; 02 asserts a
│ different seed produces a DIFFERENT summary, since a
│ seed that did not change the stream would make
│ reproducibility accidental. Each scenario declares its
│ own `exactness`: exact for uniform and triangular,
│ tolerance for normal, whose inverse-CDF tails call
│ log() - a function no standard requires to be
│ correctly rounded
├── sensitivity/ Two-axis grids (RFC 0007, Protocol §VIII.7). Five
│ <scenario>/{deal.uwx.md, sensitivity.json,
│ expected.json}. Round numbers on purpose - every
│ expected cell is checkable by hand, so a failing grid
│ points at the implementation and not at arithmetic
│ nobody can verify. `expected.grid` is a plain 2-D
│ array with null for a failed cell; cell errors are
│ asserted separately by coordinate and code.
│ 03-document-unchanged is the one that matters most:
│ it re-reads the swept paths afterwards, because a
│ sweep that WROTE instead of shadowing would silently
│ change the deal and nothing else would notice
└── signing/ Signatures, over two artifact kinds, sharing one
│ keys/ directory — a host that trusts a signer trusts
│ them for both. Every fixture is GENERATED by
│ scripts/gen-signing-fixtures.mjs: a signature over a
│ hash of the file it lives in cannot be hand-edited
│ into a meaningful state
├── blocks/ Block signatures (RFC 0010, Protocol §V.11). Six
│ <scenario>/{deal.uwx.md, expected.json} pairs.
│ expected.json names the keystore to verify against
│ (null = no signature backend at all), the expected
│ `ok`, the signatures_present / signatures_verified
│ counts, and the exact issue codes
└── modules/ Module manifest signatures (RFC 0002, Protocol §X.1).
Six <scenario>/{module.json, expected.json} pairs.
Each scenario is run under ALL THREE host policies
(ignore / verify-if-present / require), not just the
interesting one: 04-unsigned loading under
verify-if-present and refusing under require is the
entire policy distinctionThe lite, receipts, market-data, modules, packages, composition, capital-stack, lease-up, cash-flow, waterfall, portfolio-relationships, standalone, capability, locale, size-intensive, signing, sensitivity, stochastic and source suites are named rather than numbered: UW Lite is a source representation, a receipt is a detached artifact, market data and deal packages are companion document kinds, module manifests and composition are protocol machinery, and the capital stack and size-intensive registry are verified protocol surfaces, and signing is an optional capability — none is itself a protocol conformance tier. All run by default; select one alone with --suite=<name> (e.g. --suite=lite, --suite=signing).
signing belongs to the optional Integrity profile. Its required capabilities remain explicit; an unclaimed profile is unverifiable and an unclaimed capability is a reported skip, never a pass. Two block scenarios pin distinctions rather than happy paths — blocks/03-signed-unknown-kid must report INT-06 and never INT-07 ("I cannot check this" is not "this is forged"), and blocks/05-signed-no-backend verifies the same bytes as blocks/01-signed-valid with no key store at all and must report the signature as present-and-unchecked rather than passing it. blocks/07-signed-padded, 08-signed-unused-bits and 09-signed-partial-sextet refuse altered signature encodings as INT-07, even though the signed content and decoded signature prefix are unchanged. modules/ carries the same distinction as PROTO-MOD-071 vs -072, plus modules/06-malformed, which must refuse under ignore too: declining to verify is not a licence to admit a malformed signature into a frozen manifest.
document-signatures exercises RFC 0084 detached snapshot authentication under the optional Integrity capability document-signature-verification. Its relabeled, lifecycle-flipped and spliced records retain block attribution but fail document authentication. See the synthetic vectors.
Two receipt properties are asserted as invariants rather than baselines:
- Re-issuance stability (§4). Re-issuing over an unmodified record must reproduce the same
subject.digestand the sameresults. - Three-state verdicts (§5). A verifier must land on exactly one of
verified/failed/unverifiable, and must not collapseunverifiableinto either of the others.verify/04-unknown-packis the case implementations are most likely to get wrong.
Two Lite properties are asserted as invariants rather than baselines, so they hold for any conforming implementation regardless of frozen output:
- Rendering round-trip (§7). Parsing a canonical rendering must reproduce the source document's financial canonical form.
- Display equivalence (§6). Labels, headings, prose, field order, bullet character, whitespace, comma grouping, and equivalent numeric spellings are excluded from the canonical form, so fixtures differing only along those axes must hash to one digest.
Digests are computed by the runner with stock node:crypto rather than the library's own hash helper, so a frozen digest is meaningful evidence to a third-party implementer rather than a restatement of our implementation.
Adding a fixture
Add one (suite, scenario, target) row to corpus-index.json for every check reported by the reference runner. Add generated portable identities when the fixture has a CLI projection, then run npm run gen-conformance-cases. Ownership is explicit and closed; a directory name never assigns it. Coverage guards reject a fixture that has no indexed owner.
Standard fixtures are normative once merged; product fixtures specify starter compatibility. To add one:
- Open a PR with the fixture file and its expected output.
- Explain in the PR description what scenario the fixture covers and why it isn't already covered by existing fixtures.
- Reviewers will run the fixture against
@uwmd/core(the reference implementation) and against at least one third-party implementation to confirm the expected output is correct, not just what the reference happens to produce.
See ../CONTRIBUTING.md for the full contribution process.
Regenerating expected outputs
The expected/ files are generated by running the reference CLI against the fixtures. To regenerate after a non-breaking improvement:
node scripts/regen-conformance.mjsIf a regeneration changes a standard expectation, identify the normative change in the RFC and PR; its version label moves at release preparation. A changed starter expectation is a package quality-bar change, not a Protocol change. Neither should be regenerated merely to hide a failure.
Lease-up projection
The default lease-up-projection suite exercises RFC 0044 explicit cash-date mapping and candidate output. See lease-up-projection/README.md for its self-contained source/plan/expected cases. Run it alone with npm run conformance -- --suite=lease-up-projection after building core.
Property cash-flow assembly (RFC 0045)
The default property-cash-flow-assembly suite contains self-contained synthetic source/plan/expected triples. Cases exercise complete coverage, monthly/leap and same-date timing, reserve declarations, duplicates, currency, signs and original source-verifier refusals. These fixtures are engineering inputs, not real deals. Run npm run conformance -- --suite=property-cash-flow-assembly after building.
Standalone document kit (RFC 0048)
The default standalone suite exercises the worked examples under examples/standalone/: profile and lease-abstract validation, independent fragment parsing, inline/externalized canonical and semantic-digest equality, and package integrity/context-boundary behavior. Run it alone with npm run conformance -- --suite=standalone after building.
Same-day cash-flow reconciliation (RFC 0062, accepted)
Protocol 2.17.1's normative errata is accepted and prepared locally, but has not shipped in a release. cash-flow/valid-same-day-selection exercises full validation, metric verification and repeated/unique date selection on one preserved document. cash-flow/reject-duplicate-year-selection retains the other-series guard. The existing property assembly valid-same-day scenario additionally pins absent_validation_codes: ["PS-02"] while preserving its source and assembled row/binding checks. Receipt issuance baselines change only their protocol label to 2.17.1; engine/package versions are unchanged. Run the focused suites with:
npm run conformance -- --suite=cash-flow,property-cash-flow-assemblyRFC 0070 replacement funding
hedge/0070-* adds 57 cases: legacy/explicit escrow, outright/zero payments, same-day identity, expiration-independent timing, malformed or ambiguous references, contradictory escrows, stale series/index/variant/currency, reviewed rebind and the missing sources_uses/uses ESC-04 regressions. Existing 17 hedge fixtures are unchanged. expected.json carries structural codes separately from verification state/reason/codes; HDG-09 occurs only in the latter.
hedge/0075-* adds 6 cases for RFC 0075's senior preference, in the same expected.json shape: a senior beside a junior (outright and escrow), a junior keyed base, two seniors (still refused), a malformed senior hedge beside a junior, and a hedge stated only on the junior (not read).
hedge/0076-* adds 8 cases for RFC 0076: a stated hedge whose debt_structure (two seniors, components only) or sources_uses (two primaries) selection refuses reports one HDG-08 and verifies as unverifiable/unresolvable_source. The debt-only rules still run beside a refused sources_uses. A control with no hedge keeps ESC-04.
Unit controls cover unavailable crypto, pre-await immutability, known-answer Node/browser parity and explicitly unchecked consumers. The reviewed-rebind case intentionally targets an unrelated outflow with a fresh digest: UWMD can verify consistency, while producer purpose evidence must reject that mapping.
Metric dictionary verification (RFC 0078 Phase 2)
The metrics suite verifies every builtin example using the normative dictionary and safe-expression evaluator, without packs, defaults or cascade. It records 189 verified statements, zero failures and zero unverifiable statements, with per-example counts and per-statement evidence. Findings are compared exactly, and negative controls pin missing-input refusal and quantization. Run npm run conformance -- --suite=metrics; it is included in the default suite.