✅ fresh
last synced 2026-09-29T21:12:19.681221+00:00 · coverage 77% (
doc-sync)Validation by Beadloom
doc_sync— same source assync-check.
Doc Sync Engine
Mechanism for tracking synchronization between documentation and code.
Specification
How It Works
Doc Sync Engine compares document and code hashes to detect desynchronization through a multi-phase pipeline:
- build_sync_state -- finds doc-code pairs that share the same ref_id. A node's code files come from symbol annotations first and, when those yield none, from the files its declared
sourceOWNS -- read fromfile_index, so a module with no top-leveldef/class(a pure re-export facade) is paired rather than reported as absent code (BDL-061.50) - check_sync -- compares current file hashes and symbol signatures against stored baselines, then runs source coverage and doc coverage checks
The sync check pipeline operates in four phases:
- Phase 1: Hash and symbol drift detection. For each sync_state entry, compares on-disk file hashes against stored hashes. Symbols are compared at two granularities: the stored
symbols_hashis the whole NODE's surface and says the document's subject moved; the storedfile_symbols_hashis this pair's OWN code file and says whether it moved here. Only the second makes a pairstale/symbols_changed— a pair whose sibling moved isunverified/sibling_symbols_changedand names the file that did, because nothing about its own file changed to revise the document against (BDL-UX #182). A row with an emptyfile_symbols_hashkeeps the node-level answer, so a node already in drift when the column was added reports exactly what it reported before, until its next attestation. A pair whose doc or code file is GONE ismissing, notok; a pair whose only baseline was fabricated by a rebuild is corroborated against git, or reportedunverified. - Phase 2: Source coverage checks (
check_source_coverage). For each graph node with a directory-basedsource(ending in/), verifies that all Python files on disk are tracked in sync_state or code_symbols. Reportsuntracked_fileswhen gaps are found. - Phase 3: Doc coverage checks (
check_doc_coverage). For each graph node with a directory-basedsource, verifies that the linked documentation mentions all Python module names (file stems). Reportsmissing_moduleswhen the doc does not reference a module. - Phase 4: The declared surface (
find_missing_declared_docs). Every doc a graph node names in itsdocs:list is checked for existence. The declaration lives in the committed graph YAML, so deleting the file cannot remove it — which is what stops the gate being satisfied by having less to check (BDL-UX #174). - Phase 5: The document's SHAPE (
check_section_shape, BDL-061 S4b). The four phases above all compare CONTENT; this one compares structure, and it is the only phase that can see a document edited down to a title. It runs only when the caller suppliessection_requirements— the required sections are derived from the composed doc templates in theonboardingPEER domain, so they are passed in rather than read here. Nothing is written tosync_state. Seedoc-shape.
Where the baseline lives
Not in .beadloom/beadloom.db. That file is a derived cache — git-ignored, per-machine, dropped by every rebuild and absent on every fresh CI checkout — so a baseline kept only there is destroyed by the rebuild that most needs it: the rebuild records the tree it is indexing AS the baseline, and nothing can be stale against a baseline created a second ago (BDL-UX #175).
- Freshness lives in git. Each pair records where its baseline came from (
baseline_source:index_build/carried/attested). A pair whose baseline was fabricated at index-build time and would otherwise readokis corroborated againstHEAD(git_baseline.changed_paths); when git cannot answer, the verdict isunverified, never fresh. - The size of the surface lives in
.beadloom/sync-surface.json, committed, written only bysync-check --record-surface. A run whose declared-pair count FELL since it was recorded says so instead of printing the smaller number.
Sync Pair
@dataclass
class SyncPair:
ref_id: str
doc_path: str
code_path: str
doc_hash: str
code_hash: strStatuses
| Status | Description | Exit |
|---|---|---|
ok | Compared against a baseline and unchanged | 0 |
stale | Compared and drifted -- update needed | 2 |
missing | The doc or code file is gone, or the graph declares a doc that is not on disk | 2 |
unverified | There was nothing to compare against (rebuilt index, no git baseline). Reported by name; never counted as fresh | 0 |
incomplete | The document is current and does not carry the shape its kind requires. A warn: reported, never blocking | 0 |
exempt | The document is in the WORKING space and is exempt from freshness by DECLARATION (doc_roots.working in .beadloom/config.yml). The row carries the declared reason in details, prints as [exempt] with that reason, and is counted in the --json summary and in the gate line. Not ok — nothing was verified — and never blocking. The classification asks DocSpaces.project_path(doc_path), so the declaration reaches freshness and beadloom docs spaces in one spelling; the exemption covers freshness alone, and a pair whose document or code file is gone is missing before any exemption applies | 0 |
Every result also carries baseline -- index, git:HEAD or none -- so a green result says what it was green against.
Stale Reasons
| Reason | Description |
|---|---|
ok | No drift detected |
hash_changed | File hash on disk differs from stored hash |
symbols_changed | THIS pair's own code file changed its symbols (function/class signatures) while the doc hash remained the same |
sibling_symbols_changed | A different code file of the same node changed its symbols; this pair's own file did not. Reported as unverified with the moved file named in details -- not checked, and sync-update is not offered for it |
untracked_files | Python files in the node's source directory are not tracked in sync_state or code_symbols |
missing_modules | The linked documentation does not mention one or more module names from the source directory |
hash_changed_since_head | The code differs from HEAD while its doc does not -- drift the rebuilt index had absorbed into its own baseline |
doc_missing / code_missing | One side of the pair no longer exists on disk |
declared_doc_missing | The graph declares this doc and the tree does not hold it |
no_baseline | The index was rebuilt (its baseline is the tree it indexed) and git could not supply one -- not checked |
missing_sections | The document lacks sections a MAJORITY of its peers of the same kind carry |
section_not_in_use | A required section no majority of a kind's documents carries — reported once against the KIND, with its ratio, because the fix is in the template and not in every document |
surface_drift | Advisory warning (severity warning, never a hard failure / exit 2). A reference / overview doc that declared <!-- beadloom:watches=... --> has had a watched surface (cli / graph / flow.yml) change since its baseline. Stored in the separate reference_state table; cleared with beadloom sync-update <doc> --yes. |
Re-attesting clears hash_changed, hash_changed_since_head and symbols_changed, and nothing else: untracked_files needs a pair that does not exist yet, and missing_modules reads what the document says. That was measured one reason at a time through the real pipeline (BDL-069), and REASONS_ATTESTATION_CLEARS holds the result as an allow-list, so a reason added later is not told to re-attest until somebody measures that it can be. Every surface that prints an instruction for a stale pair chooses it from that flag — see Sync Check.
Modules
- engine.py -- Core sync engine: sync state building, multi-phase sync checking, hash computation, coverage analysis, and reference-doc surface-drift state (build/check/clear)
- git_baseline.py -- The baseline that cannot be lost: what git says about this working copy. Which paths differ from
HEAD, which paths this commit stages, which paths a branch changes against a ref (ref...HEAD, three dots) and which branch is checked out. BDL-068 S1.6 widened it from the first two to the last two rather than opening a second subprocess call togitelsewhere in the product: two readers of one tool are two things that can disagree about what a path is relative to. AnswersNonefor git could not tell, never for nothing changed - declared_docs.py -- The declared documentation surface checked against disk: which declarations the tree no longer satisfies
- commit_scope.py -- Which sync pairs a commit is about, and how many it therefore left unjudged (BDL-061 S6, BDL-UX #118). The commit gate judges the commit; the push gate judges the tree.
scope_to_commit(pairs, staged, docs_dir)keeps the pairs either side of which this commit stages -- either side, because the commit that FIXES a stale pair stages the doc -- and carries the number it left out so the caller can print it. AstagedofNonemeans git did not answer: nothing is narrowed and the reason says so, since narrowing on an absent answer would be inventing the scope - surface_ledger.py -- The committed record (
.beadloom/sync-surface.json) of how much there was to check last time, so a shrinking surface is reported rather than silently smaller - surface.py -- Layer 2 reference surface-drift: parses the in-doc
<!-- beadloom:watches=cli,graph,flow.yml -->annotation and computes coarse, deterministic per-surface signatures (clicommand+flag tree,graphnode+edge identity set, normalizedflow.yml) plus the order-sensitive aggregate hash - doc_indexer.py -- Markdown scanning, chunking by H2 headings, section classification, and SQLite population
- doc_shape.py -- Whether a document still carries the sections its kind requires; peer-relative by majority, so a convention is reported once and an outlier per document. Re-exports
table_cellsfromtables.py, where the row grammar moved (SPEC) - axes_section.py -- The
## Axessection's grammar, read in both directions: the seed it names, the scope decision it records, and the beadrefs:generated from it. A slice appends its rows under its ownDerived byline, so a real section holds one table per slice and each is judged against its own header. Every column is read by name, so theOwns unreadcolumn (BDL-UX #284) is read intoAxis.unread_countwhere a table carries it, and a table written before it reads exactly as it did. Two checks --axes-without-a-seedandaxis-without-a-scope-decision(SPEC) - doc_quality.py -- The five writing-standard checks over planning documents: a measurable goal, a decision with a reason, a risk with a mitigation, no
Pendingquestion in anApproveddocument, no unfilled template placeholder (SPEC) - audit.py -- Documentation audit: fact registry, comparator, and audit facade for detecting stale numeric facts.
FactRegistry.collect_set()returns aFactSet-- the facts computed for the project AND, for each fact no value was declared for, the reason -- so a collector that cannot answer no longer drops the fact and shrinks the denominator in silence - work_item_type.py -- The route a work item took, checked against the axes it was decided from (BDL-068 S1.5). Two checks over the work-item FOLDER:
routed-without-axes, for an item on the route that passes no scope approval and carries no## Axessection, androute-not-supported-by-the-axes, for one whose kept axes name more graph nodes than that route holds. Measured at2a5c0d1,## Axeswas required by the template and reported by nothing:missing-sectionis peer-relative and the corpus carried it in 0 of 12 BRIEFs, so the absence produced one kind-level statement and no document-level finding. This check is absolute, and the simplified route'sAxesrequirement is withdrawn from the peer-relative half alone, so a present-and-empty section is stillempty-section's finding (SPEC) - scope_check.py -- the paths a commit stages, judged against the axes its work item declared (BDL-068 S1.6). One check,
outside-the-declared-axes, over the WORK ITEM's axes and never the claimed bead's: the work item's axes are what a human approved and a bead may narrow freely inside them, so a commit that LEAVES them means the approval no longer covers the change. The rule was measured before it was chosen, because an always-red check is an ignored check: judging a path's owning NODE against the nodes the kept rows name is red on all three of BDL-068's own code commits, and judging at the bounded context those axes reach is silent on all three and outside for 115 of the 155 commits before the branch that touch an owned path. A node an axis rules OUT of scope is reported by that axis's name; a path no node owns is counted and stated, never reported (SPEC) - audit_self_surface.py -- Whether Beadloom's own surfaces describe the project under audit.
mcp_tool_countandcli_command_countare read out of the RUNNING package, so they are declared only when the audited project IS that distribution, decided from the name the project declares for itself with no directory-name fallback. Both used to be collected unconditionally, so every adopter was told Beadloom's numbers about their own documentation (measured oninvoice-svc: it was told the MCP-tool and CLI-command counts of the Beadloom release under test as though they were its own — and those counts move with every release, which is exactly why an adopter must never be told them, and why they are not restated here). The release that changed it is named in the CHANGELOG, which is where a version belongs -- a domain README stating a version that does not exist yet is a claim nothing can hold it to - issue_numbers.py -- The issue log's numbers: the two populations a numbered log states, the next number ALLOCATED by an exclusive create of one claim file per number, and three legs over the population one filesystem cannot span --
duplicate-number,unwritten-claimandunclaimed-number. A number read off the end of a shared markdown file collided five times on this repository (BDL-UX #187, #211, #253 and two within one hour on 2026-09-09), and the coordinator's manual duplicate check ran on the morning of the fifth without preventing it, which is why the number is allocated rather than checked for afterwards. The report states the population each leg REACHED and not only what it found:entries_below_flooris the entriesunclaimed-numberskips because they predate the ledger's floor -- 235 of 241 on this repository -- and the unaccounted numbers are named rather than counted, because a count is not something a reader can go and look for (BDL-UX #267). The three surfaces of one declaration tell the same three states apart sincebeadloom-rqma.9:declared_lograises with the refusal's ownwhyandremediation, sobeadloom issue-number allocateandbeadloom issue-number checkname the key a project misspelled instead of telling it that it declared nothing (BDL-UX #270, closed on the Gate leg one bead earlier and on the commands bybeadloom-rqma.8), and a config none of them could read is reported as an unknown rather than as an opt-out.refusal_sentenceis public for that reason -- two copies of the rule for rendering one refusal are two things that can disagree, and two surfaces disagreeing about one declaration is what #270 was. The resolver that returned the usable log and dropped the reason is gone rather than left for the next caller (SPEC) - scanner.py -- Document scanner: which markdown files are in scope (and which are skipped, with the reason), and keyword-proximity extraction of numeric fact mentions from them
- version_subjects.py -- Which named products a version token in this project's prose may belong to, derived from what the project already declares (every distribution in
pyproject.toml/package.json/Cargo.toml, the interpreter families implied byrequires-python/engines.node/rust-version,gitwhen the project is a git repository) and configured per NAME indocs_audit.subjectsfor what no manifest carries. It replaced a suppression per document: tendocs_audit.ignoretriples stood on this repository for one sentence shape -- "measured on bd 1.0.4" -- and eight went inert when a version stopped being read as a claim about this project whatever it was actually about (BDL-UX #253). A subject the environment confirms rather than the project declares is UNRESOLVED where its marker is absent, never denied: agit archive HEADroom carries no.git, so reading that absence as "this project has nothing to do with git" comparedgit 2.49.0against this project's version and made every clean-room Gate run on this repository rc 1 for one line of one document (BDL-UX #266). An unresolved name still wins the attribution walk, andaudit.pyreports the token underunjudgedwith the reason rather than judging it - audit_coverage.py -- Per-fact coverage of an audit run: whether anything was checked for each declared fact (
verified/not_covered/unreadable), so a count of findings can no longer read as a verdict on facts nobody stated - docsync.py (in
services/commands/) -- CLI commands:beadloom sync-check,beadloom sync-update,beadloom install-hooks, andbeadloom active-sync(the ACTIVE-table reconcile command; annotated ascomponent=active-tablebut housed in this module after the BDL-059 split ofservices/cli.pyintoservices/commands/) - declarations.py -- What a project declared under one
.beadloom/config.ymlkey, and what about that declaration was unusable. One module because two opt-in Gate legs were answering the question separately and both reached the same wrong answer:issue-logandreadme-pairrefused anything they could not use tologging, which the Gate does not render, so a project that had written the block and mistyped one key was told it had written no block at all and the leg it switched on never ran (BDL-UX #270; measured at HEAD on a foreign two-package project, four ways of misdeclaringdocument_pairs:produced a verdict byte-identical to the opt-out's andbeadloom ciexited 0 on all four). A declaration is in one of four states -- absent, empty, present, unreadable -- and a refusal names the ENTRY (document_pairs[1]) with the keys it does carry, which is what makes a one-letter typo visible. Nothing here decides a verdict: the refusals travel to the caller's report and the caller's Gate step renders them (DOC) - document_pairs.py -- A declared pair of documents compared by SHAPE and never by text: the sequence of blocks each file is built from (heading, paragraph, code, list, table), the heading levels, and the row counts of the lists and the tables. This repository ships two READMEs and nothing held them against each other; on 2026-09-10 the English one was missing a paragraph the Russian one had, and the only number that differed between the files was a line count -- 362 against 360 -- which nothing reads and which a translator wrapping differently moves by the same amount. The files are in two languages, so a TEXT comparison would be a check somebody has to switch off, which is the defect class BDL-069 is about. The pair is declared in
.beadloom/config.ymlunderdocument_pairs:, modelled onissue_log:, and a project that declares none is not judged. The table reading istables.py's, so this is a caller of the one table reader rather than a fifth reader of markdown (SPEC)
Features
- Sync Check -- The doc-code synchronization engine (
beadloom sync-check/sync-update). - Docs Audit -- Zero-config meta-doc staleness detection via keyword-proximity matching. CLI:
beadloom docs audit. - Document Pairs -- A declared pair of documents compared by shape: the block sequence, the heading levels and the row counts of the lists and the tables. Declared under
document_pairs:; a project that declares none is not judged. - Version Surface -- Every place a project states its own version, attributed to the instrument whose population holds it. The instruments are named and every place is DERIVED: the list of nine this project cut a release against was written by hand and was wrong by two (BDL-UX #281).
Components
- Doc Indexer -- Markdown scan + chunk +
docs/chunkspopulation; the doc half of every sync-check pair. - Markdown Tables -- What a table row is, and where one table ends and the next begins. Lifted out of
doc_quality.pywhenaxes_section.pyneeded the same answer: two readers of a table boundary is how one section holding two tables was read as one, twice in one slice (BDL-UX #213, #244). - Config Declarations -- What a project declared under one
.beadloom/config.ymlkey, and what about it was unusable. Shared byissue_log:anddocument_pairs:so that one rule about what a misdeclaration costs lives in one place: declaring none and declaring badly are two verdicts, and what tells them apart is a count of unusable entries rather than an adverb.
Git Hook Integration
Beadloom installs two git hooks by default: a pre-commit hook (lighter check) and a pre-push hook (the authoritative blocking Beadloom Gate). Use --pre-commit or --pre-push to select one.
# Install both hooks in warning mode (default)
beadloom install-hooks --mode warn
# Install both hooks in blocking mode
beadloom install-hooks --mode block
# Install only the pre-commit hook
beadloom install-hooks --pre-commit
# Install only the pre-push Gate hook
beadloom install-hooks --pre-push
# Remove the installed hook(s)
beadloom install-hooks --removePre-commit hook judges THE COMMIT rather than the working tree (BDL-UX #118), and runs:
- Ruff lint check over the Python files the commit stages, wherever they live
- Mypy type check over the subset of those inside the surface
pyprojectdeclares typed (beadloom typed-surface --filter), whose verdict the hook prints whatever it says beadloom sync-check --staged --porcelain(stale doc detection, narrowed to the pairs this commit stages)beadloom active-sync --stage(ACTIVE/table coherence; guarded no-op whenbdis unavailable)
It also states four things about its own scope: how much of the tree it did NOT judge (read from git status --porcelain, so an untracked neighbour module is counted too), how much of the COMMIT it compared against the work item's declared axes and how much of it no node owns (beadloom scope-check --porcelain, whose verdict the hook prints whatever it says — including NOT CHECKED and the reason, which used to reach only stderr and be discarded), which paths active-sync --stage corrected and did NOT add to the commit, and a # beadloom-hook-scope: commit marker that beadloom waves reads to tell whether an installed hook is the commit-scoped one. An installed hook keeps its old behaviour until install-hooks is re-run, and the old one is the silent one.
What it cannot do: a neighbour's hunk swept into a commit, inside a file the committer legitimately touches, is not caught and cannot be caught here — the swept hunk is inside the commit, which is the region the gate judges.
In warn mode, violations print warnings but do not block the commit. In block mode, ruff/mypy/sync-check violations exit non-zero and prevent the commit — with one stated exception: a type check whose surface could not be derived reads NOT CHECKED and never blocks in either mode, because a check that did not happen must not turn a missing PATH entry into a refused commit.
Pre-push hook runs the full Beadloom Gate (beadloom ci): incremental reindex → lint → sync-check → docs audit → docs-quality → doc-spaces → config-check → doctor, plus the landscape gate when the project declares satellites. This is the authoritative blocking gate; it exits non-zero on any failure, preventing the push. Fail-safe: if beadloom is not on PATH, the hook is a no-op.
Invariants
- A doc-code pair is determined by a shared ref_id
- doc_path is taken from the docs table (linked to a node via ref_id)
- code_path is taken from code_symbols (via annotations pointing to a ref_id)
- When staleness is detected, the status is updated in the sync_state table
- A pair reads
okonly when a comparison actually happened; unverifiable and unchanged never print the same word - A baseline's provenance is carried verbatim across a reindex and never promoted -- a fabricated baseline does not become earned by being copied
_compute_symbols_hashreturns an empty string when no symbols are annotated with the given ref_id, allowing callers to skip drift checks for unlinked nodes- Source coverage excludes boilerplate files:
__init__.py,conftest.py,__main__.py - Doc coverage uses word-boundary matching (
\b<stem>\b, case-insensitive) for module name detection
API
Module src/beadloom/doc_sync/engine.py
build_sync_state(conn: sqlite3.Connection) -> list[SyncPair]-- Build sync pairs from docs and code_symbols sharing a ref_id.check_sync(conn: sqlite3.Connection, project_root: Path | None = None, *, section_requirements: Mapping[str, tuple[str, ...]] | None = None) -> list[dict[str, Any]]-- Multi-phase sync check. Returns list of dicts with fields:doc_path,code_path,ref_id,status,reason,baseline, and optionaldetails. Runs hash comparison, symbol drift detection, git corroboration of fabricated baselines, source coverage, doc coverage, the declared-surface check, and — only whensection_requirementsis supplied — the document-shape phase.section_requirements=Nonemeans structure was NOT checked; the requirements come from theonboardingPEER domain and so are injected by the application layer rather than read here.find_missing_declared_docs(conn, project_root) -> list[dict[str, str]]-- (re-exported fromdeclared_docs.py) Docs the graph declares that are not on disk. Each entry carriesref_id,doc_path(project-relative) andindex_path.mark_synced(conn: sqlite3.Connection, doc_path: str, code_path: str, project_root: Path) -> None-- Recompute hashes for a doc-code pair and mark as synced. Updatessymbols_hashbaseline.pairs_of_ref(conn: sqlite3.Connection, ref_id: str) -> list[tuple[str, str]]-- Every(doc_path, code_path)pair the ref owns, in stable order. The addressable unit of an attestation.attest_ref(conn: sqlite3.Connection, ref_id: str, project_root: Path, *, scope: Collection[tuple[str, str]] | None = None) -> Attestation-- (BDL-061.85) Re-baseline a ref, claiming only the pairs in scope. A fact and a claim used to be oneUPDATE. The node-levelsymbols_hashis a fact about the index and is carried forward for every pair, so thesibling_symbols_changedverdict clears once the file that caused it is re-baselined;doc_hash_at_sync,code_hash_at_sync,file_symbols_hash,synced_atandbaseline_source = attestedare a claim about a document somebody read and are written only inside the scope.scope=Noneis the deliberate whole-ref attestation. ReturnsAttestation(attested, carried), the two populations named separately so a summary cannot report the first and hide the size of the second (BDL-UX #163).mark_synced_by_ref(conn: sqlite3.Connection, ref_id: str, project_root: Path) -> int-- The whole-ref attestation (attest_refwith no scope). Returns the number of rows updated. Kept as its own name because that is what every caller outsidesync-updatemeans.check_sync_since(conn: sqlite3.Connection, *, project_root: Path, since: str) -> list[dict[str, Any]]-- Report doc-code pairs that drifted relative to a git ref baseline (instead of the storedsync_state). A pair is stale-since-ref iff its code file changed betweensinceand the working tree and its linked doc was not correspondingly updated sincesince(if the doc also changed, the dev already touched it →ok). Reads git (git show <ref>:<path>) + disk only; mutates neithersync_statenor the working tree; result shape mirrorscheck_syncso the JSON/porcelain renderers are shared. This is what makes drift detection survive a fresh CI checkout (a clean clone re-baselinessync_stateto the just-pushed code, masking per-push drift)._validate_git_ref(project_root: Path, ref: str) -> bool--git rev-parse --verify <ref>; an all-zero SHA (force-push / first-push sentinel) never resolves so it is rejected. Mirrorsgraph.diff._validate_git_ref.check_source_coverage(conn: sqlite3.Connection, project_root: Path) -> list[dict[str, Any]]-- Check if all source files in a node's directory are tracked. Returns list of dicts withref_id,doc_path,untracked_files.attestation_clears(reason: str) -> bool-- (BDL-069) Whether re-attesting a stale pair can clear reason; readsREASONS_ATTESTATION_CLEARS, and isFalsefor a reason nothing has measured.content_remedy(row: Mapping[str, Any]) -> str-- (BDL-069) What clears a stale row whose reason re-attesting cannot clear, in the one wording the gate, the--reportfooter andsync-updateall print.check_doc_coverage(conn: sqlite3.Connection, project_root: Path) -> list[dict[str, Any]]-- Check if documentation mentions module names from the source directory. Returns list of dicts withref_id,doc_path,missing_modules.build_reference_state(conn: sqlite3.Connection, project_root: Path) -> int-- (BDL-057 Layer 2) Discoverwatches-annotated reference docs and baseline each one's aggregate surface hash into thereference_statetable. The baseline is preserved across reindex for docs already tracked with the samewatchesset (so accrued surface drift survives a routine reindex, mirroring the symbol-pair fixpoint concern); a fresh baseline is taken only for newly-discovered docs or when the declaredwatchesset changes. Docs whose annotation was removed are dropped. Idempotent. Returns the number of reference docs recorded.check_reference_drift(conn: sqlite3.Connection, project_root: Path) -> list[dict[str, Any]]-- (BDL-057 Layer 2) Recompute each reference doc's aggregate hash and report drift. Returns one dict per reference doc withdoc_path,watches,status(ok/surface_drift),reason, andseverity(alwayswarning). Persists the new status; never affects thesync-checkexit code.mark_reference_synced(conn: sqlite3.Connection, doc_path: str | None, project_root: Path, *, all_docs: bool = False) -> int-- (BDL-057 Layer 2) Re-baseline a reference doc's aggregate hash (or every reference doc whenall_docs), clearing surface drift. Returns the number of rows re-baselined. (Backsbeadloom sync-update <doc> --yes/--all.)describe_reference_doc(conn: sqlite3.Connection, doc_path: str | None, project_root: Path) -> dict | None-- the read-only counterpart: a reference doc'swatchesset and its current drift status (ok/surface_drift), with the baseline and current aggregate hashes.Nonewhen the path is not a tracked reference doc. Executes noUPDATEand does not commit. (Backsbeadloom sync-update <doc> --check, BDL-UX #189.)
Module src/beadloom/doc_sync/tables.py
cells_of(line: str) -> list[str] | None-- The cells of a table line, alignment row included, orNonewhen the line is no table row. The raw reading.is_separator(cells: Iterable[str]) -> bool-- Whether those cells are an alignment row rather than content.table_cells(line: str) -> list[str] | None-- The cells of a table row that says something;Nonefor a separator. Re-exported fromdoc_shape.py, where it used to live, so no caller moved.table_blocks(lines: Iterable[tuple[int, str]]) -> list[Table]-- The contiguous tables in numbered lines, each leading with its own header row. A separator row is dropped and does NOT end a table; everything that is not a table row does. Reading a section as one table is BDL-UX #213 indoc_quality.pyand BDL-UX #244 inaxes_section.py-- one sentence found twice, hours apart, in one slice.Table-- one table, as(line number, cells)rows.
Module src/beadloom/doc_sync/document_pairs.py
read_blocks(text: str) -> tuple[Block, ...]-- one document as its sequence of blocks. A blank line ends a block and so does a line of another kind; a fenced block is opaque, because a#inside a shell transcript is a comment and not a heading.compare_documents(source: Sequence[Block], follower: Sequence[Block]) -> Comparison-- the two sequences aligned bydifflib.SequenceMatcherover block signatures, with the number of pairs it aligned. It names WHERE the sequences diverge and the heading that position stands under; it cannot name which of several identical-shaped paragraphs went missing, and that limit is what lets it compare across two languages.read_pair_declaration(project_root: Path) -> PairDeclaration-- everydocument_pairs:entry the project wrote: the usable pairs, the count of entries LOOKED AT, and a refusal per entry that could not be used. A half-written entry or a path resolving outside the project is refused rather than guessed, and the refusal travels with the pairs. The resolver that returned only the usable pairs was deleted in BDL-069 (beadloom-rqma.9): a caller that cannot see the refusal tells a project which mistyped a key that it declared nothing, which is BDL-UX #270.check_document_pairs(project_root: Path) -> PairReport-- every declared pair compared, with the blocks each file held, the pairs aligned, and every path it could not read.UNPAIRED_BLOCK,BLOCK_KIND,ROW_COUNT(CHECK_NAMES) -- the three checks;HEADING,PARAGRAPH,CODE,LIST,TABLE(BLOCK_KINDS) -- the five block kinds.
Module src/beadloom/doc_sync/version_surface.py
read_version_surface(project_root: Path) -> VersionSurface-- every place project_root states its own version, each attributed to the instrument whose population holds it, with the places no instrument holds named as such. A project declaring no version this reader can follow gets an emptyplacesand a statedunresolvedreason.VersionSurface--source_of_truth,unresolved,places,instruments,population, and theuncheckedprojection.Instrumentcarries the population it was DERIVED to hold, so a report names what each check covers rather than that it ran.SKIPPED_DIRECTORIES,READ_SUFFIXES-- the sweep's declared bounds, reported on every run beside the suffixes it did not read and every file it could not decode.- The sweep is by the CURRENT literal, so a place that already states an old version is invisible to it: run it BEFORE the bump. Reading every version token this project's prose attributes to itself instead returns 674 claims across 137 files, because the planning archive holds every version it ever had.
Module src/beadloom/doc_sync/doc_shape.py
read_sections(text: str) -> tuple[Section, ...]-- Every heading with its line number, its depth and its body, FENCES SKIPPED: a##inside a fenced block is a quoted template, not a section of this document.Section.is_emptyis true only when neither the heading nor anything nested under it says anything -- judging a heading's own lines alone reported 155 documents on this repository whose## Code Standardsis a heading over four###.document_sections(text: str) -> tuple[str, ...]-- Every heading in a markdown document, at any depth.carries_section(sections: Iterable[str], required: str) -> bool-- Whether a required section is stated. Case-insensitive, whole-word and depth-independent, so## Features and componentscarriesFeaturesand## Featuresetdoes not.peer_section_shape(documents, requirements) -> (conventions, lost)-- the peer-relative policy over ANY corpus ofkind -> documents, so generated node documentation and the flow's planning documents are judged by ONE implementation of "does a majority carry this".check_section_shape(conn, project_root, requirements) -> list[dict[str, Any]]-- Phase 5 ofcheck_sync, expressed over that policy. Peer-relative: a section is required of a node kind only when a MAJORITY of that kind's documents carry it; a minority reports the KIND once with its ratio (Source (5/39)) instead of reporting every document that follows the project's actual convention.check_planning_sections(documents, requirements) -> PlanningShapeReport-- the same policy over PLANNING documents (BDL-068 S1.4), returningmissing-section(peer-relative) andempty-section(not: a heading with nothing under it is a defect whatever the peers do) asQualityFindings, plus the conventions and the kinds no template describes.STATUS_INCOMPLETE(incomplete),REASON_MISSING_SECTIONS,REASON_SECTION_NOT_IN_USE-- deliberately absent fromBLOCKING_STATUSES, and never written tosync_state: the column would then mean two things, and a check that writes to what it inspects cannot be trusted about it.
Module src/beadloom/doc_sync/doc_quality.py
check_document(text: str, *, path: str, placeholders=(), decision_sections=()) -> QualityReport-- The five checks over one document, returning the findings and how much each check had to READ.decision_sectionsnames the sections the shipped templates put a decision table under; it is passed in for the reasonplaceholdersis, because the templates are composed one layer up.check_documents(paths, *, project_root, placeholders=(), decision_sections=()) -> QualityReport-- The same over a set, aggregated per check and per document KIND. A document that does not decode is counted, carries a namedunreadablefinding and is left out of its kind's denominators.document_kind(path: str) -> str-- The file's stem:PRD.mdis aPRD. A project whose documents areprd.mdgetsprdas a kind of its own, which is honest — nothing was told the two are the same.document_status(text) -> str/is_approved(text) -> bool-- The> **Status:**line, and whether it is one ofAPPROVED_STATUSES(approved,accepted). A Draft is allowed its open questions.declares_decisions(section_title, header, *, declared_sections=()) -> bool-- Whether the DOCUMENT declares a table as decisions: a column naming the thing decided, or a section the shipped templates put a reason-carrying table under. Nothing here reads the cells, because nothing could -- a measurement row and a decision row are the same two strings to a checker (BDL-UX #213).sections_with_a_decision_table(text: str) -> tuple[str, ...]-- The section titles under which text states a table carrying a reason column. Run over the shipped templates byshipped_decision_sections, so the declared forms are derived rather than listed.REASON_COLUMNS-- the header cells a table states its reasons under, read once by the check and by the derivation so the two cannot classify different tables.UnclassifiedTable--path,line,section,header,rows: a table with a reason column the document never declared as decisions. A verdict, not a finding; its rows are absent fromapplicableas well as fromfindings.QualityReport--findings,documents,applicable(per check),by_kind(perKindCoverage),unclassified,unreadable, and the propertieschecks_that_read_nothingandkinds_that_read_nothing. The second exists because the first is a global OR over the corpus and goes silent as soon as one document carries one row, so it cannot see a check that is blind on a whole document kind.names_a_witness(statement: str) -> bool-- Whether a goal names something an observer could go and look at: a quantity, a named artifact (an inline code span, a--flag, a file name, asnake_caseidentifier) or an observable outcome (exits,fails,passes,detects,green).states_an_unbounded_improvement(statement: str) -> bool-- Whether a goal's predicate is a quality rather than an outcome (improve,establish,clean up,make/keepsomething better).measurable-goalreports a goal only when this holds ANDnames_a_witnessdoes not; it shipped as a numeral detector and reported 154 of 235 goal statements here, against 4 of 232 now (beadloom-mr2l.70).CHECK_NAMES(the five),CONTENT_CHECKS(the four that read ITEMS;unfilled-placeholdercounts documents OPENED and would report every kind as read).
Module src/beadloom/doc_sync/git_baseline.py
changed_paths(project_root: Path) -> frozenset[str] | None-- Project-relative paths that differ fromHEAD: modified, staged, deleted, renamed (both endpoints) and untracked. Parsesgit status --porcelain -z -uall(NUL format, so a path with spaces or non-ASCII bytes cannot round-trip wrong) and re-roots repository-relative paths onto the project viagit rev-parse --show-prefix. ReturnsNone-- git could not answer, never nothing changed -- outside a work tree, without git, or before the first commit.
Module src/beadloom/doc_sync/declared_docs.py
count_declared_docs(conn: sqlite3.Connection) -> int-- How many docs the graph declares, existing or not.find_missing_declared_docs(conn: sqlite3.Connection, project_root: Path) -> list[dict[str, str]]-- Declarations the tree no longer satisfies, in declaration order.
Module src/beadloom/doc_sync/surface_ledger.py
SurfaceLedger(declared_pairs, declared_docs, recorded_at)/SurfaceVerdict(recorded, shrank, message, headline)--headlineis the same fact in a few words for a one-line step summary,messagethe actionable form for a finding; both, so a summary never has to elide the numbers.ledger_path(project_root) -> Path--.beadloom/sync-surface.json(committed).read_ledger(project_root) -> SurfaceLedger | None--Nonewhen absent or malformed; the caller reports "not recorded" rather than crashing a gate over two integers.write_ledger(project_root, *, declared_pairs, declared_docs, recorded_at="") -> Path-- Record the surface.recorded_atis injected by the caller, nevernow()here, so the file is byte-stable for a fixed input.compare_surface(ledger, *, declared_pairs, declared_docs) -> SurfaceVerdict-- Three outcomes and only one is silence: not recorded, shrank (both numbers named), or grew/unchanged.
Module src/beadloom/doc_sync/surface.py
VALID_SURFACES: tuple[str, ...]-- The known coarse surfaces:("cli", "graph", "flow.yml").parse_watches(text: str) -> list[str] | None-- Parse the<!-- beadloom:watches=cli,graph,flow.yml -->annotation; returns the ordered, de-duplicated list of known surfaces in declared order, orNonewhen absent / no known surface (unknown tokens are silently dropped).cli_signature() -> str-- SHA-256 of the sorted Click command + option-flag tree (command paths + flag names), so adding/removing a command or flag moves it but help-text edits do not.graph_signature(conn: sqlite3.Connection) -> str-- SHA-256 of the sorted noderef_id|kindset plus the sorted edgesrc|dst|kind|contract_keyset (a coarse identity set, not node content).flow_signature(project_root: Path) -> str-- SHA-256 of.beadloom/flow.ymlre-serialized canonically (sorted keys), or""when absent/invalid; comments and key order do not move it.surface_signature(surface: str, conn: sqlite3.Connection, project_root: Path) -> str-- Dispatch to the per-surface signature; raisesValueErrorfor an unknown surface.aggregate_hash(watches: list[str], conn: sqlite3.Connection, project_root: Path) -> str-- SHA-256 of the watched surfaces' signatures concatenated in declared order (order-sensitive by design).
Module src/beadloom/doc_sync/doc_indexer.py
classify_section(heading: str) -> str-- Classify a section heading into:spec,invariants,api,tests,constraints, orother.chunk_markdown(text: str) -> list[dict[str, Any]]-- Split Markdown text into chunks by H2 headings. Each chunk containsheading,section,content,chunk_index. Chunks exceedingMAX_CHUNK_SIZE(2000 chars) are split by paragraphs.index_docs(docs_dir: Path, conn: sqlite3.Connection, *, ref_id_map: dict[str, str] | None = None) -> DocIndexResult-- Scan a directory for.mdfiles, chunk them, and insert into SQLite.
Module src/beadloom/services/commands/docsync.py
CLI commands for doc-sync operations. This module was split from services/cli.py in BDL-059.
sync_check(*, porcelain: bool, output_json: bool, output_report: bool, ref_filter: str | None, since_ref: str | None, record_surface: bool, project: Path | None) -> None-- Check doc-code synchronization status. Exit codes: 0 = all ok, 1 = error, 2 = a pair is stale or missing.--record-surfacewrites the committed declared-surface ledger from the live run.install_hooks(*, mode: str, remove: bool, pre_commit: bool, pre_push: bool, project: Path | None) -> None-- Install or remove beadloom git hooks. By default installs BOTH pre-commit and pre-push hooks. Use--pre-commit/--pre-pushto select one.active_sync(*, epic: str | None, check_only: bool, output_json: bool, no_export: bool, stage: bool, project: Path | None) -> None-- Reconcile ACTIVE.md bead-status tables frombd(the source of truth). No-op whenbdis unavailable or no ACTIVE table exists.sync_update(ref_id: str | None, *, check_only: bool, assume_yes: bool, all_refs: bool, project: Path | None) -> None-- Show sync status and update docs for a ref_id. Supports interactive editor mode, non-interactive re-baseline (--yes), and batch re-baseline (--all).
CLI (beadloom sync-check)
beadloom sync-check [--porcelain] [--json] [--report] [--ref REF_ID] [--since GIT_REF]
[--record-surface] [--project DIR]| Flag | Description |
|---|---|
--porcelain | TAB-separated machine-readable output |
--json | Structured JSON output: summary (incl. missing, unverified, declared_docs and the advisory surface_drift count), the symbol-pair pairs array (each with its baseline), an additive references array for reference-doc surface drift, and declared_surface (the ledger verdict). The --since shape is a fixed contract and is left untouched. incomplete has no summary counter: the rows are in pairs and are printed by name, but ok + stale + missing + unverified + unchecked does not sum to total when any row is incomplete, so a consumer reading only the summary does not see them (review beadloom-mr2l.15 M5, open). |
--report | Markdown report for CI posting |
--ref | Filter results by ref_id |
--since | Baseline = code state at this git ref (e.g. the push's parent) instead of the stored sync_state; reports pairs whose code drifted since the ref while the doc was not correspondingly updated. Delegates to check_sync_since; invalid/zero refs are rejected via _validate_git_ref. |
--record-surface | Record the declared surface (pair + declared-doc counts) to the committed .beadloom/sync-surface.json, so a later run can tell when it SHRANK. A deliberate act: no ordinary run rewrites it. |
--project | Project root (default: current directory) |
Exit codes: 0 = all ok, 1 = error, 2 = a pair is stale or missing. unverified pairs and a shrunken surface are reported by name and do not change the exit code -- they are warnings, not verdicts about the code.
CLI (beadloom sync-update)
beadloom sync-update [REF_ID] [--check] [--yes|-y] [--all]
[--pair DOC_PATH]... [--code CODE_PATH]... [--all-pairs]
[--project DIR]Show sync status and update docs for a ref_id. In interactive mode (default), displays stale pairs and opens each stale doc in $EDITOR for manual correction, then marks them synced via mark_synced -- already per pair. In non-interactive mode (--yes), delegates to _mark_synced_noninteractive, which resolves a SCOPE and calls attest_ref.
The scope is the fix for BDL-UX #163 (bead .85). sync-update <ref> --yes used to re-baseline every pair the ref owned, so a run that revised one document recorded a claim about its siblings; measured over one epic, 1 pair revised against 27 re-attested, then 13/20, 26/56 and 15/10. The default scope is now the pairs the check reports as stale -- the ones it demands action on. A pair reported unverified because a sibling moved is exactly the pair bead .78 said nobody can revise, and it is left unclaimed with its baseline standing, which is not cosmetic: check_sync corroborates an index_build baseline against git instead of trusting it, so a bulk re-attestation used to switch the harder check off for documents nobody had opened.
--yes names what it did not clear (BDL-069, the RFC's Q1). After attesting, _report_left_stale re-runs check_sync over the ref, or over every ref under --all, and prints each pair still stale with why: a reason re-attesting cannot clear prints content_remedy, a pair outside the run's scope prints not claimed by this run, and a pair attested and still stale says so. A run that cleared everything prints Re-checked after attesting: no pair … is still stale. The evidence is first-hand: during BDL-069's scoping sync-update --yes --all exited 0 with Marked 2 ref(s) synced (4 pair(s) total) over a missing_modules staleness, and the operator believed the defect fixed. Output only: the exit code and the scope are unchanged.
| Argument/Flag | Description |
|---|---|
REF_ID | Optional positional argument; the symbol-pair ref to update, or a reference doc's path (e.g. docs/architecture.md) to clear its surface drift. Required unless --all is used. |
--check | Report only — never write. Holds for a reference doc's path as well as for a symbol-pair ref: the branch that handles a doc path used to be reached before this guard, so --check re-baselined the doc it was asked to describe and printed Re-baselined reference doc <path> (BDL-UX #189, the shape of #147's mutating lint). It now prints the doc's watches set and its drift status through the read-only describe_reference_doc. |
--yes / -y | Non-interactive: re-baseline freshness without an editor or prompt. |
--all | With --yes: re-baseline every currently-stale ref (for the fixpoint loop). Requires --yes; mutually exclusive with an explicit REF_ID. Each ref is scoped to its own stale pairs. |
--pair DOC_PATH | Repeatable. Attest the pairs of this document, whatever their status -- the operator read it, and that is the grounds. An unknown path is refused with the ref's real document paths, because a typo that selected nothing would print Nothing to attest, which reads exactly like a ref that needed no work. |
--code CODE_PATH | Repeatable. Narrow --pair to one code file; given alone, claims that file under every document of the ref. Named as a separate axis because a document paired with several code files is a document several agents can be changing at once: measured on the BDL-061 drift branch, domains/doc-sync/README.md was stale against one agent's engine.py and another's surface.py in the same run. |
--all-pairs | The deliberate whole-ref attestation: claim every pair the ref owns, including documents this run has no grounds for. What an operator who really did read all of it types. Requires --yes; mutually exclusive with --pair/--code and with --all. |
--project | Project root (default: current directory). |
CLI (beadloom install-hooks)
beadloom install-hooks [--mode {warn,block}] [--remove] [--pre-commit] [--pre-push] [--project DIR]Installs or removes beadloom git hooks. By default installs both the pre-commit hook (lighter warn/block check) and the pre-push hook (the authoritative blocking Beadloom Gate). Use --pre-commit or --pre-push to select one. The --remove flag deletes the selected hook(s).
| Flag | Description |
|---|---|
--mode | Hook mode: warn (default) or block. In warn mode, violations print warnings but do not block. In block mode, violations exit non-zero and prevent the commit/push. |
--remove | Remove the selected hook(s). |
--pre-commit | Operate on the pre-commit hook only. |
--pre-push | Operate on the pre-push Gate hook only. |
--project | Project root (default: current directory). |
CLI (beadloom active-sync)
beadloom active-sync [--epic EPIC] [--check] [--json] [--no-export] [--stage] [--project DIR]Reconcile ACTIVE.md bead-status tables from bd (the source of truth). For each epic's ACTIVE.md, rewrites the bead-status table's Status cells to match bd (rich coordinator notes are preserved when the state agrees). Default = fix mode (writes + syncs the tracked .beads/issues.jsonl via bd export); --check reports drift without writing (exit 1 on drift).
No-op contract: if bd is unavailable OR there is no ACTIVE file with a bead-status table, this exits 0 and writes nothing (a non-flow repo is never affected).
Every run states what it did NOT reconcile, in three lists rather than one. Rows it could not map onto a bead are reported by CELL with the shape that made each unresolvable. Beads the tracker holds under a table's epic are reported by BEAD, split in two: those a row NAMES and this run could not read (the cell is quoted; fix it), and those no row names at all (add a row). The split is BDL-068 S5's review Major 1 — the two were one list, so 38 of the 79 beads reported as carried by no row had a row the same run had already printed as bead-and-text or more-than-one-bead, and a reader acting on the message would add a row that was already there. No list is ever written into a table.
--stage (fix mode) re-stages the corrected content of the paths this commit ALREADY stages and names every path it corrected and did not stage, under a fixed withheld: line. It never adds a path to a commit: the index at commit time is the set of paths the committer chose, and this command used to override that choice (BDL-UX #207). Under --stage, bd export runs only when the commit already carries .beads/issues.jsonl — a refresh that cannot be committed keeps no tracked artifact honest and dirties a shared working tree instead. The by-hand path (no --stage) exports as before.
| Flag | Description |
|---|---|
--epic | Reconcile only this epic's ACTIVE.md. |
--check | Report drift without writing; exit 1 if any drift, 0 if clean. |
--json | Machine-readable JSON output. |
--no-export | Skip the bd export jsonl sync (fix mode only). |
--stage | Re-stage the reconciled ACTIVE.md(s) + the exported jsonl that this commit ALREADY stages (fix mode only). Never adds a path to a commit; the ones it corrected and withheld are named. Best-effort (no git → skip). |
--project | Project root (default: current directory). |
Testing
Tests: tests/test_sync_engine.py, tests/test_cli_sync_check.py, tests/test_cli_sync_update.py, tests/test_source_coverage.py, tests/test_doc_coverage.py, tests/integration/doc_sync/test_surface.py, tests/test_reference_drift.py, tests/test_cli_reference_drift.py