Skip to content

Testing plan: a repository may bring its own graph hooks

Derived from the approved requirements.md and design.md, before tasks.md — each task's _Test:_ names a row of the matrix below. Authored at test-planning and completed at verification.

Test matrix

#TypeApplies?Scope / what it provesWhere it runs
T1Unityesextensions.read_declaration / load_modules / apply, the registry collector, Graph.hook_for, the outcome drop in run_chainmake test (cli/tests/test_graph_extensions.py)
T2Integration (scenario)yesa real repository tree declaring a hook module: loaded, appended, run, and reported by the-loop check — Gherkin-documentedmake test (cli/tests/test_graph_extensions_integration.py)
T3Contract (OpenAPI / GraphQL SDL)n/a — this work item adds no API operation; the control plane's contract is untouched.
T4End-to-endn/a — the shipped PDLC e2e runner (test_pdlc_e2e) walks the loop with no repository hooks declared, which is exactly the R1.6 "unchanged when absent" case T1 asserts directly.
T5UI / visualn/a — no product UI (design § UI/UX).
T6Snapshotn/a — no rendered artifact is snapshotted; the CLI action's output is asserted field-wise.
T7Performance / loadn/a — one import per module per process, on a path that already reads two YAML files.
T8Security / abuse caseyesone negative test per abuse case in design.md § Security designmake test
T9Accessibilityn/a — no UI.
T10Migration / upgradeyesa repository with no graph.hooks block, and one written against the pre-rename config.yaml, both keep workingmake test
T11Manual exploratoryyesthe-loop graph hooks and the-loop check run by hand against this repository with a sample hook module, to confirm the operator-facing text reads the way the docs claimterminal
T12Docs/schema parityyesgraph.hooks resolves in the harness schema, is documented as CLI-read, and the new CLI-config key has an option pagemake test (test_harness_config.py, test_docs_parity.py)

Scenarios & requirement trace

RowRequirement(s)Scenario / case
T1R1.1–R1.4a declaration naming a path module and an attachment produces a graph whose node chain ends in the x- hook
T1R1.6a repository with no block compiles to a graph identical to the shipped one, and imports nothing
T1R2.1apply appends; the shipped entries keep their order and their position
T1R2.3, R2.4run_chain drops an x- hook's data["outcome"] and logs that it did
T1R2.5an attachment naming an undeclared node, or a boundary other than entry/exit, fails to load
T1R3.1–R3.3the collector refuses a non-x- registration; @hook refuses an x- name outside it
T1R3.4two repositories declaring the same x- name each resolve their own function
T1R4.1, R4.2missing module, module that raises, module that registers nothing, attachment naming an unregistered hook
T1R5.3absolute path, .. escape, symlink out of the tree, non-.py suffix
T2R1.5Scenario: a repository's own hook gates a node for the CLI and the daemon alike
T2R1.1, R1.3Scenario: a repository declares a hook module and the loop runs it at the boundary it named
T2R4.1, R4.4Scenario: a hook module that cannot be imported stops the loop instead of quietly disappearing
T2R5.1Scenario: the operator inspects a repository's hook declarations without importing them
T2R5.2Scenario: the operator refuses repository hooks and nothing from the repository is imported
T8abuse 1test_a_repository_hook_cannot_rescue_a_blocked_chain
T8abuse 2test_a_repository_hook_cannot_declare_an_outcome
T8abuse 3test_a_module_outside_the_repository_is_refused
T8abuse 4test_a_module_registering_a_shipped_name_fails_to_load
T8abuse 5test_two_repositories_keep_their_own_implementations
T8abuse 6test_a_raising_repository_hook_blocks
T8abuse 7test_the_operator_kill_switch_imports_nothing
T10R1.6the shipped graphs still compile with repo= pointing at a repository that has no harness config at all
T12R1.1graph.hooks is in READS, resolves in the schema, and appears in the harness-config doc's read table

Verification environment

  • Repositories: this repository only.
  • Services / containers: none — every test is filesystem-local (tmp_path repositories).
  • Fixtures & data: hook modules written into tmp_path by the tests themselves; no checked-in fixture executes.
  • Credentials: none. No test reaches the network.
  • Bring-up: uv sync --project cli · Tear-down: none.
  • If bring-up fails: record it under Verification results and escalate.

Evidence plan

RowEvidencePath under evidence/
T1, T2, T8, T10, T12test run output (counts, duration)unit-and-integration.md
T1, T2lint + typecheck outputlint-and-typecheck.md
T8security review recordsecurity-review.md
T11terminal transcript of graph hooks and check against a sample modulemanual.md

Verification activities

  • [x] T1/T2/T8/T10/T12 — make test
  • [x] T1/T2 — make lint && make typecheck
  • [x] T8 — security review of the diff, recorded (human sign-off still pending)
  • [x] T11 — the-loop graph hooks and a real runtime evaluation against a sample module

Verification results

Executed 2026-08-18 on claude/github-issue-248-o4u7z7.

ActivityCommand / procedureOutcomeEvidence
T1/T2/T8/T10/T12make test2412 passed, 1 skipped (2405 before this change; 48 of the new ones are this work item's)unit-and-integration.md
T1/T2make lint · make typecheckruff clean, markdownlint 0 errors over 807 files, pyright 0 errorslint-and-typecheck.md
T12uv run --with jsonschema python scripts/validate_config.pyall seven configs valid against the changed schemaslint-and-typecheck.md
T8security review of this work item's diff, boundary by boundarytwo findings, both accepted and documented; no change requiredsecurity-review.md
T11a hand-built repository declaring one module and one attachment, driven with the-loop graph hooks, the-loop check and a real build_runtime evaluationthe hook blocked the node with its own message, passed once the file was fixed, and never ran behind a shipped gate that blocked firstmanual.md

Not executed: none. The one activity replanned is T4 (end-to-end): the shipped PDLC e2e runner walks a repository that declares no hooks, which is the R1.6 case test_a_repository_that_declares_nothing_is_untouched asserts directly — the matrix marked it n/a for that reason and the run confirmed it (the e2e suite is green, unchanged).

Review comments

Appended by the-loop's record-feedback hook when a human gate approves with comments.

Released under the MIT License.