Skip to content

<!-- Authored per the the-loop:writing skill. -->

Testing plan: an operator can bring their own graphs and bind commands to them ​

Derived from requirements.md and design.md, before tasks.md. Authored at test-planning, completed at verification.

What is testable here, and what is not ​

Everything this change adds is a parser over a mapping, a compiler over a YAML file, a name resolver, a comment parser and a dispatcher branch — and the suite already drives each shape: test_graph_extensions.py compiles graphs against a tmp_path repository, test_control.py parses comment bodies, and test_control_integration.py runs the real dispatcher with the session layer faked at its injection point. No network, no real provider, no tmux.

One row cannot run here: arming a real ticket on a real daemon (T11) needs a GitHub repository and a running instance. T5 drives the same dispatcher with the same comment.

Test matrix ​

#TypeApplies?Scope / what it provesWhere it runs
T1Unityesread_catalog / read_bindings: every R1 and R3.1–R3.4 rule (grammar, reserved prefix, shipped names, duplicates, unknown keys, command classes, a binding to an undeclared graph, keyword only on a new word), the loader's routing._graphs fan-in, path resolution against a base, an absent blockcd cli && uv run python -m pytest tests/test_graph_catalog.py
T2Unityescompile_custom / load_graph: same compiler errors, name mismatch, unknown phase, missing/invalid file, x- hook resolution (declared, undeclared, no modules), the unknown-name error, the cache, slash_command, the command: grammar on every shipped loopcd cli && uv run python -m pytest tests/test_graph_catalog.py
T3Unityesresolve_outer_loop(name, declared), build_runtime choosing / refusing / guest, _outer_loop_name state-first then record loop then command, core/graphs._recorded_loopcd cli && uv run python -m pytest tests/test_graph_catalog.py
T4UnityesControlConfig bindings and keywords, parse_command for a new word, an override, an undeclared word, ambiguity; ControlRecord loop round trip and an older record without itcd cli && uv run python -m pytest tests/test_control_custom_commands.py
T5Integration (scenario)yesThe dispatcher end to end: an authorized the-loop triage arms and records loop, the spawn enters the custom graph and work-item-state.json records its name; an overridden do does the same; an unauthorized new command is refusedcd cli && uv run python -m pytest tests/test_custom_graph_integration.py
T6Unityesattach[].loops: scoped attachment skipped on other loops, applied on its own, unknown loop refused, digest changes, unscoped behaviour unchangedcd cli && uv run python -m pytest tests/test_graph_extensions.py
T7Unityesthe-loop graph loops: text and JSON rows for shipped and declared loops, a broken graph reported with exit 1, no module importedcd cli && uv run python -m pytest tests/test_graph_catalog.py -k loops
T8Security / abuse caseyesThe six abuse cases of requirements.md, each a named negative test (design § Security design table)cd cli && uv run python -m pytest tests/test_graph_catalog.py tests/test_control_custom_commands.py tests/test_custom_graph_integration.py
T9Contract / schemayesBoth copies of cli-config.schema.json stay identical; the documented shape (top-level graphs, routing.control.commands) validates and the retired routing.graph.graphs does not; the shipped sample configs validatecd cli && uv run python -m pytest tests/test_config_schema_parity.py tests/test_configschema.py tests/test_docs_parity.py tests/test_graph_catalog.py -k schema
T10RegressionyesThe whole Python suite — the control parser, the dispatcher, the graph coupling and every shipped loop are touchedmake test
T11Manual exploratoryyesA real daemon: declare a graph and a new command, type it on a ticket, watch the session walk the custom graphan operator's instance; not available in this environment
T12Documentation parityyesCapability docs, the config reference and the skill describe the feature; markdown lint greenmake lint
T13Performancen/a — one extra YAML compile per declared graph per process, cached like the shipped loops; no hot path gains work
T14UI / accessibility / visualn/a — the-loop has no product UI; the new surface is a config key and a CLI verb

Scenarios & requirement trace ​

RowRequirement(s)Scenario / case
T1R1.1–R1.5a malformed or colliding declaration fails to load, naming the entry
T1R3.1–R3.3a command binds to one graph, and only an arming command can be overridden
T2R2.1–R2.6a custom graph is held to the shipped compiler, the phase vocabulary and the hook registry
T3R4.1–R4.4, R5only a declared name selects a custom graph, on every path; guest follows the declaration
T4R3.4–R3.6, R3.8a declared word parses as start with a loop; an undeclared one is no command
T5R3.5, R3.6, R4.1Scenario: a new command arms a work item onto the operator's graph
T5R3.6Scenario: an overridden arming command selects the operator's graph
T5abuse 3Scenario: an unauthorized user's new command is refused
T6R6.1, R6.2an attachment scoped to loops leaves every other loop alone
T7R7.1–R7.3graph loops lists and checks every loop without importing a module
T8abuse 1–6design § Security design table
T2R8a repository's graph file is ignored, and the warning names the top-level graphs

Verification environment ​

This container, against the repository checkout: uv sync (with uv ≥ 0.12), then cd cli && uv run python -m pytest -q — the command make test and the pytest pre-commit hook run. Linting is uv run ruff check cli hooks, uv run ruff format --check cli hooks, uv run pyright cli and markdownlint-cli2, matching make check. No network: sessions and GitHub writes are faked at their injection points.

Evidence to capture ​

Under docs/specs/issue-343/evidence/:

  • verification.md — each row of this matrix with its command, outcome and counts.
  • self-review.md — the review cycles and every finding's disposition.
  • security-review.md — each abuse case with the mechanism and the test establishing it; the tier-4 named sign-off requested.
  • documentation.md — the capability, config and skill docs updated in this PR.

Verification results ​

Recorded at the verification node in evidence/verification.md: T1–T10 and T12 pass (4634 passed, 1 skipped in the full suite); T13 and T14 are n/a as planned; T11 did not run for want of a real repository and daemon — T5 walks the same dispatcher with the same comment against a real checkout.

Activities checklist ​

  • [x] T1–T4, T6, T7 unit tests written red, then green
  • [x] T5 integration scenarios written with Gherkin docstrings, red then green
  • [x] T8 abuse cases asserted in the suite, not only in prose
  • [x] T9 schema parity and sample validation green
  • [x] T10 full make test green
  • [x] T12 docs updated; lint green
  • [x] Evidence committed

Released under the MIT License.