Skip to content

Testing plan: the session that opens a pull request is the one that records it

Derived from the approved bugfix.md and design.md, before tasks.md. Authored at test-planning, completed at verification.

Test matrix

#TypeApplies?Scope / what it provesWhere it runs
T1Unityescore.sessions.link_pull_request: the happy path and its session.pr_linked, idempotence, no session record, self-link, bare-number / #N / cross-repository resolution, malformed refsmake test (uv run pytest cli/tests)
T2Integration (scenario)yesthe reproduction end to end — a review comment on a the-loop-authored spec PR (no closing reference, loop/… branch, no closing keyword) is dropped as awaiting-start without the binding and delivered into the work item's session with it — Gherkin-documentedmake test
T3Contract (OpenAPI)yesthe authored the-loop.v1.yaml and the served schema both carry POST /api/v1/sessions/link-pr / linkSessionPullRequest, and the embeddable router carries it toomake test (test_api_contract_parity.py)
T4End-to-endn/a — an E2E run needs a real GitHub repository, a real gh credential and a real tmux/harness. T2 drives the real Dispatcher, the real SessionRegistry and the real router against injected tmux/provider fakes, which is how this path has always been proved in this repo
T5UI / visualn/a — no user-facing surface; the change is a CLI action, an API route, an MCP tool and documentation
T6Snapshotn/a — no rendered artefact changes
T7Performance / loadn/a — one appended endpoint per pull request, written once by an operator-initiated call. Nothing on a hot path changes
T8Security / abuse caseyesthe operation cannot create a work item (no record → no write), cannot link a work item to itself, and refuses a malformed ref before touching the filesystemmake test
T9Accessibilityn/a — no user interface
T10Migration / upgraden/a — no state shape changes. The endpoint written is the pullRequests[] entry the registry has held since issue-172, and a record without one is exactly today's record
T11Manual exploratoryn/a — the reproduction is mechanised as T2, which is stricter than the observed symptom: it asserts which session the event reached, not merely that it was not dropped
T12Static analysis (lint + types)yesruff, pyright, markdownlint over the changed modules and docsmake lint
T13Docs ↔ code parityyesthe CLI, SDK and event-catalogue documentation still match the code (test_docs_parity.py, test_sdk_docs_parity.py, test_writing_parity.py, test_eventlog.py)make test

Scenarios & requirement trace

RowRequirement(s)Scenario / case
T1R1.2a pull request not yet listed is added to the record and session.pr_linked is emitted once
T1R1.3a second call for the same pair rewrites nothing, emits nothing, and still exits 0
T1R1.4a work item with no session record: exit 1, no file written
T1R1.5linking a work item to itself: exit 2
T1R1.6--pull-request 275, --pull-request '#275' and a full ref in another repository all resolve to the intended endpoint
T1R1.7a malformed work-item ref, a malformed pull-request ref and a non-positive number are all exit 2, nothing written
T1R1.1the CLI action, the HTTP route and the MCP tool all reach the one core function
T2R1.2, R3.2Scenario: a review comment on a spec PR the-loop opened reaches the work item's session — with and without the binding
T3R1.1the served schema, the authored contract and the embeddable router agree on linkSessionPullRequest
T8Security designno record → no write; self-link refused; malformed ref refused before the registry is opened
T13R2.1, R2.2, R2.4the skill, the two commands, the execution-log template and the capability doc carry the rule; the new SDK method is documented (P1) and reaches core (P2); the event catalogue still matches what is emitted
T12R3.1lint and type checks pass over the changed modules

Verification environment

  • Repositories: this repository only.
  • Services / containers: none. No tmux, no harness, no network: the dispatcher's tmux runner is an injected fake, as in cli/tests/test_routing.py, and the CLI tests run on the in-process seam (THE_LOOP_SERVICE_LOCAL=1) rather than standing a service up.
  • Fixtures & data: the existing registry/dispatcher fixtures; tmp_path for the state root so the record is asserted on disk.
  • Credentials: none. No gh is invoked by any path under test.
  • Bring-up: make test · Tear-down: none (pytest tmp_path).
  • If bring-up fails: record it under Verification results, leave the dependent activities unticked, and escalate.

Evidence plan

RowEvidencePath under evidence/
T1, T2, T3, T8red run — the new tests, run before any production code changedred.md
T1, T2, T3, T8, T13green run — full suite summary and the per-file runsunit-and-integration.md
T12make lint / type-check outputlint-and-typecheck.md
security review record (checklist per reference/security.md)security-review.md

Verification activities

  • [x] T1 — uv run --project cli python -m pytest -q cli/tests/test_core_sessions.py cli/tests/test_cli.py
  • [x] T2 — uv run --project cli python -m pytest -q cli/tests/test_webhook_routing_integration.py
  • [x] T3 — uv run --project cli python -m pytest -q cli/tests/test_api_contract_parity.py cli/tests/test_api_routers_integration.py cli/tests/test_mcp_integration.py
  • [x] T8 — uv run --project cli python -m pytest -q cli/tests/test_core_sessions.py -k link
  • [x] T13 — uv run --project cli python -m pytest -q cli/tests/test_docs_parity.py cli/tests/test_sdk_docs_parity.py cli/tests/test_writing_parity.py cli/tests/test_eventlog.py
  • [x] T12 — make lint (ruff check, ruff format --check, markdownlint-cli2) and uv run pyright cli
  • [x] Full suite — make test
  • [x] Red run captured before the fix — evidence/red.md
  • [x] Security review — the checklist in reference/security.md, against the diff

Verification results

ActivityCommand / procedureOutcomeEvidence
Red runthe 19 new tests, written and run before any production code changed18 failed, 1 passed — the pass is the control, which asserts the bug and passes on both sides of the fixevidence/red.md
T1pytest -q cli/tests/test_core_sessions.py cli/tests/test_cli.py and pytest -q cli/tests/test_routing.py -k link_pr68 passed; 2 passedevidence/unit-and-integration.md
T2pytest -q cli/tests/test_webhook_routing_integration.py29 passedevidence/unit-and-integration.md
T3pytest -q cli/tests/test_api_contract_parity.py cli/tests/test_api_routers_integration.py cli/tests/test_mcp_integration.py18 passedevidence/unit-and-integration.md
T8pytest -q cli/tests/test_core_sessions.py -k link16 passedevidence/unit-and-integration.md
T13pytest -q cli/tests/test_docs_parity.py cli/tests/test_sdk_docs_parity.py cli/tests/test_writing_parity.py cli/tests/test_eventlog.py35 passedevidence/unit-and-integration.md
Full suitemake test2501 passed, 1 skipped after rebasing onto main at 71e7dff; pre-rebase, on main at 50c2a27: 2495 passed, 1 skipped, up from 2476 — the 19 this work item addsevidence/unit-and-integration.md
T12uv run ruff check cli hooks, uv run ruff format --check cli hooks, uv run pyright cli, markdownlint-cli2 (850 files), scripts/validate_config.pyclean on the first run of eachevidence/lint-and-typecheck.md
Security reviewchecklist (reference/security.md), effective risk tier 3pass, no findingsevidence/security-review.md

Every planned activity ran. One row was replanned before it ran: T13 named test_harness_usage.py, which turned out to be the harness token-telemetry parser and has nothing to do with documentation parity. It is replaced by test_sdk_docs_parity.py and test_eventlog.py — the two parity tests this change actually has to satisfy. Nothing is left unticked.

Released under the MIT License.