Skip to content

Testing plan: a default harness config for repositories that never adopted the-loop

Derived from the approved requirements.md and design.md, before tasks.md — each task's _Test:_ names a row of the matrix below. Authored at the test-planning node and completed at the verification node: the same file is written once as a plan and once as a record, so intent and outcome sit in one diff.

This file is executable content. It names commands an agent will run, so review it like code. Credentials appear by reference only — this change needs none.

Test matrix

#TypeApplies?Scope / what it provesWhere it runs
T1Unityesdefaults() and scaffold(): the written bytes, the provenance header, owner/repo substitution, idempotence, and each degradation pathpytest cli/tests/test_harness_config.py
T2Integration (scenario)yesthe two real call sites: GraphLink adopting a git checkout with no .the-loop/, the contribution loop and the non-driving actions refusing to, and core.graphs mutating-vs-readingpytest cli/tests/test_harness_config_scaffold_integration.py
T3Contract (OpenAPI / GraphQL SDL)n/a — the control-plane API gains no route and no field; core.graphs._runtime is internal
T4End-to-endn/a — T2 drives the real GraphLink and the real core.graphs entry points against real git checkouts, which is as far as the daemon path goes without a live GitHub
T5UI / visualn/a — no user-facing surface (design § UI/UX)
T6Snapshotn/a — the one "golden file" is the packaged default, and T7 pins it by byte parity with the template rather than by a snapshot of its own output
T7Parity / drift (repo-level)yesthe packaged default vs the /the-loop:init template (bytes), vs the schema, vs the graph's phase sequence, vs the module's per-key default constantspytest cli/tests/test_graph_parity.py cli/tests/test_harness_config.py
T8Security / abuse caseyesa negative test per trust boundary in design.md § Security design: forged owner/repo, foreign checkout, existing config, guest repository, escaping .the-loop symlinkpytest … -k "forged or foreign or overwrite or contribution or escapes"
T9Accessibilityn/a — no user interface
T10Migration / upgradeyesa repository that already carries a config — current name or the pre-rename config.yaml — is left untouched, so upgrading the-loop never rewrites an operator's policypytest cli/tests/test_harness_config.py -k present
T11Manual exploratoryn/a — every path is reachable from the test suite; a manual run would exercise the same two functions with less coverage
T12Regression (whole suite)yesevery existing test still passes: adoption now writes a file into checkouts many of them build without onemake test
T13Lint / type-checkyesrepository gates: ruff, pyright, markdownlint over the new module code and the new markdownmake lint format-check typecheck validate

Scenarios & requirement trace

RowRequirement(s)Scenario / case
T1R1.1, R1.4defaults() returns the packaged mapping; an unreadable package file yields {}
T1R2.1, R2.2scaffold() writes the file and names owner/repo under ticketing.github
T1R2.4, R2.5a second call returns "present"; an unwritable tree returns ""
T2R2.1, R2.3Scenario: the ingress adopts a repository that never ran the-loop's setup
T2R2.1Scenario: an adopted repository's graph is still skipped while the work item has no spec directory
T2R4.1, R4.2Scenario: a contribution never adopts its host repository
T2R3.1, R3.2Scenario: a mutating graph verb adopts, a read-only one does not
T2R2.1Scenario: resolving a prompt's graph context adopts nothing (self-review)
T2R2.1Scenario: releasing a work item's resources adopts nothing (self-review)
T7R1.2, R1.3the packaged default is byte-identical to the template, valid against the schema, and declares the graph's phase sequence
T7R1.1DEFAULT_SPEC_DIR and the runtime's phaseLabelPrefix fallback equal what the packaged default declares
T8abuse 1Scenario: a forged owner is dropped rather than written into the YAML
T8abuse 2Scenario: a checkout that is not the work item's repository is never adopted
T8abuse 3Scenario: an existing harness config is never overwritten
T8abuse 4covered by the contribution scenario above
T8abuse 5Scenario: a committed .the-loop symlink does not redirect the write (security review)
T10R2.4a pre-rename config.yaml counts as adopted; nothing is written beside it

Verification environment

  • Repositories: this repository only.
  • Services / containers: none. The tests that need a repository run git init in a tmp_path; GitHub is never contacted (the existing fake_github double serves the hooks).
  • Fixtures & data: the shipped cli/tests/conftest.py doubles; no recorded traffic.
  • Credentials: none — this change reads no environment and no secret.
  • Bring-up: uv sync · Tear-down: none.
  • If bring-up fails: record it under Verification results, leave the dependent activities unticked, and escalate — do not pass the gate on an environment that never came up.

Evidence plan

RowEvidencePath under evidence/
T1, T2, T7, T8, T10targeted run: command + raw outputunit-and-integration.md
T12whole-suite run: counts, duration, raw tailregression.md
T13lint + type-check outputlint-and-typecheck.md

Verification activities

  • [x] T1 — uv run --project cli python -m pytest -q cli/tests/test_harness_config.py
  • [x] T2 — uv run --project cli python -m pytest -q cli/tests/test_harness_config_scaffold_integration.py
  • [x] T7 — uv run --project cli python -m pytest -q cli/tests/test_graph_parity.py + uv run python scripts/validate_config.py
  • [x] T8 — uv run --project cli python -m pytest cli/tests/test_harness_config.py cli/tests/test_harness_config_scaffold_integration.py -k "forged or foreign or overwrite or contribution or escapes"
  • [x] T10 — uv run --project cli python -m pytest cli/tests/test_harness_config.py -k "present or pre_rename or overwrite"
  • [x] T12 — make test
  • [x] T13 — make lint format-check typecheck validate

Verification results

ActivityCommand / procedureOutcomeEvidence
T1pytest cli/tests/test_harness_config.pypass — 37 passedevidence/unit-and-integration.md
T2pytest cli/tests/test_harness_config_scaffold_integration.py -vpass — 9 passed, one per scenarioevidence/unit-and-integration.md
T7pytest cli/tests/test_graph_parity.py · scripts/validate_config.pypass — 9 passed; the packaged default reported VALID against the harness schema in its own rightevidence/unit-and-integration.md
T8pytest … -k "forged or foreign or overwrite or contribution or escapes"pass — 10 selected, 10 passed (one per abuse case; the forged-input row is parametrized six ways)evidence/unit-and-integration.md
T10pytest cli/tests/test_harness_config.py -k "present or pre_rename or overwrite"pass — 3 selected, 3 passedevidence/unit-and-integration.md
T12make testpass — 1715 passed, 1 skipped; 29 of those are this work item's (19 unit, 9 integration, 1 new parity parametrization)evidence/regression.md
T13make lint · format-check · typecheck · validatepass — ruff clean, 523 markdown files 0 errors, pyright 0 errors, 7 configs VALIDevidence/lint-and-typecheck.md

Not executed: none — every activity in the matrix ran, and none was replanned.

Review comments

Appended by the-loop's record-feedback hook when a human gate approves with comments (issue-109). Append-only and attributed: an approval never silently discards a reviewer's suggestions, and the feedback travels with the document it concerns rather than living in a side-channel tracker.

Released under the MIT License.