Skip to content

Verification evidence: one Slack surface, two identity allow-lists

Every row of testing-plan.md that is in scope, with what was run and what came back. Nothing here carries a token, a secret, or a path outside the checkout — the whole matrix is filesystem and in-process checks.

Every new test was run against the unfixed tree first

The source changes were stashed (schemas, configschema.py, migrations.py, the packaged copies, the templates and this repo's configs), leaving only the new tests, and the two touched test modules were run:

text
$ git stash push -- .the-loop/*.schema.json cli/the_loop/configschema.py \
    cli/the_loop/migrations.py cli/the_loop/schemas/ .the-loop/*.yaml \
    skills/the-loop/templates/{cli-config,collaborators}.yaml
$ uv run pytest -q tests/test_configschema.py tests/test_migrations.py

15 failed, 61 passed in 2.29s

All fifteen failures are the new assertions — every retired-key refusal, the guidance messages, the RETIRED-vs-schema cross-check, the differential run over the collaborators corpus, and each of the six migration cases. None of the pre-existing tests failed there, which is the other half of the claim: the new tests fail for the reason they were written, not because the stash broke the module.

T1, T2, T5, T8 — the schema refusals and the drift guards

text
$ cd cli && uv run pytest -q tests/test_configschema.py
40 passed in 1.63s

Covering: notificationChannel and collaborator.notifications gone (R1.1); a collaborator carrying notifications refused with channels.slack andthe-loop migrate-config in the message (R1.2); handle/kind/roles still validating (R1.3); neither collaborators nor notifications a leaf of the CLI config schema, and each refused with the same guidance (R2.1, R2.2); an ordinary typo still getting plain unknown key and no invented advice; both identity allow-lists still schema leaves and still accepting a real value (R2.4); every RETIRED row asserted absent from the schema it names, so a row that could never fire fails the build.

The keyword guard and the differential test against real jsonschema both pass, and the differential now runs the collaborators corpus too — that schema carries every $ref left in the tree since the CLI config's one cross-schema reference was retired.

T3, T4 — the migration

text
$ cd cli && uv run pytest -q tests/test_migrations.py
36 passed in 0.15s

A 0.5.0 config carrying both filled-in blocks migrates to a clean 0.6.0; both removals are reported; state, routing (with its authorizedUsers) and channels come out byte-identical to what went in, and the input mapping is never mutated (A3). A second run reports no change and produces an identical config (R3.3). A config that stamps the current version while still carrying collaborators is still detected and still refused (A1). The refusal names the key, channels.slack, routing.authorizedUsers and the upgrade command (R2.2, A2). A filled-in block gets the "here is what to configure instead" note; the shipped empty default gets the removal and no lecture (R3.4).

The command-level round trip, as an operator sees it:

text
$ the-loop migrate-config --dry-run   # 0.5.0 config with both blocks
migrated the CLI config:
  · collaborators removed — nothing read it (issue-304); Slack is declared once under `channels.slack`
  · notifications removed — nothing read it (issue-304); Slack is declared once under `channels.slack`
  · version '0.5.0' → '0.6.0'
  note: what you had declared there was never delivered on. To be notified, set
        `channels.slack`'s `botTokenEnv` and `channel` and subscribe its `events` list to
        the event names you want — one channel for the daemon, not one list per person;
        per-person routing is not built

$ the-loop migrate-config --dry-run   # the same file, second run
config is already current; nothing to migrate

T6, T7, T11 — the whole suite, unchanged behaviour

text
$ cd cli && uv run pytest -q
2698 passed, 1 skipped in 147.89s

2679 → 2698, nineteen added, none removed and none weakened. The channel suite (test_channels.py, test_channels_integration.py, test_standing_channels_integration.py) is green untouched — no channel code was modified — and the pinned harness-config read-surface test still holds: notifications remains a harness-config key and still gates the graph's notify hook (R2.5, NFR1, NFR2).

Six test fixtures that hard-coded "0.5.0" now read CURRENT_CONFIG_VERSION. That is the fix rather than six new literals: a fixture pinned to a version the code is about to bump fails on every future migration for a reason that has nothing to do with what it tests.

text
$ cd cli && uv run ruff check . && uv run ruff format --check .
All checks passed!
258 files already formatted
$ cd cli && uv run pyright
0 errors, 0 warnings, 0 informations
$ npx markdownlint-cli2@0.18.1 "docs/**/*.md" "skills/**/*.md" "commands/**/*.md"
Linting: 876 file(s)
Summary: 0 error(s)

T8, T9 — the shipped configs and the docs

text
$ uv run --project cli python scripts/validate_config.py
VALID   .the-loop/harness-config.yaml
VALID   skills/the-loop/templates/harness-config.yaml
VALID   cli/the_loop/harness-config.default.yaml
VALID   .the-loop/collaborators.yaml
VALID   skills/the-loop/templates/collaborators.yaml
VALID   .the-loop/cli-config.yaml
VALID   skills/the-loop/templates/cli-config.yaml

All seven validate under real jsonschema, with the retired shapes gone from both templates and both of this repo's own configs, commented examples included (R4.1). The schema/docs parity tests pass in both directions — no doc heading survives for a removed key (P3), and no schema leaf is undocumented (P4) — as do the .the-loop/the_loop/schemas/ byte-parity and manifest assertions.

T10 — no doc promises what no code delivers

text
$ grep -ri "channel-list" docs/ skills/ commands/
docs/config/harness-config.md: transport and a `channel-list` — and **no code ever read it**, …

The one live hit is the sentence that says the shape is gone. The others are under docs/specs/issue-82/, docs/specs/issue-245/ and docs/decisions/decision-035.md — the per-work-item record and the decision log, which are history and are not rewritten. decision-035 instead carries a superseded in part by marker naming this work item, the convention decision-005, decision-021 and decision-030 already set.

text
$ grep -rn "delivered on each recipient" . --exclude-dir=.git
(only this work item's own requirements/testing-plan, quoting the sentence that was removed)

skills/the-loop/reference/collaboration.md now says a notification is gated by notifications.events and delivered to a channel, and names the two allow-lists as the only places identity is declared (R4.2, R4.4). docs/capabilities/channels.md carries two new behaviour bullets and a History row (R4.3).

Released under the MIT License.