Skip to content

<!-- Authored per the the-loop:writing skill. -->

Testing plan: the listener measures its own share of the traffic ​

Derived from bugfix.md and design.md. Planned at test-planning, results recorded at verification (below).

Test matrix ​

#TypeApplies?Scope / what it provesWhere it runs
T1UnityesSplitWatch.run_cycle against a fake client: every beat echoed → ok; a beat withheld → split-suspected; every posted heartbeat deleted; the window aborts on the stop eventmake test (tests/test_channels_splitwatch.py)
T2Unityesthe escalation ladder: warning on the first short check, silence until the ESCALATE_EVERYth, channel.split_cleared and a reset on the next clean onemake test (tests/test_channels_splitwatch.py)
T3UnityesSplitState round-trips through the file; recent keeps RETAINED newest; suspected() is true while any retained check is short and false after a full clean window; a corrupt or absent file reads as Nonemake test (tests/test_channels_splitwatch.py)
T4Unityesthe refusals: no channel, no bot token, a channel that resolves to nothing, and a client that raises — each unverifiable with a reason, never ok, never raisingmake test (tests/test_channels_splitwatch.py)
T5Unityes_split_check_beats clamps: non-integer and negative → default with a warning; 0 → off; above the cap → the capmake test (tests/test_channels.py)
T6Integrationyesrun_socket_listener end to end over the fake Socket Mode client: a cycle at connect, one per reconcile deadline, observe fed from the heartbeat branch, channel.heartbeat still emitted for the doctor, and a raising cycle that does not end the listenermake test (tests/test_channels_splitwatch_integration.py)
T7Integrationyesthe surfaces: status prints the report and carries slackSplit in JSON while suspected, prints nothing when clean or unwritten, and leaves ok and the exit code unmoved; channels status prints [!] when short and the cadence line when not; /the-loop status carries the same linemake test (tests/test_channels_splitwatch_integration.py)
T8Contract (catalog + docs + schema parity)yesboth event types are in EVENT_TYPES; the new config key is in the schema and documented; slack_split is classified in GENERATED_PATHS and docs/cli/state.md; test_docs_parity, test_config_schema_parity and test_state_portability stay greenmake test
T9Security / redactionyesno token value, and no part of one, appears in the state file, either event record, or any rendered line — asserted on the clean, short and unverifiable pathsmake test (tests/test_channels_splitwatch_integration.py)
T10Regression (full suite)yesevery suite stays green; ruff, ruff format, pyright, markdownlint, validate_configmake check
T11End-to-end against real Slacknoneeds a workspace and a deliberate second consumer; the incident transcript in the ticket is the field evidence, and T6 reproduces the split by withholding an echo
T12UI / visualn/a — no product UI
T13Accessibilityn/a — CLI text only
T14Migration / upgradeyesan existing deployment needs no migration: the config key is additive with a working default, and an absent state file renders nothingwith T7/T10
T15Manual exploratoryno — every surface is asserted directly

Scenarios & requirement trace ​

RowRequirement(s)Scenario / case
T1R1.4a cycle posts exactly beats messages, counts the echoes it observed, and deletes each ts it got back
T1R1.4one echo withheld → verdict: split-suspected, echoed one short of beats
T1R1.8a stop event set during the window ends the wait inside one poll interval and writes nothing
T2R2.1the first short check emits channel.split_suspected at warning with beats, echoed, window, channel and consecutiveShort: 1
T2R2.2short checks 2…5 emit nothing; check 6 emits again
T2R2.3a clean check after a short one emits channel.split_cleared at info and resets the counter
T3R2.4the state written by one cycle is the state the next process reads
T3R2.4, design § Data modelsrecent holds the newest 8; the 9th append drops the oldest
T3R3.1, R3.2suspected() true on a clean check that follows a short one; false once 8 clean checks have passed
T3R2.5an unwritable path warns once and the cycle still returns its state
T4R1.6no channels.slack.channel → unverifiable, reason names the key, nothing posted
T4R1.6a name that resolves to no conversation → unverifiable, nothing posted
T4R1.5splitCheckBeats: 0 → for_config returns None, nothing posted, no state written
T4R1.7a client whose chat_postMessage raises → unverifiable with the exception text, no raise out
T5R1.4splitCheckBeats: "many" and -1 → the default, one warning each; 9 → the cap
T6R1.1the listener runs one cycle between catch_up and its first tick
T6R1.2with catchUpSeconds reached, the next cycle runs beside the reconcile
T6R1.3catchUpSeconds: 0 → exactly one cycle, at connect
T6R1.4, R2.1a heartbeat arriving on the socket handler is observed by the watch and emitted as channel.heartbeat, so doctor slack still reads it
T6R1.7a run_cycle that raises is logged and the listener keeps serving envelopes
T7R3.1a written short state → one slack report in the-loop status naming beats, echoed, the recent window and the remedy
T7R3.2a clean state, and no state file at all → no report
T7R3.3status --format json carries slackSplit with the same counts
T7R3.4channels status prints [!] while suspected and the cadence + last verdict otherwise
T7R3.5render_status (the slash command) carries the same sentence
T7R3.6every rendered report contains the evidence-not-proof caveat
T7R3.7a suspected split leaves status's ok true and its exit code 0
T7R4.1, R4.2the remedy string in the event, status, channels status and doctor slack is byte-identical and names both stopping and rotating
T8R2.1, R2.3channel.split_suspected and channel.split_cleared are in EVENT_TYPES
T8R1.4channels.slack.read.splitCheckBeats is in the schema and has a documented Type and Default
T8R2.4StateLayout.slack_split is classified local in code and in docs/cli/state.md
T9bugfix § Securityneither token value appears in the state file, the two event records, or any line of the three rendered surfaces
T10allmake check green

Requirements not covered by an automated test ​

  • R4.3 (the rotation procedure in the Slack guide) is prose; its presence is checked by review, not by a test. A test asserting that a paragraph exists would pin wording without pinning correctness.
  • A real second consumer. T6 reproduces the split by withholding an echo, which is what a split does to this process; that the cause is Slack's load balancing is the incident transcript's evidence and Slack's documented behaviour, neither of which a test in this repository can restate.

Verification environment ​

make check on the repository checkout: uv (pinned >=0.12,<0.13), Python as cli/pyproject.toml declares, no network. Every Slack interaction is a fake client; no test opens a socket.

Evidence to capture ​

  • the full make check transcript (evidence/verification.md);
  • the rendered the-loop status, channels status and /the-loop status output for a suspected split, captured from the tests;
  • the security review (evidence/security-review.md) and the documentation record (evidence/documentation.md).

Activities checklist ​

  • [x] test matrix agreed and traced to requirements
  • [x] tests written before the code they cover (TDD)
  • [x] make check green
  • [x] evidence recorded under evidence/
  • [x] results table below completed at verification

Results ​

Recorded at the verification node — see evidence/verification.md.

RowCommandOutcomeArtifact
T1–T5uv run --project cli python -m pytest -q cli/tests/test_channels_splitwatch.pypassevidence/verification.md
T6, T7, T9uv run --project cli python -m pytest -q cli/tests/test_channels_splitwatch_integration.pypassevidence/verification.md
T8uv run --project cli python -m pytest -q cli/tests/test_docs_parity.py cli/tests/test_config_schema_parity.py cli/tests/test_state_portability.py cli/tests/test_eventlog.pypassevidence/verification.md
T10make checkpassevidence/verification.md

Released under the MIT License.