<!-- Authored per the the-loop:writing skill. -->
Bugfix spec: channels.slack.read.mode is hot-reloaded in the config but not in the running process
Phase 1 of 4 (bugfix → design → testing plan → tasks). Source: the 2026-09-19 e2e Slack run's B5 (
docs/reports/e2e-slack-test-2026-09-19.md), left out of issue-393 and filed as issue-395.
Summary
The set of ingresses the service hosts is decided once, at boot; the config that decides it is re-read forever. Every long-lived process re-reads the CLI config from its content hash — the poller before each cycle, the receiver on each event, the service once per request — and each applies what it can: sources, intervals, routing, grants. But which ingresses run inside the service (the poller, the webhook receiver and, since issue-334, the Slack Socket Mode listener) is composed by start_hosted_ingresses in the lifespan's first line and never revisited. So when an operator sets channels.slack.read.mode from socket to off:
- the poller's reload fires
config.reloaded— the file did change; - the hosted listener keeps its Socket Mode connection and keeps consuming events;
the-loop statusreads the config (enabled: false) and the lock (held by the service's pid) and prints the contradiction:slack-listener running (hosted in the service, pid …) [disabled].
This matters most for the B3 workaround: two instances sharing one Slack app split Socket Mode events across their connections, and the way to stop the second one stealing half the traffic is to turn its listener off — which, today, does nothing until the whole service is restarted.
Steps to reproduce
channels.slack.enabled: true,read.mode: socket, both tokens exported,service.hostIngressesat its default;the-loop start.the-loop status→slack-listener running (hosted in the service, pid N) [enabled].- Edit the config:
read.mode: off. Wait forconfig.reloadedinthe-loop events. the-loop status.
Expected vs actual
- Expected: the listener has closed its connection and released its lock;
statusprintsslack-listener not running [disabled]. - Actual:
slack-listener running (hosted in the service, pid N) [disabled]; a mention in the room is still acknowledged by this instance;the-loop restartis the only way out.
Root cause (confirmed)
api/lifespan.py calls ingress.start_hosted_ingresses(cli_config) once and holds the returned list until shutdown. Nothing in the process compares that list with what the current config asks for. The listener itself (run_socket_listener) freezes the config it was started with, by design — its per-message pipeline must not change under it — so the decision to stop or start it can only be taken outside it, by whatever owns its thread and lock: the hosting.
Requirements
Requirement 1 — the hosted ingress set follows the config without a restart
User story: As an operator, I want a change to what the config enables to take effect in the running service the way every other config change does, so that turning a listener off stops it consuming events, and turning it back on brings it back.
Acceptance criteria (EARS)
- WHEN the CLI config file changes so that an ingress the service hosts is no longer enabled (the Slack listener:
channels.slack.enabledfalse orread.modenotsocket; the poller:polling.enabledfalse; the receiver:webhooks.ghWebhook.enabledfalse) THEN the service SHALL stop that hosted ingress — its run loop ended through its stop event, its pidfile lock released — within the supervisor's check interval, with no restart. - WHEN the CLI config file changes so that an ingress the service does not host is newly enabled THEN the service SHALL start it exactly as
the-loop startwould have — the same starter, the same lock, the same refusals (a held lock, missing tokens, an emptypolling.sources) — and a refusal SHALL be recorded asingress.hosted_failed, as at boot. - WHEN the file changes without changing the enabled set THEN no hosted ingress SHALL be stopped or started; each keeps reloading what it reloads today.
- WHEN a change is applied THEN the service SHALL record it:
ingress.hosted_stopped/ingress.hostedper ingress, with areasonnaming the config change, and oneconfig.reloadedwhosedetailnames what was stopped and started. - IF the edited file cannot be loaded (invalid YAML, a schema or version failure) THEN the hosted set SHALL be left as it is, as every other reloader leaves its value.
- WHEN the service shuts down THEN the supervisor SHALL stop first, then every hosted ingress, in reverse start order as today; a check that is mid-flight SHALL never race the shutdown.
- The reconcile SHALL apply only under
service.hostIngresses(a change to that key staysrestartRequired); a foregroundthe-loop channels listenis not the service's to stop. - The fix SHALL include regression tests that fail before the fix and pass after: the listener stops when
read.modeleavessocket, starts again when it returns, and is left alone by an unrelated edit and by an unparseable one.
Requirement 2 — status never prints a contradiction
User story: As an operator reading the-loop status, I want a row that is running against its own config to say what is about to happen, not to assert two opposite things in one line.
Acceptance criteria (EARS)
- WHEN a row is
runningand its config has it disabled THENthe-loop statusSHALL NOT printrunning … [disabled]; the flag SHALL say the row is disabled in the config and still running, and what ends it: for a hosted row, the service's next config check; for any other,the-loop stop. status --format jsonSHALL be unchanged:running,enabledandhostedalready carry the same facts, andokSHALL keep counting enabled rows only.
Security considerations
- Actors & trust: the config file is already the operator's control surface: who can edit it can already start harness sessions, and the daemons already act on its every change. The supervisor reads the same file through the same loader (
load_cli_config(strict=True)), so no new principal and no new input. - New attack surface: none. The reconcile composes the starters
the-loop startcomposes today, with the refusals they already make: the single-instance lock is still taken, a missing token is still refused by name and never by value, and the schema and migration gates still stand between an edit and a load. - Abuse / failure cases: (1) a flapping edit (socket ↔ off every second) — each change stops or starts at most once per check interval, and a start that fails is recorded and not retried until the file changes again; (2) an unparseable edit — no change (R1.5); (3) the supervisor thread raising — caught per check, logged, the service keeps serving, exactly as a starter that raises is handled at boot.
- Secrets / data: none touched; the events name ingresses and key paths, never values.
Out of scope
- Reloading a running ingress's own config (a changed Slack
channel, grants, the catch-up interval). The listener freezes its config on purpose; restarting it on any Slack edit is a different trade-off (a dropped connection per keystroke) and is not what B5 asked for. Only the membership of the hosted set is reconciled. - The foreground
the-loop channels listen: its owner is at the terminal. - B7 (
graph statusfrom a shell), filed separately.
Open questions
None. The report offered two fixes; the design picks the first (stop/start on reload) and takes the second's status wording as a guard for the check-interval window and for a listener nobody hosts.