Observability reference

the-loop configures nothing here (issue-352, third pass): log levels are the project's own logging configuration, and browser logging uses whatever browser-logging tool the harness discovers. What this file carries is the rule. The CLI event log below is configured separately, in the independent CLI config (eventLog; decision-032).

Core rule: dev-time == run-time

Logging and observability for debugging MUST be EXACTLY the same during development and at runtime (when the system is actually running). The ONLY advantage at dev-time is that you can set breakpoints and inspect variable values. Do not add log statements or instrumentation that exist only in dev — that defeats the purpose.

Levels

The levels are the project's own — set wherever its logger is configured, never in the-loop's config:

• Dev-time (typically debug): all debug logs and above are accessible.
• Run-time (typically info): all info logs and above are available.

Use the same logger and the same log lines in both; only the active level differs.

CLI event log — the audit trail of the-loop's own actions

Config: eventLog in the CLI config (cli-config.yaml: enabled, default true; path, default .the-loop/logs/events.jsonl, git-ignored) — independent of any repo's plugin config, resolved via --config/env/cwd/home (see docs/config/cli/, decision-032). Decision: decision-025 — JSONL, not SQLite.

the-loop's CLI processes (the-loop gh-webhook, the poller, the-loop sessions) append every accept / reject / dispatch / spawn / retry / close decision they make as one JSON object per line to the event log. It is the durable, machine-queryable answer to:

• which events triggered a particular session? — session.spawned / dispatch.succeeded records name the work item, harness, session id and the triggering GitHub event + delivery id;

• which events were rejected / accepted, and why? — webhook.rejected, routing.dropped and dispatch.dropped carry a machine-readable reason (e.g. invalid-signature, unauthorized-actor, duplicate-delivery, spawn-policy, awaiting-start, session-paused, session-occupied);

• was a comment refused, or did a delivery fail? — poll.comment_settled says the-loop refused the event on purpose (an unstarted item, a paused session) or consumed it as a control command, so it was baselined and will never be retried or replayed; poll.comment_failed says a delivery was attempted and gave up. The two used to be indistinguishable in the ledger (issue-270);

• why did nothing get respawned? — session.respawn_averted says the tmux session was alive after all, so the pending event was pasted into it (issue-146); a dispatch.dropped with reason: session-occupied says a dead session held the loop-<slug> name and would not clear, so the event was skipped rather than retried into the same collision forever;

• who started (or stopped) this autonomous run? — control.command records the command, its source (comment | cli), the actor and the effect (spawned/resumed/paused/stopped/noop); control.rejected and control.ambiguous record the requests that were refused (issue-106);

• what failed, and will it retry? — dispatch.failed, session.spawn_failed, poll.provider_error carry error and will_retry (whether GitHub redelivery / the next poll cycle heals it);

• what has the daemon changed on this machine? — workspace.prepared / workspace.cleaned name every checkout it created and removed, and workspace.trusted / workspace.trust_failed name every directory it marked trusted in the harness's own config before a spawn (issue-90). Those two are the audit trail for the one thing the-loop writes outside its own state: the-loop events --type 'workspace.trust*';

• what did a conversation channel carry? — channel.posted / channel.reply_received / channel.mirrored trace a question's fan-out and a reply's round-trip (Slack thread → work-item mirror → session delivery), and channel.dropped carries the machine-readable reason (unmapped, self-authored, unauthorized-actor, undeliverable) — issue-245. Payloads name ids, never message text and never tokens;

• did a handoff take, and if not, why? (issue-517) — the-loop events --type 'handoff.*' --work-item <ref>:

  Event	Meaning
  handoff.honoured	A GitHub issue not yet on the graph was armed with a handoff record (work_item, stage, enter, author, carried_by, branch, record): its checkout is made on the record's spec branch and Runtime.start freezes the record into work-item-state.json.
  handoff.ignored	One or more records were not honoured (work_item, reasons: each record's URL and why — author not authorized, unknown version, malformed, item already on the graph). The item arms as one with no handoff.
  handoff.unreadable	The comments of an item being armed could not be listed, so no record was read (work_item, error). Warning-level; the item arms with no handoff — fail closed, never a guessed stage.
  handoff.imported	the-loop handoff import wrote the carried files into the spec folder (work_item, written, refused: counts). Warning-level when any file was refused; the import-handoff hook then demotes the walk.
  handoff.demoted	The import-handoff hook refused a carried artifact (missing, a sha256 mismatch, or a failed section check), so the runtime un-skipped the nodes from its producing node up to the record's enter (work_item, nodes). The item starts earlier, never later; the refusal is in handoffDemoted.
  arm.refused	A work item was not armed (work_item, reason). reason=mirror: a Jira copy labelled the-loop-mirror — the GitHub issue it mirrors runs the loop. Settled, never retried.

Record shape

Envelope on every record: ts (UTC ISO-8601, ms), source (gh-webhook | poll | sessions), event (dot-namespaced type), level (debug|info|warning|error), pid. Common per-type fields: work_item / work_items (github:owner/repo#15), delivery_id (joins one delivery's whole trail), gh_event + action, actor, harness, harness_session_id, reason, error, will_retry.

The full catalog of event types with descriptions is the-loop events --types (source of truth: EVENT_TYPES in cli/the_loop/eventlog.py; a unit test keeps emitted types and the catalog in sync).

Querying it (humans and agents)

the-loop events                                    # last 50 events, table
the-loop events --work-item github:octo/repo#15    # one item's full history
the-loop events --delivery-id <uuid>               # one delivery, end to end
the-loop events --type 'dispatch.*' --level error  # what failed
the-loop events --since 2h --follow                # live tail
the-loop events --format json                      # machine-readable (agents)
the-loop events --types                            # documented event catalog

The file itself is plain JSONL — grep, jq, tail -f and any dashboard/DB ingestion work directly on it (e.g. jq 'select(.event=="session.spawned")' .the-loop/logs/events.jsonl). Agents asked to "dig through the events" should start with the-loop events --types, then filter with the flags above (or read the file directly when unavailable).

Writes are append-only and multi-process safe; a broken log never breaks ingress (write failures are warned once and swallowed). There is no built-in rotation — truncate/rotate externally; readers tolerate it.

Observing services

• Locally running services: prefer file-system based logging that the agent harness can read directly (tail/grep the log files).
• Browser-based logging: surface browser console/network logs to the harness through whatever browser-logging tool the harness discovers (e.g. chrome-devtools-mcp, the Chrome DevTools MCP server) — tools are discovered, not declared.
  • Open question: confirm chrome-devtools MCP is the right mechanism for browser logging, and document the setup. Record a decision once validated.