Requirements: one rule for where a work item's attributes live
Phase 1 of 3 (requirements → design → tasks). Following the Kiro spec approach (https://kiro.dev/docs/specs/). This phase MUST be reviewed and approved by the required collaborators before moving to design.
The ticket asks for two reports. The audit is § Audit below — it is this document's "what was broken". The viewpoint and the from → to example are
design.md— they are the "what we did about it", and the owner approves them atdesign-approval. Neither is duplicated underdocs/reports/.
Introduction
Issue-368: "various attributes that are tracked by the-loop for a work item are scattered across many different files and folders — some listed as portable, some as private — and there is no principle-driven approach to this." The ticket states the principle it wants:
- A remote entity created for a work item (a pull request, a Slack thread) is tracked in a file that belongs to the work item.
- An entity local to the machine (a tmux session, a harness conversation id, a pid, a heartbeat) is tracked in a local file.
- One file for the work item — the ticket proposes
work-item-state.json. - One file for all local tracking — the ticket proposes
work-item-local-state.json.
The audit below confirms the diagnosis and sharpens it: the-loop already classifies its generated files by portability (decision-046), but it has no rule for attributes, and the two files that would carry a work item's remote entities today carry none of them — a hand-off to another machine loses which pull requests deliver the item and which Slack thread carries it — while the one file that is checked into the repository carries a harness session id it should not.
Audit — what is tracked where today
Five kinds of file hold state about one work item, written by four components, and no two of them agree on why an attribute is where it is. For a work item github:octo/app#15 that three pull requests deliver:
flowchart LR
subgraph repo["origin repository · tracked on the work item's branch"]
WS["docs/specs/issue-15/<br/>work-item-state.json"]
IL["docs/specs/issue-15/pr-loops/<br/>pr-16/ · octo__lib/pr-7/ · octo__infra/pr-3/<br/>work-item-state.json ×3"]
end
subgraph root["operator's state.root"]
subgraph portable["portable/ · tracked in the operator's repo"]
PR["github-octo-app-15.json<br/>control · poll · graph · collaborators · ended"]
PP["github-octo-app-16.json · github-octo-lib-7.json · github-octo-infra-3.json<br/>one more per pull request: poll · ended"]
IX["index.json (derived)"]
end
subgraph local["local/ · never tracked"]
LR["github-octo-app-15.json<br/>session handles + one endpoint per PR"]
end
subgraph channels["channels/ · never tracked"]
SL["slack.json<br/>threads · conversations · cursors · pending<br/>(every work item, one file)"]
end
MW["poll.pid · gh-webhook.pid · slack-listener.pid<br/>poll-status.json · logs/ · self-diagnosis.json<br/>local/model-verdicts.json"]
end
S["session / runtime<br/>(the-loop graph …)"] --> WS & IL
D["daemon: GraphLink"] --> WS & IL
D2["daemon: dispatcher · poller"] --> PR & PP & LR & SL & MW
A["agent CLI:<br/>sessions link-pr · ask · channels"] --> LR & SLThe attributes, file by file
Each attribute is classified by what it is — the axis the ticket asks for — and checked against the ticket's two principles. Kinds:
| Kind | Meaning | Where the ticket's principle puts it |
|---|---|---|
| P pointer | where the graph is for this work item | with the work item |
| H human decision | a choice an authorized human froze (skips, surface, model…) | with the work item |
| R remote entity | something that exists upstream because of this work item (a PR, a thread) | with the work item |
| L operator ledger | what this deployment was told or has already seen (armed, roster, seen comments, closure) | with the work item, on the operator's side |
| M machine handle | a conversation id, a tmux name, a cwd, a cursor, a pid | a local file |
| D derived | rebuildable from other attributes; read by nothing or by a dashboard only | anywhere, declared derived |
docs/specs/<id>/work-item-state.json — checked in, written by the runtime (session verbs and the daemon's GraphLink), a cache re-derived from the artifacts (graph/state.py).
| Attribute | Kind | Principle held? |
|---|---|---|
version, workItem, loop | P | yes |
currentNode, nodes{attempts, outcome, enteredAt, exitedAt, lastBlock, forced}, parked, forced[], completions | P | yes |
decisions (gate outcomes with provenance) | H | yes |
skips, optIns (with via, by, at) | H | yes |
surface | H | yes |
repos[] (declared by the agent through graph repos) | H | yes |
session {id, runner, alive} | M | no — a harness conversation id in a tracked file |
docs/specs/<id>/pr-loops/[<owner>__<repo>/]pr-<n>/work-item-state.json — one per inner loop, same shape. Same verdict: the pointer belongs here; the session block does not. The inner loop's identity (which repository, which number) is carried only by the directory name.
<state.root>/portable/<slug>.json — tracked in whichever repository holds the operator's state.root; written by execution control, the poller, the collaborator store and the dispatcher's close path (workitem.py, control.py, poller.py, collaborators.py).
| Section · attribute | Kind | Principle held? |
|---|---|---|
ref, url | identity (D for url) | yes |
control {command, source, actor, requestedAt, note, instance} | L | yes |
poll {seenComments, commentAttempts, spawn{attempts, gaveUp, deliveryId}, lastPolledAt, closureCheckedAt, title} | L (title is D) | yes |
collaborators {users[{login, addedBy, addedAt, source, note}]} | L | yes |
ended {state, kind, reason, at, source, actor} | L (an upstream fact this deployment observed) | yes |
graph {loop, workItem, surface, sessionPerPr, model, effort, nodes[]} | H | partly — the same decision as the state file's, written a second time; sessionPerPr, model and effort exist only here, so the work item's own file does not say what it runs on; nodes[] is D |
a second record, portable/<pr-slug>.json, per pull request — its own poll (a labelled PR is listed by the poller as an item of its own) and, on merge, its own ended | L | no — one work item, N+1 records and N+1 index entries; the control plane joins them client-side (issue-302) |
<state.root>/local/<slug>.json — never tracked; the session registry (sessions/registry.py), written by the dispatcher on spawn, sessions register, sessions link-pr, and on every delivered event.
| Attribute | Kind | Principle held? |
|---|---|---|
workItem {ref, provider, owner, repo, number}, url | identity | yes |
harness, harnessSessionId, cwd, tmuxTarget, status, createdAt, lastEventAt, recentDeliveries[], model, effort, harnessArgs[] | M (model/effort here mean launched as) | yes |
pullRequests[{workItem{ref…}, url, harness, harnessSessionId, cwd, tmuxTarget, status, recentDeliveries}] | R + M mixed | no — the fact "PR #7 in octo/lib delivers #15" is a remote entity, recorded only in a file that must never leave the machine |
<state.root>/channels/slack.json — never tracked; one file for every work item; written by the agent's session (the-loop ask, the notify hook), both daemons' ingress and the poll watcher (channels/state.py).
| Map | Kind | Principle held? |
|---|---|---|
conversations {workItem → {channel, thread, opened, origin, permalink}} | R | no — the thread the-loop opened for a work item is recorded only locally, so a second machine opens a second thread (the defect issue-312 fixed within one machine) |
threads {thread → {workItem, channel}} | D (the reverse of conversations) | derived, but read on every inbound message |
cursors {thread → last ts} | M (what this deployment mirrored) | yes, but keyed by thread in a file shared by every work item |
pending {ts → {channel, author, text, options, asked}} | M (a question with no work item yet) | yes — not a work item's |
Machine-wide files — poll.pid, gh-webhook.pid, slack-listener.pid, poll-status.json, logs/events.jsonl, logs/poller.out, self-diagnosis.json, local/model-verdicts.json, local/standing/<name>.json: all M, none per work item, all already local. The ticket lists three of them as examples of principle 2; they already satisfy it and nothing about them moves (a pidfile cannot become a key in a JSON document — it is the flock the single-instance guard holds, and an atomic rewrite would release it, decision-076).
Findings
- F1 — a work item's pull requests are recorded only in the file that never travels.
local/<slug>.json.pullRequests[]is the only record that PR #7 inocto/libdelivers#15. After a hand-off the new machine re-derives that fromgh's closing references on each event — the inference issue-172 stopped trusting — or waits for a session to re-runsessions link-pr. A pull request the-loop opened itself carries none of the derivable linkages, so it is simply lost. - F2 — the Slack thread is recorded only locally, once for every work item. The binding is a remote entity the-loop created; on the next machine the next event opens a second root, and replies in the first thread are dropped as
unmapped. - F3 — a machine handle is checked into the repository.
work-item-state.json.sessionholds the harness conversation id and runner; every inner-loop state holds one more.local/is never tracked precisely because a resumable session id and an absolute path do not belong in a repository (state.pyGENERATED_PATHS); the state file carries the first of them anyway. - F4 — the frozen selection is written twice, and the work item's copy is the incomplete one.
skips,optIns,surface,repos,loopare in the state file;sessionPerPr,model,effortand the renderednodes[]are only in the portable record. The daemon reads the portable copy on every pull-request event; the session reads the state file; neither file says everything. - F5 — one pull request has three spellings. Its endpoint in
local/, its inner-loop directory (pr-loops/octo__lib/pr-7/), and — when it is labelled or linked upstream — a portable record of its own (the two identitiescontrol-plane.mdjoins by hand). - F6 —
channels/slack.jsonmixes two lifetimes. Per-work-item state (conversations,threads,cursors) shares a file, a cap and a lock with state that has no work item (pending, the kickoff cursors), so a work item's thread is forgotten by a cap on someone else's count. - F7 — there is no rule to check a new attribute against.
GENERATED_PATHSpins every generated file to portable/local and fails the build when a path is added unclassified. Nothing does that for an attribute, andwork-item-state.jsonis outside the declaration entirely (it does not live understate.root). - F8 — one work item, N+1 portable records. The poller lists labelled pull requests as items of their own (
list_labeled_prs), so each PR delivering a work item gets its ownportable/<pr-slug>.jsonwith apollledger, and the close path stampsendedon it when it merges (_record_closureruns for the PR's own ref, because_trackssees the ledger). Three pull requests, four records, four index entries — and a board that reconciles "the PR's own record" with "the PR nested under its work item" by hand (issue-302). The owner's review of this spec named it: one file per work item, with every pull request linked to it.
Requirements
R1 — one rule, written down and enforced
User story: As the owner of the-loop, I want every attribute of a work item to have one home chosen by a stated rule, so that the next attribute lands where it belongs without a review argument.
- The system SHALL classify every attribute it records about a work item by kind (pointer, human decision, remote entity, operator ledger, machine handle, derived) and SHALL state, in one place, which file each kind is written to.
- WHEN an attribute is a remote entity or a human decision about the work item THEN it SHALL be recorded in the work item's checked-in state file.
- WHEN an attribute is a machine handle THEN it SHALL be recorded only in a file that is never tracked.
- WHEN an attribute is what this operator's deployment was told about the work item or has already seen (armed, roster, seen comments, closure) THEN it SHALL be recorded in the operator's portable record.
- A parity test SHALL fail when a state file gains a top-level key that no classification names, in the same way
GENERATED_PATHSfails an unclassified path. - WHERE an attribute is derived, the derivation SHALL be stated and the value SHALL be rebuilt from its sources rather than maintained by hand (decision-047).
R2 — a work item's remote entities travel with the work item
User story: As an operator moving a deployment to another machine, I want the new machine to know which pull requests deliver each work item and which thread carries it, so that nothing is re-derived and no second thread is opened.
- WHEN a pull request delivers a work item — recorded by the daemon on the first event that routes, or by the session that opened it — THEN the system SHALL record it in the work item's checked-in state file: its ref, its repository, its number, its URL, its inner-loop state path and its upstream state (
open,merged,closed). - WHEN a pull request merges or closes THEN its entry's upstream state SHALL be updated in that file by the daemon's one close path.
- WHEN the-loop opens a channel thread for a work item THEN the binding (channel, thread, permalink, when and how it opened) SHALL be recorded in that work item's portable record, because the thread is an entity of the operator's workspace and its identifiers SHALL NOT be written into the work item's repository (R7.3).
- Each remote entity SHALL be recorded once: any other file that refers to it SHALL carry its identity only (a ref, a thread ts), never a second copy of its record.
R3 — no machine handle in a tracked file
User story: As a repository owner, I want no harness session id, tmux name or absolute path committed to my repository, so that a public spec directory leaks nothing about the operator's machine.
- The system SHALL NOT write a
sessionblock intowork-item-state.json, nor into any inner-loop state file. - WHEN a node runs with
session: inheritTHEN the runtime SHALL resolve the bound session from the machine-local session record for that work item (or that PR's endpoint) and SHALL fall back tofresh-with-artifactswhen none is live — including in CI, where no local record exists. - WHEN a state file written before this change carries a
sessionblock THEN the system SHALL ignore it on read and drop it on the next save; it SHALL NOT be used to resume a conversation (a checked-in id may be anyone's).
R4 — the frozen selection is recorded once, in the work item's file
User story: As a session reading work-item-state.json, I want it to say everything a human froze about this work item, so that the daemon and I read the same record.
- WHEN an authorized reply answers
phase-selectionTHENsessionPerPr,modelandeffortSHALL be recorded inwork-item-state.jsonbesideskips,optIns,surfaceandloop— as the chosen values,""meaning "no choice", exactly as the portable copy distinguishes today. - The system SHALL NOT write a
graphsection into the portable record. - WHEN the dispatcher routes a pull-request event, or launches a session, THEN it SHALL read the work item's choices from
work-item-state.jsonin the checkout the work item's session record names; an absent file, an absent key or a value outside the vocabulary SHALL resolve to the operator's configured default, as an unreadable portable record does today. - WHEN a work item was frozen before this change — its state file lacks the keys and its portable record carries
graph— THEN the dispatcher SHALL read the portable copy, so a work item in flight keeps its routing across the upgrade; the copy SHALL NOT be rewritten. - The rendered
nodes[]view SHALL NOT be recorded; whoever needs it (the control plane) SHALL derive it from the compiled graph plusskipsandoptIns, through the verb that already renders it (the-loop check).
R5 — one local file per work item holds every handle
User story: As an operator debugging "why did nothing happen?", I want every handle this machine holds for a work item in one file, so that I open one file, not three.
- The machine-local session record SHALL be a map of sessions keyed by the ref they serve — the work item's own, and one per pull request that has (or may get) a session on this machine — each carrying only handles: harness, conversation id, cwd, tmux target, status, recent deliveries, launched model, effort and argv, plus the read cursor of every channel thread bound to the work item (what this deployment last mirrored). It SHALL carry nothing else about a pull request — no URL, repository, number or upstream state: the ref is the join key to the work item's checked-in file, which is where the pull request is recorded (R2.1).
- The system SHALL keep one local file per work item, not one for the machine: the registry's file-per-item property is what lets concurrent sessions write without a shared lock and what lets
cleanuprelease one item without touching another. - WHEN an inbound channel message names a thread THEN the reader SHALL resolve the work item from the portable records' bindings (R2.3) and read the cursor from the work item's local record;
channels/slack.jsonSHALL keep only state that belongs to no work item (pending, the per-channel kickoff cursors). - WHEN a
channels/slack.jsonwritten before this change still binds a thread that no portable record binds THEN the reader SHALL honour that binding, and the next write for that work item SHALL record it in the new place; nothing SHALL open a second thread for a work item that already has one. - The local file SHALL keep its location under
<state.root>/local/: the daemon's "which work items have a session on this machine" is a scan of that directory, and a record inside each checkout would be unfindable for a session registered in a checkout outsiderouting.workspace.rootand would need a.gitignorerule in every consuming repository to keep a session id out of git.
R6 — machine-wide daemon files are named by the rule and left alone
- The classification SHALL name
poll.pid,gh-webhook.pid,slack-listener.pid,poll-status.json, the logs,self-diagnosis.json,model-verdicts.jsonand the standing-session records as machine-wide, not per work item, and SHALL NOT move or merge them (decision-076 for the pidfile/heartbeat pair).
R7 — the security boundaries of each file are kept
control,collaboratorsandendedSHALL stay in the operator's portable record and SHALL NOT move into the work item's repository: a file on a work item's branch is proposable by anyone who can open a pull request, and these three decide whether an unattended agent runs, whose comments it reads, and whether the item is over.- WHEN
cleanupremoves a work item's checkout THEN everything the daemon or a control plane still needs about the item afterwards (control,poll,ended,collaborators, the channel bindings) SHALL still be readable — which is the second reason those stay outside the checkout. - The system SHALL NOT write a Slack channel id, thread ts, member id or workspace permalink into the work item's repository.
R8 — no migration step; old shapes are read, new shapes are written
- WHEN a file of any of the four shapes written before this change is read THEN the system SHALL read it as it did, and SHALL write the new shape on that file's next save; no command SHALL rewrite state files in bulk.
/the-loop:upgrade-the-loopSHALL report the retired locations (sessionin a state file,graphin a portable record,conversations/threads/cursorsinchannels/slack.json) as no longer written and SHALL NOT delete them.portable/index.json'ssectionslist SHALL stop naminggraphand SHALL name the channel-binding section.
R10 — one portable record per work item
User story: As an operator reading portable/, I want one file per work item that names every pull request delivering it, so that the directory answers "what is tracked" without a join.
- The system SHALL keep exactly one portable record per work item. A pull request that delivers a tracked work item SHALL NOT have a portable record of its own: its poll ledger (
seenComments,commentAttempts,lastPolledAt) SHALL be kept under the owning record, keyed by the PR's ref. - WHEN the poller lists a pull request THEN it SHALL resolve the work item the PR delivers before writing any ledger — through this machine's session records, then through the portable records' pull-request maps, then through the router's own linkage rules — and SHALL write under the owner it finds.
- WHEN a pull request delivers no tracked work item — it is a work item of its own: a review (issue-279), or a labelled PR that closes no issue — THEN it SHALL keep its own record, because it is the work item.
- WHEN a pull request merges or closes THEN the system SHALL NOT stamp
endedon a record for the PR: it SHALL record the PR's upstream state in the work item's checked-in file (R2.2) and SHALL drop the PR's nested ledger from the owner's record, aspollis dropped for an ended work item today. - WHEN a control keyword or a collaborator grant is typed on a pull request's thread THEN it SHALL be recorded on the owning work item's record — the router's target already resolves the owner first; this makes the rule explicit.
- WHEN a record written before this change exists for a pull request that a work item's record now lists THEN the daemon SHALL read the PR's ledger from the old record until the owner's record carries it, SHALL then stop reading the old record, and SHALL NOT delete it;
upgrade-the-loopSHALL report it. index.jsonSHALL list one entry per work item and SHALL name, per entry, the pull requests its record links.
R9 — the change is legible
docs/cli/state.mdSHALL describe every file by the classification in R1, attribute by attribute, and the parity test between that page and the code SHALL cover the attribute table.- The affected capability docs SHALL be updated in the same PR:
cli,process-graph,interactive-sessions,channels,control-plane,webhook-triggers,spec-workflow. - The rule SHALL be recorded as a decision that supersedes the file-level classification of decision-046 with an attribute-level one, naming the alternative the owner proposed (everything in
work-item-state.json) and why R7 refuses part of it. - This is a breaking change to the shape of every state file and SHALL be committed as such (
!).
Non-functional requirements
- No new network calls. Every read this change adds is a file read: the dispatcher reads one state file from a checkout it already knows; the channel reader scans the portable records (tens of small files) or an index built from them at start.
- No new contention. The state file keeps its
work-item-state.lock; the portable record keeps read-modify-write per section; the local record keeps file-per-item. - The
.gitignorerecipe stays three lines. Nothing new is tracked, nothing tracked becomes local.
Security considerations
- Actors & trust. Untrusted: anyone who can open a pull request against the origin repository (they can propose a change to
work-item-state.jsonon a branch); Slack members in the operator's channel; webhook payloads. Semi-trusted: the agent's own session, which writeswork-item-state.jsonand is the subject of the gates it records. Trusted: the operator, whosestate.rootthe daemon writes. - Trust boundaries & data. Two boundaries move. (1)
sessionPerPr,modelandeffortare read by the daemon from the agent-writable state file instead of the operator's record — the dispatcher already treats the portable record as agent-writable and re-validates on the way in (dispatcher.py:_tmux_for), andskips/repos— a bigger blast radius — already live in the state file. (2) A harness session id leaves the repository (R3). Data: no secret is stored; a session id and a Slack workspace identifier are the two things that must not be committed to a repository, and both end up in files that are never tracked or in the operator's own record. - Abuse cases (EARS):
- WHEN
work-item-state.jsonnames amodeloreffortthe operator did not declare, or one this machine's verdict cache refuses, THEN the daemon SHALL launch on the harness's own arguments and record the refusal — a choice can only ever be one the operator offered. - WHEN
work-item-state.jsonnames asessionPerProutsidenever | cross-repository | alwaysTHEN the daemon SHALL route by the operator's default. - WHEN
work-item-state.jsonlists a pull request THEN that entry alone SHALL NOT spawn a session or route an event: a session is spawned only for an event on that pull request that passed the ingress's own authorization, exactly as today'slink-prentry behaves. - WHEN a state file carries a legacy
sessionblock THEN the runtime SHALL NOT resume that conversation (R3.3). - WHEN a local record is copied to another machine THEN the duplicate-spawn guard behaves as documented today — unchanged, and the reason the file stays local.
- WHEN a portable record's channel binding names a thread the bot is not a member of THEN a post SHALL fail closed as a
ChannelErrorand SHALL NOT open a second root.
- WHEN
- Fail closed. An absent or unreadable state file, portable section or local record resolves to the operator's defaults and never to a spawn; a thread with no binding is
unmappedand dropped, as today.
Out of scope
- Deriving
controlfrom the ticket thread instead of recording it. The keyword comments are on the ticket, so it is derivable in principle; it is a separate question about API cost and edited comments, and decision-040 chose the record. - Merging the two tracked files into one. R7 says why the operator's record and the repository's file must stay two files; § Trade-offs in
design.mdsays what the owner's literal proposal would cost. - The standing-session records and
pending. Neither belongs to a work item. - Renaming
<state.root>/local/<slug>.json. The ticket'swork-item-local-state.jsonis what that file already is, one per work item; a rename buys nothing and costs a legacy reader.
Open questions
- Does the owner accept that the daemon reads
model/effort/sessionPerPrfrom the agent-writable state file (R4.3), bounded as in abuse cases 1–2? The alternative keeps a derived copy in the portable record for checkout-free readers, which is the duplication the ticket objects to. Raised on the ticket with this document. - One local file per work item, or one for the machine? The ticket's wording admits both; R5.2 picks per work item and says why. Raised on the ticket.
Review comments
Appended by the-loop's
record-feedbackhook when a human gate approves with comments (issue-109). Append-only and attributed: an approval never silently discards a reviewer's suggestions, and the feedback travels with the document it concerns rather than living in a side-channel tracker.