Requirements: Integration tests and OpenAPI specs
Source of truth: GitHub issue #11 is the canonical requirements input for this work item. This file distills it into reviewable, testable requirements. Design and the task DAG live in
design.mdandtasks.md.
Introduction
the-loop should make integration-test coverage queryable by a coding-agent harness and make API contracts declarative (spec-first). This work item codifies both as loop conventions, driven by config, with a CLI command that extracts and tabulates the tested scenarios.
Requirements
R1 — Gherkin scenario docstrings on integration tests
User story: As a harness (or reviewer), I want every integration test to state its scenario in a structured docstring, so that coverage is self-describing and traceable.
- Every integration test SHALL have the scenario being tested listed in Gherkin-like syntax (
Feature:/Scenario:/Given-When-Then) as a docstring (or the language's nearest comment equivalent). - IF a test is linked to a
requirements.mdTHEN the docstring SHALL include aRequirement:link to thatrequirements.md(with a section anchor where possible). - The convention SHALL be language-agnostic (Python docstrings, JS/TS block comments, Go comments) and SHALL be driven by
config.testing.
R2 — Queryable, tabular scenario view
User story: As a coding-agent harness (Claude/Cursor), I want to query all the scenarios being tested, so that I can present a tabular view without running the suite.
- The
the-loopCLI SHALL provide ascenarioscommand that scans the configured integration-test globs (testing.integrationTestGlobs, with built-in defaults) and extracts every Gherkin scenario. - WHEN invoked THEN it SHALL present a tabular view (Feature, Scenario, Requirement, file:line) as an aligned table, a Markdown table, or JSON (
--format). - The command SHALL keep the CLI's zero-runtime-dependency guarantee (stdlib only; PyYAML remains optional for config defaults).
R3 — RESTful API specs in OpenAPI
- All API specs for RESTful APIs SHALL be authored in the
specs/openapi/folder in the OpenAPI spec format (apiSpecs.rest, defaultopenapi-3.1). - Documentation SHALL be automatically generatable from the OpenAPI specs (
apiSpecs.rest.generateDocs/docsTool); generated docs are never hand-written.
R4 — GraphQL best practice
- the-loop SHALL define the GraphQL equivalent of R3 (the issue's TODO): an SDL-first, checked-in schema under
specs/graphql/with generated documentation and a CI breaking-change check (apiSpecs.graphql). Recorded indecision-014.
Non-functional requirements
- New config keys validate against
.the-loop/config.schema.json. - CLI quality gates stay green: ruff, pyright, pytest, markdownlint.
Out of scope (this work item)
- Enforcing the docstring convention mechanically in the test runner (review-enforced for now).
- Wiring OpenAPI/SDL lint & docs-generation pipelines into the-loop's own CI (the-loop has no HTTP API yet).