CLI Agent Spec

CLI Agent Spec starts from a premise worth stating plainly: your CLI works for humans, and for an agent it hangs, retries a half-completed deploy, or returns exit 0 with unparseable text — and you would never know, because nobody files a bug on behalf of a robot. It catalogs 74 such failure modes across seven parts (ecosystem/runtime, execution, security, output, environment, errors, observability), each scored for severity, frequency, detectability and — unusually — the token, time and context cost borne by the caller. Against those it sets 158 requirements. It is MIT, at v1.6, and it is the work of one person.

• Exit codes carry a machine-readable promise, not just a number - fourteen reserved codes, 0 through 13, each declaring retryable: true/false and side_effects: "none" | "partial" | "complete". Exit 11 (RATE_LIMITED) is safe to retry; exit 3 (PARTIAL_FAILURE) means inspect state first. This is the single most valuable idea in the document, because “exit 1” is where every blind agent retry begins, and no existing convention distinguishes nothing happened from half of it happened.
• Exit 9 is PAYMENT_REQUIRED - the table reserves a code for a paywalled command and its description names x402. A CLI is the last place most people expect to meet machine-payable metering, which is precisely why it is a signal: the pricing layer is being designed into the base contracts of the agent stack rather than bolted on later.
• One envelope, invariant keys - every command wraps output in {ok, data, error, warnings, meta} with the same keys always present, so an agent never parses free text to find out whether something worked. Five JSON Schema documents ship the types — exit codes, response envelope, tool manifest, dispatch request, error details — which makes this the rare specification at this stage of life with a fetchable artifact rather than a promise of one.
• Discovery in one call instead of N - tool manifest --output json returns the whole command tree: subcommands, flags, types, examples, exit-code map. It is the same instinct behind MCP tool listing and ARD, applied to a binary: stop making the agent crawl --help and pay for it in context.
• Three tiers decide who does the work - F (78 requirements, framework-automatic, authors get them free), C (29, declared by the command author at registration), O (50, opt-in per application). The split is the practical part of the document: it tells a framework maintainer and a tool author which half of the list is theirs.
• The evaluation layer is the real asset - the repository scores 12 existing frameworks (argparse, Click, Cobra, Clap, Typer, Commander.js and others) against the mapped failure modes, with no framework exceeding 59%, and audits fourteen real CLIs — gh, resend, temporal, shopify, neon, firecrawl and more — with a repeatable skill. That is a rubric applied to a market, which is a more useful contribution than the requirement list itself.
• The prose and the shipped schema disagree about the exit codes - the README’s worked example says exit 11 is CONFLICT and safe to retry, and exit 6 is PARTIAL_FAILURE. exit-code.json, in the same repository, enumerates 11 as RATE_LIMITED, 6 as CONFLICT and 3 as PARTIAL_FAILURE. The schema is the artifact a generator consumes, so build against it and not the narrative. The counts drift the same way: “74 failure modes” in the headline against “71 currently mapped” in the framework matrix, and “158 requirements” against a tier table summing to 157 — which matters because the framework comparison percentages are computed on that denominator.
• The benchmark argues against its own headline - benchmark/ runs a good-CLI/bad-CLI differential against a fixed model at temperature 0, over five scenarios. Three result files from the same day give total token deltas of -24.7%, -34.4% and -47.0%, and scenario s1 swings -4.9%, +20.3%, +134.7% across those identical runs. The methodology says run three times and take the median; nothing in the repository aggregates them. Temperature 0 does not make an agent loop deterministic, so the token-savings number is noise at n=1. What does hold across all three runs is correctness — good mode 4/5, 5/5, 5/5 against bad mode 2/5, 4/5, 2/5. The stable result is that a compliant CLI gets the right answer more often, and it is buried under the unstable one.
• No body, no RFC 2119, no adopters yet - six stars, no forks, a single maintainer, created 14 March 2026 and last pushed 7 August 2026. It calls itself a living document and uses no normative keyword language, so “158 requirements” are requirements by assertion. There is no governance structure and no independent implementation.

Take it as a well-argued problem statement with a schema attached rather than a standard anything conforms to. The failure catalog is the part that survives regardless of what happens to the project — it is the most complete public inventory of how CLIs break under machine callers, and any team shipping a tool that agents invoke can read it as a test plan tomorrow. Where OpenCLI describes what a CLI offers, this describes how it must behave when the caller cannot read a man page, retries on impulse, and pays for every token it wastes finding out.