Guide

Last updated: October 29, 2026

Keelmarker keeps a list of things you promise about your software (requirements), links tests to them, and answers one question: is every promise backed by a passing test, or by a named person who accepted the risk? The answer is the gate: pass or blocked, with the reasons. It is the same answer for people, scripts and AI agents.

• Get started
• Reading the gate
• Sending test results
• Using it from an AI agent
• Decisions only a person makes
• Findings and signal rate
• Extra gate rules and shadow mode
• Using the gate in CI
• Evidence from other tools
• Tidying old tests
• Command and tool reference

Get started

1. Sign in at /app/. Access is by invitation while Keelmarker is in beta; the person who invited you sends a link.
2. Create a product (a short code and a name) and one component for each part of it you want gated on its own.
3. Add requirements, one sentence each, saying what must be true (“Orders are saved with the right total”). A starter pack (Packs tab) adds common ones for you to review.
4. Link a test to each requirement, by the test’s name. The names are the link: renaming a test breaks the link.
5. Send results from your test runs (next sections). The gate recomputes and tells you what is missing.

Requirements nobody assigned to a component still gate, as “Unassigned”, so a pass can never skip them.

Reading the gate

Each requirement is in exactly one state:

A component is blocked when any requirement in it has no executed test, or when its tests’ pass rate is below the threshold (default 100%). The product passes only when no component is blocked. A product with no requirements is blocked: nothing has been checked.

Partly proven. A requirement with one passing and one failing test reads verified, but the gate is still blocked by the failing test. The summary and the Markdown output list such requirements as “partly proven” with the failing test names. It is only an explanation: the verdict is computed exactly as above.

Commits. Pass a commit to pin the verdict to shipped code: a test then counts only through runs reported for that commit. Use the short SHA; a full SHA shows a false “blocked”.

Stale. A stored verdict says when requirements, links, sign-offs or results changed after it was computed.

To see what blocks and how to fix it in a few lines:

keelmarker gate status --product-id PRODUCT_ID           # latest stored answer, no recompute
keelmarker gate check  --product-id PRODUCT_ID --commit-sha abc1234 --format markdown

Sending test results

Send the JUnit XML your test runner already writes. Nothing about your code is uploaded.

keelmarker results ingest --product-id PRODUCT_ID --junit junit.xml --commit-sha abc1234

Tests are matched by name. Names that match no test in Keelmarker are listed in the reply so you can add or link them.

Using it from an AI agent

Create an agent token in Settings (it can read and write, never sign off), then connect any MCP client that supports streamable HTTP:

claude mcp add --transport http keelmarker https://YOUR-HOST/api/v1/mcp --header "Authorization: Bearer YOUR_TOKEN"

Each call runs as an ordinary API request with the token’s own permissions. A typical task:

1. at_task_start with a title and objective, then at_promise for each thing the agent will make true.
2. Make the change, add or link tests, report results with at_report_results.
3. at_check_gate returns a compact answer: verdict, counts, what blocks, how to fix it. Pass full=true for every requirement row (large on big products). at_gate_summary reads the latest stored answer without recomputing.
4. at_task_close succeeds only when every promise is settled and the gate passes.

An agent can add and strengthen requirements, report findings and results, and read everything. It cannot sign off a risk, dismiss a finding, promote a rule or retire a test: those need a person.

Decisions only a person makes

A person is someone whose token or role carries the attest permission (a workspace owner or admin). These actions can weaken the gate, so agents are refused:

• Accept a risk (Coverage tab, or keelmarker risk accept): for a requirement tests cannot prove, a named person records the criterion and the evidence. It shows as attested, never as verified, and can expire or be revoked.
• Human-only requirements. Flag a requirement “needs a person”. It blocks as awaiting sign-off until a person accepts it, even when its tests pass. Anyone who can write may set the flag; only a person can clear it.
• Dismiss or reopen a finding and promote or revoke a gate rule (below).
• Retire a test, or deprecate a requirement.

Findings and signal rate

A review or audit finding becomes a requirement with no test yet, so the gate stays blocked until a real test proves the fix (keelmarker finding ... or at_report_finding). Findings keep their source, severity and location. Their outcome is worked out from the requirement:

• Open: not yet proven.
• Fixed: a linked test passes. Nobody can claim this by hand.
• Accepted risk: a person signed it off.
• Dismissed: a person judged it wrong, a duplicate, not worth fixing or out of scope.
• Withdrawn: the requirement was deprecated by hand.

Dismissing (Findings tab, or keelmarker findings dismiss) always asks you to choose what happens to the requirement:

• Also deprecate it: the gate stops waiting for it. Usually right for a false positive or a duplicate.
• Keep gating: only the dismissal is recorded; the gate stays blocked until the requirement is fixed or deprecated. The finding shows “still blocking the gate” so nobody wonders why.

For a risk you are knowingly accepting, a recorded sign-off usually fits better than a dismissal, because it stays visible in the gate as attested. Dismissing suggests tests that could be retired but never retires anything.

Signal rate (keelmarker findings stats, or at_finding_stats) shows per source and severity how many resolved findings were fixed or accepted rather than dismissed, and the median time to resolve. With fewer than 5 resolved findings it says “low sample”.

Extra gate rules and shadow mode

A product can add built-in rules on top of the standard gate: pass_rate_threshold, no_unassigned_requirements, max_attested. An agent or script can propose one, but every new rule starts in shadow mode: the gate records what the rule would have decided and never lets it change the verdict. Read the track record in the Rules tab (gates watched, how many it would have decided differently). When you trust it, a person promotes it and from then on it can block. A person can revoke it at any time. Rules only ever add blocks; they never turn a blocked gate into a pass.

Using the gate in CI

The command exits 1 when the gate is blocked, so a CI job is your status check. Nothing is posted to your repository by Keelmarker, and no repository token is stored here.

- run: echo "SHA=$(git rev-parse --short HEAD)" >> "$GITHUB_ENV"
- run: pytest --junitxml=junit.xml || true
- run: keelmarker results ingest --product-id "$PRODUCT_ID" --junit junit.xml --commit-sha "$SHA"
- run: keelmarker gate check --product-id "$PRODUCT_ID" --commit-sha "$SHA" --format markdown | tee -a "$GITHUB_STEP_SUMMARY"

Set KEELMARKER_BASE_URL and KEELMARKER_API_TOKEN as variables and secrets. Make the job a required check in branch protection to block merging on a blocked gate. --format json gives the same answer for scripts.

Evidence from other tools

Browser runners, scanners, load tests and other reviewers can send evidence about your product. Evidence is shown beside the requirements it bears on, but it never counts as proof: only tests and people’s sign-offs move the gate.

keelmarker evidence template > record.json      # a record to fill in
keelmarker evidence ingest record.json --product-id PRODUCT_ID --spec-id SPEC_ID --note "what this shows"

A good record has an evidence_id (yours; sending it again is harmless), provider.id, operation, subject.name and predicate_type, and for a run of the app a result with status (passed, failed or inconclusive) and observed_at, plus commit_sha, confidence and limitations. A thin record is still accepted; the reply lists warnings saying what to add.

Tidying old tests

Tests that no requirement uses and that never ran can be listed (keelmarker test retire-unlinked --product-id PRODUCT_ID is a dry run) and retired by a person from an explicit list of ids. Each is checked again when you retire it. There is no “retire everything”.

Command and tool reference

Questions or something unclear? Use the contact form.