Operate

Use this page to verify releases, run the review service, sync outcomes, generate governance reports, and backfill existing comments.

Trust · Compliance · Provenance

Verify a release

All Bubo releases are signed with cosign and shipped with SLSA provenance, so you can prove an artifact came from Bubo’s release workflow and wasn’t tampered with. To validate a release:

1. Install cosign and slsa-verifier.
2. Download the artifact from the releases page with its attestations — each artifact has a matching <file>.bundle (cosign), and the release carries one multiple.intoto.jsonl (SLSA provenance covering every artifact).
3. Run both checks:

# point this at the file you downloaded (wheel, sdist, deploy bundle, or the SBOM)
ARTIFACT=bubo-1.2.3-py3-none-any.whl
 
# 1. Signature — signed keyless by the release workflow via Sigstore + GitHub OIDC.
cosign verify-blob \
  --bundle "$ARTIFACT.bundle" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  --certificate-identity-regexp "^https://github.com/mountainowl/bubo/\.github/workflows/release\.yml@refs/tags/v" \
  "$ARTIFACT"
 
# 2. Provenance — built by that workflow at a real tag (SLSA Build L3).
slsa-verifier verify-artifact "$ARTIFACT" \
  --provenance-path multiple.intoto.jsonl \
  --source-uri github.com/mountainowl/bubo

The same two commands verify the wheel, sdist, deploy bundle, and the SBOM — printing Verified OK for an authentic artifact.

Install

Install Bubo on the host with uv tool install bubo, then bubo init — the per-stack steps live on the recipes page.

Review service

The portable service is the only supported long-running review scheduler. It owns the poll schedule, subscription monitor, and periodic outcome sync:

bubo-poller service start
bubo-poller service start --foreground
bubo-poller service status
bubo-poller service stop

start detaches and writes service output to var/log/service.log; --foreground keeps the process attached for an external supervisor. The service waits for the monitor’s first durable heartbeat before its first poll. [poller].interval_seconds controls the poll cadence, and [poller].outcome_sync_interval_seconds controls the service-owned outcome-sync cadence.

The service is not an OS boot service. Use your platform’s supervisor to run bubo-poller service start --foreground and restart it after process or machine failure.

Migrate from cron or systemd timers

Before starting the service, manually disable and delete every old Bubo cron, timer, and systemd scheduler entry. Running an old poller alongside the service is unsupported.

The previously shipped generated scheduler templates are no longer used. On upgrade, bubo init reports any it finds below $BUBO_ROOT/deploy/templates/ but preserves them. Manually delete those generated templates after disabling their installed entries, reload your platform supervisor if needed, then start the Bubo service. Retain an external supervisor only for boot startup and restart; it must invoke the foreground service command above.

Health check

bubo doctor runs these non-mutating checks:

Pass --no-agent-config to skip generated agent profiles and manage them yourself. Doctor returns 0 on a full pass and non-zero on any failure.

Troubleshooting

Host and infrastructure issues — sandbox failures, AppArmor / user-namespace errors, and “no findings” results that are not a genuine clean review — live on a dedicated page: Troubleshooting.

Posting & thread resolution

The poller drives both GitLab and GitHub, but each platform posts inline comments and reads thread resolution differently.

GitHub

• Posting goes through the GitHub REST API — a pull-request review comment anchored to the diff line. No SCM MCP server or extra CLI is involved.
• Thread resolution (service-owned outcome sync) reads GitHub’s GraphQL reviewThreads (isResolved), since REST doesn’t expose per-thread state. If GraphQL is unavailable, it falls back to a resolution-blind REST path that still records posted/deleted/replied transitions.

GitLab

• Posting goes through the GitLab REST API — a merge-request discussion anchored to the diff line. No SCM MCP server or extra CLI is involved.
• Thread resolution needs no GraphQL — GitLab’s REST discussions API already carries the resolved / resolvable flags, so outcome sync reads accurate state (including merged_unresolved) straight from REST.

Reconcile Bubo findings fixed in later commits

Set reconcile_fixed_findings = true to let a successful non-dry-run rescan close a Bubo-owned finding that later commits have fixed. This applies to blocking and non-blocking findings.

Bubo requires a separate high-confidence FIXED proof with current-checkout path and line evidence. It posts Verified fixed as of <full SHA> and resolves only its own GitLab or GitHub thread.

Developer-resolved or deleted threads are no-ops. Uncertain proof, stale heads, duplicate threads, provider races, or a still-valid finding are left open; Bubo does not post a no-findings acknowledgement in those cases.

New commits on a reviewed change

[review].rereview decides what a new head gets after a completed review: "full" (default) reviews the whole change again, "incremental" reviews only the commits since the last completed review and runs the reconciliation above for earlier findings even if reconcile_fixed_findings is off, and "off" reviews only the first commit. If the earlier reviewed commit is no longer an ancestor of the head (force-push or rebase), an incremental review falls back to a full one.

In every mode:

• One review per change at a time. A head pushed while a review is queued or running waits for the next poll.
• Newest commit wins. If the head moves while the agent is reviewing, the run is recorded superseded and posts nothing, so findings are never anchored to line numbers from an older commit. The new head is reviewed by a later poll.
• No repeats. A finding with the same file and canonical text as an open Bubo thread on the change is skipped, even if the line moved or the in-voice comment is worded differently.

[review].settle_seconds debounces bursts of pushes: a head is reviewed only once it has been unchanged that long, measured from the first poll that saw it, so the effective delay rounds up to a poll interval.

Git checkouts

Each project keeps one bare mirror under $BUBO_BASE_DIR/work/_mirrors/. A review takes an exclusive lock on the mirror, fetches only the target branch and the change ref (with the same per-call auth header; no credential is written to any git config), and adds a detached git worktree for the review, which is removed afterwards. The first review of a project downloads the repository once; later reviews download only new objects. If any mirror step fails, Bubo deletes the mirror and falls back to a full clone for that review.

A worktree’s .git is a file pointing into the mirror, so the review agent must be able to read $BUBO_BASE_DIR/work/_mirrors/. The default Codex read-only sandbox and Claude Code allow reads outside the working directory; a custom sandbox that limits reads to the checkout must also allow the mirror directory.

Outcome sync

The review service grades posted findings against current SCM state at [poller].outcome_sync_interval_seconds. Outcome-sync failures are logged and do not stop review polling.

It records whether each finding was resolved, left unresolved after merge, deleted, replied to, marked disputed, marked false-positive, or marked duplicate — feeding the llm_review.findings{status=…} counter described in telemetry.md.

Reply classification

A thread can be resolved because the developer fixed the finding or because they rejected it (“working as intended”, “not a blocker”). The resolved flag alone can’t tell them apart, which would overcount the reviewer’s precision. So when a finding’s discussion has a developer reply and no explicit dispute marker, outcome sync asks an LLM to read the bot’s finding plus the reply and decide whether the developer agrees, merely accepted it, rejected it, or is unclear. A rejection sets disputed (and false_positive when the reply says the finding is factually wrong). Explicit agreement (“good catch”, “you’re right”, “fixed”) records developer_disposition = agrees; a plain “will look at it” stays unknown, because it doesn’t confirm the finding was correct.

• Model-agnostic. Classification runs the same agent you set in [agents].reviewer_command (default codex exec --profile bubo; a custom command like claude -p is reused as-is). The review contract lives in the prompt, not the command, so the same command makes the accept/reject call here.
• Classified once. Each finding is classified one time (tracked by the reply_classified column) to bound LLM cost across the hourly sync. A reply that arrives after the first classification isn’t re-graded.
• Cold-start safe. At most a few dozen findings are classified per run, so the first sync after upgrading — when the whole backlog of resolved-with-reply findings is unclassified — won’t fire hundreds of agent calls at once. The backlog drains over later runs.
• Fail-safe. A transient classifier failure (timeout, non-zero exit) leaves the finding unclassified and retries on a later sync; unparseable output degrades to “unclear”. Either way the sync keeps going.

Prefer the explicit path? Tag a reply with [llm-review:disputed] / [llm-review:false-positive] — an explicit marker short-circuits the LLM call.

Governance report

bubo report assembles an auditable governance report from the metrics already in SQLite: review counts (with a no-findings acknowledgements rollup), a provenance breakdown (counts by band/source), the accept-vs-dispute rate, a noise trend (daily false-positives), a bug-catch ROI proxy, review latency (p50/p95/max/avg seconds), per-category dispute rates, token/cost rollups, policy-decision stats (from the Phase 2 governance_decisions), and a per-change audit trail. It’s the compliance-facing companion to the provenance + governance-decision capture in configuration.md, “Governance & provenance”.

It is strictly read-only — it queries existing state, never mutates it, so it’s safe to run from a monitoring cron alongside bubo doctor. The data comes from your own SQLite and never leaves your infrastructure.

# Full nested report as JSON (the default).
bubo report
 
# A week of the per-change audit trail as CSV.
bubo report --since-hours 168 --format csv > audit.csv

Flags

JSON vs CSV

• JSON (default) is the full nested report. Sections: meta, reviews (with a nested acknowledgements rollup), provenance, outcomes, noise_trend, roi, latency, dispute_classes, policy_decisions, and audit. Scalar rollups (review counts, accept/dispute rate, ROI proxy, latency, token/cost) are JSON-only — they have no tabular form.
• CSV is a single tabular section, picked with --section. The default is the per-change audit trail; --section noise_trend emits the daily false-positive trend, and --section dispute_classes the per-category dispute rates (CLI path: raw stats, no would_suppress). Use CSV for spreadsheet/BI import of the row-shaped sections.

MCP tool

The same report is available to MCP-capable chat clients (Codex, Claude Desktop, Cline) as get_governance_report(since_hours, since, until, project). It returns the same nested JSON as bubo report --format json, so a governance analyst can pull the report into a chat session without shell access.

A companion get_dispute_classes(project) returns just the per-category dispute stats (cumulative across all recorded outcomes, so it takes no window argument). Unlike the CLI path, it reads your real [review].dispute_suppress_threshold / dispute_suppress_min_samples and adds a would_suppress flag per category, so an analyst can see which classes suppression would drop if enabled.

Policy-decision stats are populated only when Phase 2 governance is enabled ([governance].policy_mode not off). When it’s off, the policy_decisions section reports available: false rather than empty counts, so an auditor can tell “no decisions recorded” apart from “governance decisions were never turned on”.

Backfill — one-time recovery, not a scheduled job

The backfill commands import bot comments that already exist on the SCM into local SQLite. Reach for them when:

• You just deployed against a project where the bot has posted before from another install.
• You reset var/state/reviewer.sqlite (test, rebuild, host migration) and need the per-finding metrics to reflect history, not just go-forward.

They are deliberately outside the review service. Each run scans every MR/PR updated since the cutoff — suitable for one-time recovery, inefficient as a frequent poll task. Run them once, then let service-owned outcome sync take over for go-forward grading.

# GitLab
bubo-poller --backfill-gitlab-bot-comments-since 2026-05-25T00:00:00Z
 
# GitHub
BUBO_PROVIDER=github bubo-poller --backfill-github-bot-comments-since 2026-05-25T00:00:00Z
 
# Then grade the imported rows once:

Both backfill commands are idempotent — a comment already in SQLite is upserted, not duplicated — so re-running with a different cutoff is safe.

Dashboard management summaries

The dashboard rolls up recorded production spend, audit evidence coverage, policy activity, and discussion outcomes, with links to supporting records. Audit rows and CSV include dry_run and cost_recorded; missing cost is not a free review.

Audit coverage matches distinct production runs started in the period to policy records by run ID, even if decisions were written outside that period. Known provenance is counted separately. Gaps can include unfinished work or disabled controls; these are evidence-coverage fractions, not compliance certification. Policy activity uses its own decision-creation-time cohort. Advisory actions are not enforced merge gates. The policy report’s availability flag describes the existence of its storage table, not whether policy was enabled for every review in the period.

The discussion proxy counts resolved findings without recorded disputes or false-positive flags among observed findings updated in the period. It does not prove acceptance or a code fix. The legacy report ROI proxy mixes finding-update and run-start cohorts and includes dry runs in cost and success totals. Its accepted-per-dollar value is not measured financial return or directly comparable to production-only dashboard figures.

The optional planning calculator uses your minutes-saved and hourly-cost assumptions, recorded production spend, and other costs. It shows a break-even time per successful production review; missing recorded costs can understate investment. It is a scenario, not measured savings.