AgentCI

Overview

An agent that only learns whether CI passes after pushing burns a round-trip, some Actions minutes, and a commit on every attempt. AgentCI closes that loop locally:

edit → run CI locally → structured failure → fix → rerun what failed → full CI → push

It parses your existing workflow files, builds the job DAG, and executes each job the same way GitHub would: natively on this host, or in a real Linux container via act, then hands back a verdict, structured failures, and a log per job.

What it is

• A reader of .github/workflows/*.yml. That file is the only source of truth; nothing about a job's behavior is redefined elsewhere.
• Two execution engines: native (shell steps on this host) and act (a real Linux container, real uses: actions included).
• A verdict (FULL_CI_PASS, PARTIAL_PASS, or FAIL) that tells you honestly whether it is safe to push.

What it is not

Quick start

c3_ci(action='inspect')
c3_ci(action='run')
c3_ci(action='run', job='lint')
c3_ci(action='failures')
c3_ci(action='rerun')
c3_ci(action='logs', job='CI::lint')
c3_ci(action='runs')

c3 ci inspect
c3 ci run --job lint --allow-foreign
c3 ci rerun
c3 ci failures
c3 ci logs "CI::lint" --tail 200
c3 ci runs
c3 ci inspect --json

A real c3 ci inspect against this repository, from Windows:

Workflows in . (host Windows):
  CI  [pull_request, push, workflow_dispatch]  jobs=4
  Release  [push, workflow_dispatch]  jobs=4

Job graph (dependency order):
  CI::lint                                   runnable (container)
  CI::test (macos-latest, 3.12)              other-OS (macos-latest)
  CI::test (ubuntu-latest, 3.12)             runnable (container)
  CI::test (windows-latest, 3.12)            runnable here
  CI::build                                  runnable (container)  needs=lint,jsx,test
  Release::verify-ci                         UNSUPPORTED
  Release::publish-pypi                      runnable (container)  needs=build

Runnable here: 12 of 16  (3 native, 9 container)
  engines: native + act (act version 0.2.89)
  other-OS (3): these target a runner no available engine can reproduce. …
  unsupported (1):
    Release::verify-ci: step 1 env GH_TOKEN has unresolved expression(s): github.token

That last line is the point: an unresolved ${{ secrets.X }} or github.token blocks rather than silently becoming an empty string, because a job that quietly passes locally on a blank secret and fails in real CI is worse than a job that refuses.

Verdicts & job status

FULL_CI_PASS is the only verdict that means "safe to push." Everything else is PARTIAL_PASS, and partial means something did not run here — a different OS, an unsupported construct, or a selection you made yourself.

c3 ci run exits non-zero for anything but FULL_CI_PASS, so a pre-push hook or script can gate on the exit code alone.

Job statuses

Job DAG & conditions

• Order. Jobs run in topological order of needs. A needs edge is scoped to its own workflow, so two workflows that both define a job called build stay unrelated.
• Matrix. strategy.matrix expands to one job instance per cell, including include/exclude. Each cell resolves its own runs-on.
• Identity. A job's key is <workflow>::<id>, e.g. CI::test (ubuntu-latest, 3.12). A selector accepts the key, the bare id, or a bare job name (which selects every matrix cell of it).
• Skipped on failed dependency. A job whose needs did not pass is skipped — never run, never counted as passed.

if: conditions (v2.80.0+)

Job- and step-level if: are evaluated with GitHub's own semantics, not just parsed and ignored:

• A condition naming no status function is implicitly success() && (...) — which is why if: always() is how a step runs after a failure.
• A job-level if: replaces the needs success gate, so always() and failure() run even when a dependency failed.
• A step with no if: keeps its implicit success() gate and is skipped once something earlier in the job failed.

Supported operators: == != < <= > >= && || !, parentheses, literals, and success failure always cancelled contains startsWith endsWith format join toJSON fromJSON. Supported contexts: github env matrix runner job needs steps strategy.

A job skipped by its own if: gets status skipped_if and does not cost coverage — CI would have skipped it too, so FULL_CI_PASS stays reachable. Two things still block rather than guess: a condition that cannot be parsed, and one that reads github.event_name with no event declared. There is no event locally, so declare what you are simulating:

c3 ci run --event pull_request

github.ref, ref_name and sha are read from git and always available.

Job outputs & run-time needs.* (v2.130.0+)

A workflow shaped like a changes job feeding a gate (if: needs.changes.outputs.code == 'true') into shards, then an aggregator reading needs.<job>.result, now resolves end to end:

• Every native step gets its own GITHUB_OUTPUT file (k=v and k<<EOF forms); a job's declared outputs: resolve from steps.<id>.outputs.<k>. Under act the same values come from its ::set-output:: log lines.
• An output that is declared but never written is refused, not blanked — a blank standing in for a real value is exactly the bug this exists to catch.
• needs.* and steps.* references inside run: text are deferred at DAG-build time and filled in just before each step executes; a leftover at run time still refuses the step.
• A job that publishes outputs is never served from the cache — a cache hit replays a verdict, not the values a dependent needs.
• github.event.* fields read "" when an event is declared but the field is absent from the local (nonexistent) payload — matching what GitHub itself sends for a sparse payload. Without a declared event it still blocks the native engine (act resolves it itself).

Execution engines (v2.81.0+)

Two engines, picked per job. c3 ci doctor says which are available.

engine='auto' (the default) runs a job natively when runs-on matches the host, hands Linux jobs to act when act and Docker are present, and refuses otherwise. engine='act' forces containers and fails the run if act is unavailable — it does not fall back silently, because you asked for container fidelity and would not have gotten it.

Setup

# Windows
winget install nektos.act
# macOS
brew install act
# plus a running Docker daemon

Images come from catthehacker/ubuntu:act-* (~1 GB on first pull), which mirror much of the GitHub runner's preinstalled toolchain. A bare ubuntu:24.04 does not, and a job assuming a preinstalled tool would fail locally while passing in real CI.

c3 ci doctor — a real run on this repo

Local CI execution engines:

  native    available — runs jobs whose runs-on matches Windows
  act       available — act version 0.2.89, docker 29.6.2
            runs Linux jobs in a container (catthehacker/ubuntu:act-latest), real `uses:` actions included
            a runner failure before any step runs is reported as a container-setup failure, not as 0 parsed failures

macOS jobs cannot run locally on any engine — there are no macOS containers, so a matrix containing them can never reach FULL_CI_PASS here.

Two things worth knowing

• --bind is always used. act's default copy-mode workspace arrives empty against a Windows host path, so every step then fails on missing files. Binding means the container writes into your real working tree — the same thing the native engine already does.
• Network is on by default (act's own default). Pass network='none' (c3_ci) or --network none (CLI) to cut egress.

Blockers are per engine

An unknown uses: action is fatal to the native shell and routine for act, so inspect reports both answers separately. A missing ${{ secrets.X }} or ${{ vars.X }} blocks on either engine — no engine can reproduce a job whose input does not exist.

Foreign-OS jobs

runs-on: macos-latest on a Windows or Linux box is not that job — by default it is refused and reported foreign.

c3_ci(action='run', allow_foreign=true)
c3 ci run --allow-foreign

allow_foreign runs it anyway on this host's OS, labels the result cross-os, and caps the verdict at PARTIAL_PASS — permanently, no matter how many times it passes. That is what makes the feature honest rather than merely optimistic: an indicative pass on the wrong OS is not evidence the real job would pass.

Secrets & side effects

Running real uses: actions under act means a publishing job goes from unrunnable to one command away from actually publishing. Two things stand in the way — a mechanism and a policy, and neither is a sandbox.

No secrets, ever

act reads .secrets and .env from the repository by default; C3 points both at an empty file. A publish step therefore runs and fails at authentication rather than silently succeeding with real credentials it should never have had.

The side-effect gate

Jobs using known publishing actions (pypa/gh-action-pypi-publish, softprops/action-gh-release, …) or running publishing-shaped commands (twine upload, npm publish, docker push, …) are refused unless you pass allow_side_effects=true (--allow-side-effects on the CLI).

The native engine has no isolation

Running this repository's own test job natively once genuinely uninstalled C3 mid-run, because the job's own steps included pip install -e . against the same environment the agent was running in. The native engine now refuses steps that would reconfigure this machine (pip install, npm -g, apt, brew install, and the like) unless you pass allow_host_mutation=true. Prefer engine='act' for jobs that install things — the container absorbs the mutation instead of your machine.

Required mode (v2.83.0+)

By default (mode='full') every runnable job executes. mode='required' narrows the run to the jobs a change could plausibly have broken.

c3 ci plan                     # what a change requires, and WHY, per job
c3 ci plan --base origin/main  # diff against a ref instead of the working tree
c3 ci run --required           # execute just those jobs

The planner is conservative: it reasons from changed paths, a workflow's own paths: filters, and an explicit ci.required_map in .c3/config.json. Anything it cannot reason about runs — on a repo with no filters and no map it selects everything, because guessing which jobs a change cannot affect, with no real dependency graph, is how a skipped job becomes a shipped bug.

{ "ci": { "required_map": { "unit": ["src/**"], "docs": ["docs/**", "*.md"] } } }

A narrowed run is always PARTIAL_PASS — required mode is for the inner edit loop, never the final check before a push.

A real c3 ci plan against this repo, with no required_map configured yet:

Required-mode plan (16 run / 0 skip) vs the working tree
changed files: 1
  cli/guide/credentials.html

  RUN  CI::lint
       no path filter or required_map rule covers this job — running it, because skipping on a guess risks a false pass
  …

Caching (v2.84.0+)

A job whose definition, engine and inputs are unchanged since it last passed is reused: status cached, nothing executed, and the verdict note says how many. no_cache=true (--no-cache) forces execution of every selected job regardless.

ci.required_map pays for itself twice: a job that declares its inputs is fingerprinted over only those paths, so unrelated edits stop invalidating it. Without a map, the fingerprint covers the whole tree and any edit invalidates every cached result.

actions/cache is honored for real — restore at the step, save only after the job passes, keys immutable. A job that publishes outputs (v2.130.0+) is never served from cache, since a cache hit replays a verdict, not the values a dependent needs.

c3 ci cache            # size + whether required_map narrows invalidation
c3 ci cache --clear     # drop every cached result and dependency cache

A real c3 ci cache on this repo, before any run has happened:

Cached results: 0
Dependency cache keys: 0
Location: .c3\ci\cache

A job is reused when its definition, engine and inputs are all unchanged since it last passed.
No ci.required_map: every job is fingerprinted over the whole tree, so any edit invalidates every cached result.

Run history, flakes & publishing (v2.85.0+)

c3 ci history

Per-job pass/fail counts, average duration, and flake detection: a job that both passed and failed on the identical fingerprint (same definition, same inputs) changed its mind without the world changing. That is the only local flake signal that is not a guess. Sample sizes under five executions are labelled (low n) rather than presented as a rate, and cached results are never counted as observations, because reuse is not evidence about behavior.

c3 ci publish

Posts a GitHub commit status through your existing gh authentication — no App to register, no token for C3 to hold. It refuses three things, because a status is a claim other people act on:

• a dirty tree — the status attaches to a commit, and a differing working tree means the thing that ran is not the thing being labelled;
• an unpushed commit, which has nothing to attach to;
• a PARTIAL_PASS, unless forced — GitHub's states are success/failure/pending/error and none of them honestly mean "we checked some of it," so it refuses, or on --force posts pending with the reason attached.

c3 ci publish --dry-run
c3 ci publish
c3 ci publish --force   # posts a PARTIAL_PASS as `pending`, reason included

The context is agentci/local, so nobody mistakes a laptop run for hosted CI.

c3 ci CLI reference

The human mirror of c3_ci — same handler, so the two surfaces can never drift into disagreeing about what a verdict means. Common flags on every subcommand: --path (project directory, default current), --json (machine-readable, same shape the Hub reads), --event (simulate push / pull_request for an if: that reads github.event_name).

Exit code follows the verdict: c3 ci run / rerun exit non-zero for anything but FULL_CI_PASS, so a pre-push hook can gate on $?. Run python -m cli.c3 ci <subcommand> --help from a checkout to see the live flag list for yourself.

Troubleshooting