Audit Gate Logic

---
name: audit-gate-logic
description: >
  Read-only audit of CI gate logic: silent bypass, ratchet gaming,
  required-but-not, duplicate gates. Use when "can CI be bypassed" or "why did
  a regression pass CI". Cost → audit-cicd. Rule content → audit-doctrine.
license: MIT
effort: high
---

# audit-gate-logic — Does the gate actually stop what it claims?

**Degree of freedom: MIXED** — Phases 0–2.5 `[HIGH freedom]`; Phase 3
history/probe `[LOW freedom — run exactly]` (no scratch branch unless asked).

Read-only. `audit-cicd` asks "is the pipeline fast, cheap, and safe to run."
This asks **"does the gate stop what it claims to stop."** A green check is a
claim. The expensive failures are the ones where CI passed and shouldn't have.

Present findings. Do not edit workflows, branch protection, or baselines.
Consolidation of the archaeology map → `housekeep-gates`. Rule-to-mechanical
gaps → `enhance-agent-guardrails`. Whether the *rule content* is right
(remedy, named token, Tier-D practice) → `audit-doctrine`.

## This skill vs neighbors

| Skill | Owns |
|---|---|
| **audit-gate-logic** (this) | Bypass, ratchet integrity, conflicts, required-vs-actual, archaeology map |
| `housekeep-gates` | *Executes* the map — aggregator, delete losers, prove |
| `audit-cicd` | Pipeline cost, speed, storage, runner safety |
| `workflow-quality-gate` | *Runs* the pre-release sweep — does not audit its soundness |
| `enhance-agent-guardrails` | *Installs* new gates; does not audit existing gate logic |
| `burndown-full` / `workflow-green-repo` | Execute MATCH/DONE or green the repo — ratchet holes stay here |
| `audit-codemod-safety` | Bulk-transform behavior; if CI waved a semantic bug through, that is also a finding here |
| `test-mutation` | Assertion-strength gate this map should name if coverage is the only test metric |

**Degrade honestly.** Highest-value "is it actually required" checks need
branch-protection / rulesets via the repo settings API (`gh api`), not just
YAML. If settings are invisible, audit the workflow files and say so — you
can still catch `continue-on-error` and ratchet classes, but you cannot
confirm the required-check list.

## How to reason

1. **Observe** — quote the `if:`, `continue-on-error`, required-check list, or history
2. **Interpret** — can a merge go green while the claimed check did not run or failed?
3. **Classify** — silent bypass / ratchet game / conflict / archaeology / correct-as-is
4. **Severity** — ships-broken-green = critical

## Worked example

> **Observe:** `lint` job has `continue-on-error: true` and is the only
> required status check named "Lint".
> **Interpret:** lint can fail and the required check still reports success.
> **Classify:** silent bypass (`continue-on-error`).
> **Severity:** critical — ships-broken-green.
> **Finding:** lint | continue-on-error | critical | `.github/workflows/ci.yml:24`

---

## Phase 0 — Map every gate and its enforcement  [HIGH freedom]

Inventory the full gating surface, not just the obvious jobs:

- **Workflows & jobs** (`.github/workflows/*`, or GitLab/CircleCI equivalent):
  each job, its triggers (`on:`), its `if:` conditions, path/branch filters.
- **Required status checks** (branch protection / rulesets): which checks
  *actually* block merge vs which merely run.
- **Ratchets / baselines**: coverage floors, lint-error baselines, bundle
  budgets, type-error counts, MATCH/DONE burndown, `workflow-green-repo`
  green baseline.
- **Pre-commit / pre-push hooks**: local gates that may not mirror CI.
- **Bot gates**: CodeRabbit / Sonar / Danger — block vs advise.

For each gate record: what it checks, what happens on failure (block / warn /
silently pass), and whether it is enforced server-side or only by convention.

---

## Phase 1 — Silent-bypass failure modes  [HIGH freedom]

**`continue-on-error` / soft failure** — `continue-on-error: true`, `|| true`,
`exit 0` swallowing, or `if: always()` masking a failure. The check runs,
"passes", and enforces nothing.

**Required-in-name-only** — Runs in CI but is not in the required list →
mergeable while failing. Or a required check whose *job* can be skipped
(`if:`) so it reports success-by-absence. A skipped required check often
counts as passed.

**Path/branch filters that skip the wrong changes** — `paths:` /
`paths-ignore:` that exclude a gate for the files that needed it (security
scan skips `.github/`, so a PR that disables the scan is not scanned).
Filters that never run on the default branch, or only on PRs and not on
direct pushes.

**Condition conflicts** — Two workflows whose `if:` / triggers **both fire**
on the same event (duplicate work, race on a shared resource, contradictory
status posts) or **both skip** (each assumes the other ran). Missing
concurrency groups so two runs race and the loser's failure is discarded.

**Fork / permissions bypass** — `pull_request_target` running privileged with
untrusted code, or gates that do not run on external-contributor PRs.

**Event mismatch** — Gate triggers on `push` but merges use squash/PR (or
vice versa), so the enforced commit was never the gated one.

---

## Phase 2 — Ratchet and baseline soundness  [HIGH freedom]

**One-way that isn't** — Can the baseline be regenerated in the same PR that
regresses (`--update-snapshot`, `coverage --update-baseline` committed
alongside the drop), locking the regression in as the new floor? Baseline
moves should be a separate, reviewed act.

**Exclusion gaming** — Coverage/lint passes because the dropped files are in
`.eslintignore`, `coveragePathIgnorePatterns`, or `// istanbul ignore`. A
ratchet that measures a shrinking surface always passes.

**Threshold vs reality** — Floor so far below actual that a real drop still
clears it (60% gate, 90% actual → 89% "passes"). Global thresholds hiding
per-file collapse. Bundle budgets set above current so growth is free.

**Baseline staleness / drift** — Never regenerated (no longer reflects the
codebase) or auto-regenerated every run (never actually ratchets).

**Direction bugs** — Comparison operator backwards (passes when *worse*), or
comparing against `main` after `main` already absorbed the regression.

**Burndown ratchet integrity** — Can MATCH/DONE mark an item DONE without
verification evidence? Can the count go down (work un-done) without failing?
Cross-check `completion-judge` / `burndown-full`.

---

## Phase 2.5 — Gate archaeology (accreted gates from multiple sessions)  [HIGH freedom]

Gates accumulate: each dev, agent session, or "let's add a check" iteration
layers another workflow without removing the last. Audit the *set*:

**Duplicate gates, different vintages** — Two+ jobs checking the same
thing (two lint jobs with disagreeing configs, ESLint and Biome both
wired, an old `test.yml` and a newer `ci.yml` both running the suite).
For each pair: which actually blocks, whether configs disagree, which is
vestigial.

**Competing baselines / ratchets** — Multiple floors for one metric (jest
threshold + codecov + a committed floor), updated at different times. The
agent satisfies whichever loaded; the strictest one silently stopped
being the real gate.

**Hook-vs-CI divergence** — Pre-commit / pre-push enforcing a different
ruleset than CI, so "passes locally" and "passes CI" are different claims
and people learn `--no-verify`.

**Dead gates** — Triggers that can no longer fire (renamed branches,
removed paths, disabled but not deleted). Still cost comprehension:
every reader must reason about a gate that does nothing.

**Best-of selection** — For each duplicate cluster, name the winner:

1. Server-side-enforced beats convention
2. Aggregator-gate pattern (one required check that `needs:` all others) beats N scattered required checks
3. Merge-queue-compatible (`merge_group` + matching check names) beats PR-only
4. Auto-tightening ratchet with reviewed resets beats a static threshold
5. AST / structural checks beat regex greps
6. Owned + recently maintained beats abandoned

Record winner + losers per cluster — this is the input `housekeep-gates`
executes.

---

## Phase 3 — Verify against history (where possible)  [LOW freedom — do not invent a branch]

Where CI history is inspectable, look for a merged PR that introduced a
failure the gate should have caught. Each one is a proven hole, not a
hypothetical.

Optional cheap probe on a **scratch branch the user already has**: introduce
a deliberate violation the gate claims to catch and confirm it blocks. A
gate never tested against a real violation has unknown efficacy. Do not
create that branch or push it unless the user asked.

---

## Definition of Done

- [ ] Every gate inventoried with on-failure behavior and server-side vs convention
- [ ] All `continue-on-error` / soft-fail / `always()` masking found
- [ ] Required-check list cross-checked against what actually runs — or explicitly skipped because settings were invisible
- [ ] Path / branch / event filters checked for excluding the changes that most need gating
- [ ] Cross-workflow both-fire / both-skip / races surfaced; concurrency groups checked
- [ ] Fork / `pull_request_target` / external-PR bypass assessed
- [ ] Every ratchet checked for same-PR overwrite, exclusion gaming, slack threshold, staleness, direction bug
- [ ] Burndown / completion ratchet integrity checked against its evidence requirement
- [ ] History scanned for regressions-shipped-green; deliberate-violation probe noted or skipped with reason
- [ ] Duplicate-gate clusters, competing baselines, hook-vs-CI divergence, and dead gates mapped; a winner named per cluster
- [ ] Read-only — findings presented, gates not edited; consolidation handed to `housekeep-gates`

## Self-critique before reporting  [LOW freedom — do not skip]

1. **Evidenced** — YAML line, `gh api` result, or merged PR — not "probably bypassable"
2. **Settings honesty** — if branch protection was invisible, say so; do not invent the required list
3. **Severity justified** — critical = ships-broken-green
4. **Right owner** — consolidation → `housekeep-gates`; cost → `audit-cicd`
5. **Winner named** — every duplicate cluster has a winner + why

## Output format

1. **Gate inventory** — gate | checks what | on-failure (block/warn/silent) | enforced where
2. **Bypass findings** — gate | failure mode | severity (critical = ships-broken-green) | evidence
3. **Ratchet findings** — ratchet | weakness | can it lock in / be gamed? | fix
4. **Conflict map** — workflow A × workflow B | both-fire / both-skip / race
5. **Archaeology map** — duplicate cluster | members | winner (+ why) | losers to retire
6. **Proven holes** — merged PR / probe that passed a gate it shouldn't have
7. **Fix plan** — silent-bypass first, then ratchet integrity, then conflicts. Consolidation → `housekeep-gates`; pipeline-side → `audit-cicd`; rule-to-mechanical gaps → `enhance-agent-guardrails`. Present and stop.