dev-devops

Dev-DevOps — Production Infrastructure & Delivery

Build reliable, secure, and automated infrastructure and delivery pipelines.

This skill has modular references for specialized guidance — read the relevant ones before coding.

> C0/C1 work (small local patches): See dev §0.0 Work Classifier + §0.1 Patch Fast-Path before reading references.

> dev is canonical: dev §0.2 Rule Classes, §3 Verification Gate, and §5 Safety Rules apply to all work governed by this skill.

Severity mapping: CRITICAL/HIGH ⇒ STRICT; MEDIUM ⇒ DEFAULT (aligned with dev §0.2).

Modular References

Read package-release.md for package publishing, registry auth, npm/PyPI

trusted publishing, Bun-to-npm release decisions, and downstream package

channels. Read cross-platform-release.md when a release claim depends on

OS-local behavior that CI may not prove. Read homebrew.md for Formula/Cask

distribution work. Read platform-engineering.md for broader DevOps capability

refresh, DORA, provider routing, and platform guardrails. Read docker.md + ci-cd-deploy.md first for containerized deploy workflows.

For K8s-specific work, add kubernetes.md. For SRE/on-call, add sre-foundations.md.

When release, registry-auth, provider-doc, service-status, image/platform

version, or package-manager behavior depends on current external evidence, read

the active search skill and follow its source-fetch and evidence-status rules

instead of relying on stale memory or copied snippets.

---

§1 Container Builds

§1.1 Dockerfile Rules (STRICT)

For canonical Dockerfile templates, read references/docker.md §1.

§1.2 Image Security & Supply-Chain Baseline (STRICT)

CRITICAL/HIGH findings → block push. No exceptions. The 2026 baseline is ONE workflow,

not separate tips: minimal base image (§1.1) → SBOM generation (Syft / Docker Scout) →

vulnerability scan (Trivy or Grype) → sign + attest (Cosign/Sigstore) → digest-based

promote (§2.4). Read references/docker.md §4 for scan/SBOM/sign command examples, and

../dev-security/references/supply-chain-sbom.md for deeper SBOM/signing policy.

§1.3 Anti-Patterns

---

§2 Deploy Pipeline

§2.1 Pipeline Stages (DEFAULT)

[dev-testing §5]  lint → typecheck → test → contract → e2e
[dev-devops]      build-image → scan → push-registry → deploy-staging → smoke → promote → deploy-prod

§2.2 GHA Reusable Workflows (DEFAULT)

# .github/workflows/ci.yml (caller)
jobs:
  build:
    uses: org/templates/.github/workflows/build-test.yml@v2
    with:
      service: payments
    secrets: inherit

§2.3 Deploy Strategies (DEFAULT)

§2.4 Rollback Rules (STRICT)

• Every deployment must be rollback-capable within 5 minutes
• Digest-based promote only — mutable tags are banned
• DB migrations: forward-only + backward-compatible (expand-contract pattern)
• Post-rollback: automatic Slack/PagerDuty notification

§2.5 Secret Management (STRICT)

Rule (DEVOPS-AUTH-01): prefer OIDC, workload identity federation, trusted

publishing, or another short-lived credential flow before static long-lived

tokens. When a static token is unavoidable, scope it narrowly, store it in the

managed secret system, and rotate on schedule or on incident.

§2.6 GitOps (DEFAULT)

• Actions = CI, ArgoCD = CD — separation of concerns
• Actions updates deploy repo (image digest PR/commit) → ArgoCD reconciles
• Self-heal: ArgoCD auto-reverts drift
• Environment protection: GitHub Environments for prod approval gate

§2.7 Release Proof Contract (STRICT)

Rule (DEVOPS-RELEASE-PROOF-01): a release claim must name the artifact digest,

the workflow or builder identity, the deploy target and environment, smoke-test

evidence, and rollback evidence. Five items, each an artifact rather than an

assertion — "deployed successfully" names none of them.

Keep the proof at this level; detailed package, platform, and supply-chain mechanics

live in the release references.

§2.8 Freeze and GO/NO-GO Gates (STRICT)

DEVOPS-RELEASE-PROOF-01 governs the proof bundle for an artifact you already

published. This section governs the decision to publish at all — the readiness

report, and the gates it claims to have passed.

Why gate-weakening is the load-bearing one. In the case that produced these

rules, a freeze audit rejected the first GO report on three counts: unresolved

review threads on merged PRs, a red gate argued into an exception, and missing

frozen-head receipts. The red-gate part is the instructive one — the suite was red,

the red tests were known to be load-sensitive, and the fix was real, so the report

explained the exception. The audit rejected that anyway, and correctly: **an

exception argued *after* a gate fails is indistinguishable from an exception argued

*because* it failed.**

The honest move was to decompose the gate to match what CI actually runs

(DEVOPS-SUITE-PARTITION-01), which turned the general suite green. The one

remaining local failure was waived separately on the DEVOPS-BASELINE-DEFECT-01

triple — not by the partitioning. Two different mechanisms for two different

problems; collapsing them is how a partition becomes a way to hide a defect.

Operational mechanics — suite partitioning, baseline-versus-defect attribution,

instrument stability, and exact-head evidence — live in

references/ci-cd-deploy.md §6. Runtime and operator-signal evidence rules live in

references/sre-foundations.md §7.

§2.9 Branch Lifecycle Hygiene (STRICT)

Delivery repositories accumulate dead refs, and the cost is not disk. Stale branches

make the remote branch list unusable for triage, keep superseded heads reachable by

tooling that resolves names, and hide the handful of branches that actually still

matter. Treat branch lifecycle as delivery infrastructure.

Why the merged/closed distinction is load-bearing. delete_branch_on_merge

reads as complete branch hygiene, so a repository with it enabled looks solved. It

is not: in the case behind this rule the setting was on and the repository still

carried 59 dead remote branches, because closed-unmerged PRs fall outside what it

covers. The failure is silent and compounds — nothing reports it, and the branch

list simply degrades until triage stops using it.

Why stacked PRs break naive cleanup. A stacked child PR targets its parent's

head branch, so deleting a closed parent closes the open child. "The PR that

owned this branch is closed" is therefore insufficient grounds for deletion: the

base of any open PR is protected regardless of its own PR state

(dev DEV-STACK-04).

Fork heads are out of scope, and identity is by repository id. A fork's head

lives in the contributor's repository; deleting refs there is neither permitted nor

intended. Compare repository ids, not names — a fork commonly carries the same

branch names as upstream, so name comparison silently misclassifies it.

Mechanics, the deletion-plan algorithm, and a worked audit live in

references/branch-lifecycle.md.

---

§3 Kubernetes Basics

§3.1 Minimum Viable K8s (DEFAULT)

§3.2 Gateway API (v1.6+, verified 2026-07-02)

Gateway API is the successor for new routing while Ingress remains GA but feature-frozen

(not planned for removal). v1.6.0 (2026-06) graduates TCPRoute and UDPRoute to GA. Role

separation: platform team owns GatewayClass + Gateway, app team owns HTTPRoute.

Progressive delivery pairs directly with it: Flagger supports Gateway API HTTPRoute canaries.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: payments-route
spec:
  parentRefs:
    - name: shared-gateway
  hostnames: ["payments.example.com"]
  rules:
    - matches:
        - path: { type: PathPrefix, value: /api }
      backendRefs:
        - name: payments-svc
          port: 8080

§3.3 Scaling (DEFAULT)

§3.4 Anti-Patterns

---

§4 Infrastructure as Code

§4.1 OpenTofu/Terraform Rules (DEFAULT)

§4.2 Tool Selection (HEURISTIC)

§4.3 Anti-Patterns

---

§5 SRE Foundations

§5.1 SLO/SLI (DEFAULT)

SLIs measure user experience, not infrastructure metrics. "CPU is fine ≠ users are fine."

Error budget = 1 − SLO (99.9% → 0.1% budget).

DORA 2025 (verified 2026-07-02): AI acts as an *amplifier* — returns depend on the

underlying sociotechnical system. For AI-agent-heavy delivery, invest in golden paths,

guardrails, observability, provenance, and review gates, not just faster merges.

§5.2 Error Budget Policy (DEFAULT)

Single incident consuming >20% → mandatory postmortem.

Two consecutive window misses → architecture review.

§5.3 Incident Response (DEFAULT)

1. Detect → Triage (S1/S2/S3) → Stabilize → Fix → Postmortem
2. Roles: IC, Primary Responder, Comms Lead, Scribe
3. Mitigation first, diagnosis second during active incidents
4. All S1/S2 → mandatory blameless postmortem within 5 business days
5. Status updates: S1 every 15min, S2 every 30min

§5.4 Runbook Template (HEURISTIC)

## [Service] — [Symptom]
### Diagnosis
1. Logs: `kubectl logs -l app=<name> --tail=100`
2. Metrics: Grafana → [dashboard URL]
3. Dependencies: `curl -s http://<dep>/health | jq .`
### Emergency Mitigation
1. Rollback: `argocd app rollback <app>`
2. Traffic block: ...
### Root Fix
1. ...
### Escalation
- Owner: @team-sre
- PagerDuty: [policy]

§5.5 Anti-Patterns

---

§6 Cross-References

dev-devops owns: container builds, deploy pipelines, K8s manifests, IaC modules, SRE/incident response, edge infra, ML infra. DevOps owns operational supply-chain execution (scans, SBOM/signing steps, attestations, and release gates); dev-security owns supply-chain security policy, severity thresholds, and evidence requirements.

dev-backend owns: application-layer observability code, API design, health check implementation.

Overlap: observability alerting rules (dev-devops §5) ↔ observability code instrumentation (dev-backend observability.md). Cross-ref both.

Pre-flight Checklist

Before submitting infrastructure changes:

• [ ] Dockerfile is multi-stage with distroless/slim final image and non-root user
• [ ] Image scanned (Trivy/Scout) with CRITICAL/HIGH gate — no unresolved findings
• [ ] SBOM generated and attestation signed (Cosign) for production images
• [ ] Deploy pipeline uses digest-based promotion, never mutable tags
• [ ] K8s manifests have resource requests/limits, probes, PDB, and use Gateway API (not Ingress)
• [ ] IaC uses remote state, pinned versions, and modular decomposition
• [ ] Secrets are managed through Vault/AWS SM/GHA Secrets — no .env commits, no ARG secrets
• [ ] SLO/SLI defined with error budget policy and burn-rate alerting
• [ ] Rollback plan documented and tested — <5min rollback capability confirmed
• [ ] Runbook exists for critical failure scenarios