dev-devops
Source: skills/dev-devops/SKILL.md
Container builds, deploy pipelines, K8s, IaC, SRE, edge/serverless, ML infra.
인프라 작업은 단일 실수가 프로덕션 전체를 다운시킬 수 있다. unpinned base image, root로 실행되는 컨테이너, 시크릿이 노출된 CI 로그. 이 스킬은 컨테이너 빌드부터 배포 전략, Kubernetes, IaC, SRE까지 인프라 전 영역의 STRICT 규칙을 한곳에 모아서 에이전트가 인프라 작업 시 안전하게 움직이게 한다.
LLM의 학습 데이터에는 2022-2023년의 Dockerfile과 CI 설정이 압도적으로 많다. 그래서 FROM node:latest, USER root, ARG로 시크릿 전달 같은 안티패턴을 자연스럽게 생성한다. 이건 학습 데이터의 분포 문제다 — 나쁜 Dockerfile이 좋은 Dockerfile보다 인터넷에 훨씬 많다. 또한 OIDC/Trusted Publishing은 2024-2025년에 주류가 되었기 때문에 학습 데이터에 반영이 부족하다. LLM은 기본적으로 NPM_TOKEN을 먼저 요구하는데, 이건 이미 열등한 폴백 경로다. Kubernetes도 마찬가지다 — Ingress 예제가 학습 데이터에 압도적으로 많아서 Gateway API v1.6+를 자발적으로 쓰지 않는다.
이 스킬이 해결하는 실제 문제
2025-2026년 npm과 PyPI가 Trusted Publishing(OIDC)을 기본 지원하면서, 장기 토큰(NPM_TOKEN, PYPI_TOKEN)은 보안상 열등한 폴백이 되었다. 에이전트는 관성적으로 토큰을 먼저 요청하는데, package-release.md의 Auth Decision Table은 OIDC를 먼저 시도하고 토큰은 OIDC가 불가능할 때만 쓰게 한다.
에이전트가 Dockerfile을 만들 때 FROM node:latest를 쓰면 다음 주에 빌드가 깨진다. section 1.1은 버전+SHA256 다이제스트 고정을 STRICT 규칙으로 강제한다.
2026년 Sigstore/Cosign 기반 이미지 서명이 표준이 되었고, SBOM 생성(Syft/Docker Scout)이 컴플라이언스 요구사항이다. section 1.2는 스캔 -> SBOM -> 서명 -> 다이제스트 프로모트를 하나의 파이프라인으로 묶는다.
Ingress는 deprecated되고 Gateway API가 표준이 되었다. 에이전트의 학습 데이터에는 Ingress 예제가 더 많아서 구형 패턴을 생성한다. kubernetes.md는 Gateway API v1.6+ 기준으로 작성되었다.
Triggers: Dockerfile, K8s, CI/CD, Terraform, IaC, release, deploy, SRE
Key Concepts
- Dockerfile Rules (multi-stage, non-root, pinned)
- Image Security & Supply Chain
- CI/CD Pipeline Design
- Kubernetes (Gateway API v1.6+)
- Infrastructure as Code (OpenTofu/Pulumi)
- SRE Foundations (SLO/SLI/Error Budget)
Related Skills
Reference Documents
| Document | Description |
|---|---|
| CI/CD & Deploy — Pipeline & Delivery Patterns | GHA 재사용 워크플로, 배포 전략, 롤백, GitOps |
| Cross-platform Release — CI Matrix vs Local OS Proof | |
| Docker — Container Build & Security Patterns | 멀티스테이지 빌드, distroless, Docker Scout/Trivy, SBOM/Cosign |
| Edge & Serverless — Edge Computing Patterns | 엣지 요청 셰이핑, Cloudflare Workers, Vercel Edge |
| Homebrew — Formula & Cask Distribution | |
| Infrastructure as Code — OpenTofu, Terraform, Pulumi | OpenTofu/Terraform 모듈, Pulumi, 상태 암호화, 블래스트 반경 격리 |
| Kubernetes — Deployment & Orchestration Patterns | Gateway API, Kustomize, HPA/VPA, Helm, ArgoCD GitOps |
| ML Infrastructure — GPU Clusters, Model Registry, Serving | GPU 클러스터, 모델 레지스트리, 스케일링, 엣지 추론 |
| Package Release — Trusted Publishing & Registry Auth | npm/PyPI 신뢰 퍼블리싱(OIDC), 레지스트리 인증 모델 |
| Platform Engineering — DevOps Capability Routing | |
| SRE Foundations — SLO, Incident Response, Postmortem | SLO/SLI/에러 예산, 번 레이트 알러팅, 인시던트 대응 |
Sections Overview
| Section | Summary |
|---|---|
| Container Builds | Dockerfile 규칙, 멀티스테이지, non-root, 의존성 우선 복사. |
| Image Security | SBOM 생성, 취약점 스캔, 서명, 다이제스트 기반 프로모션. |
| CI/CD Pipeline | 재사용 워크플로, 배포 전략(blue-green/canary), 롤백 절차. |
| Kubernetes | Gateway API, Kustomize 오버레이, 오토스케일링, GitOps. |
| SRE Foundations | SLO/SLI 정의, 에러 예산, 인시던트 대응, 사후 분석. |
Academic References
| Paper | Year | Relevance |
|---|---|---|
SoK: Software Supply Chain Attacks and CountermeasuresarXiv:2307.07529 | 2023 | SSC 공격 분류와 SLSA/Sigstore/SBOM 대응책 매핑. |
Systematically Identifying Security Smells in IaC ScriptsarXiv:1907.07159 | 2021 | Puppet/Ansible/Chef의 하드코딩 시크릿, 암호화 누락 등 안티패턴 식별. |
Official Guides
- SLSA Framework — 소프트웨어 서플라이 체인 보안 레벨.
- Sigstore Documentation — 키리스 서명 및 투명성 로그.
- Kubernetes Gateway API — Ingress를 대체하는 표준 K8s API.
- PyPI Trusted Publishers — OIDC 기반 PyPI 패키지 퍼블리싱.
- npm Trusted Publishers — OIDC 기반 npm 패키지 퍼블리싱.
- npm Provenance — npm 패키지 출처 증명.
- GitHub Actions OIDC — GitHub Actions OIDC 토큰 인증.
- DORA Research — DevOps 4 핵심 메트릭 연구.
- Google SRE Book — SRE 원칙과 실천 표준 참고서.
- OpenTelemetry Documentation — 분산 트레이싱, 메트릭, 로깅의 공식 가이드.
Full Specification
Show full SKILL.md content
Dev-DevOps — Production Infrastructure & Delivery
Build reliable, secure, and automated infrastructure and delivery pipelines.
This skill has modular references for specialized guidance — read the relevant ones before coding.
> C0/C1 work (small local patches): See dev §0.0 Work Classifier + §0.1 Patch Fast-Path before reading references.
Severity mapping: CRITICAL/HIGH ⇒ STRICT; MEDIUM ⇒ DEFAULT (aligned with dev §0.2).
Modular References
| File | When to Read | What It Covers | |
|---|---|---|---|
references/docker.md | Container build/deploy | Multi-stage builds, distroless, Docker Scout/Trivy, BuildKit secrets, SBOM/Cosign | |
references/package-release.md | Package publishing / release auth | npm/PyPI trusted publishing, Bun-to-npm, registry auth model, downstream distribution table | |
references/cross-platform-release.md | Cross-platform release proof | CI matrix vs local OS proof, Windows App/RDP prompts, desktop verification boundaries | |
references/homebrew.md | Homebrew distribution | Formula vs Cask, audit/test, livecheck, artifact trust, install/uninstall proof | |
references/platform-engineering.md | Platform / DORA / provider routing | DORA capabilities, platform guardrails, provider table rows, SLSA handoff |
references/kubernetes.md | K8s deployment | Gateway API (v1.6+), Kustomize overlays, HPA/VPA, Helm, ArgoCD GitOps |
references/ci-cd-deploy.md | Deploy pipeline | GHA reusable workflows, deploy strategies, rollback, GitOps, progressive delivery |
references/iac.md | Infrastructure code | OpenTofu/Terraform modules, Pulumi, state encryption, blast radius isolation |
references/sre-foundations.md | Operations/incidents | SLO/SLI/error budget, burn-rate alerting, incident response, blameless postmortem |
references/edge-serverless.md | Edge/serverless work | Edge request shaping, auth at edge, Cloudflare Workers, Vercel Edge, edge AI triage |
references/ml-infra.md | ML infrastructure | GPU cluster mgmt, model registry, scaling, edge inference, MLOps platform patterns |
Read package-release.md for package publishing, registry auth, npm/PyPI
trusted publishing, Bun-to-npm release decisions, and downstream package
channels. Read cross-platform-release.md when a release claim depends on
OS-local behavior that CI may not prove. Read homebrew.md for Formula/Cask
distribution work. Read platform-engineering.md for broader DevOps capability
refresh, DORA, provider routing, and platform guardrails. Read docker.md + ci-cd-deploy.md first for containerized deploy workflows.
For K8s-specific work, add kubernetes.md. For SRE/on-call, add sre-foundations.md.
When release, registry-auth, provider-doc, service-status, image/platform
version, or package-manager behavior depends on current external evidence, read
the active search skill and follow its source-fetch and evidence-status rules
instead of relying on stale memory or copied snippets.
---
§1 Container Builds
§1.1 Dockerfile Rules (STRICT)
| Rule | Detail | ||
|---|---|---|---|
| Multi-stage | Separate build and runtime stages; final image has no build tools | ||
| Base image | Pin version + SHA256 digest: node:22-slim@sha256:abc... | ||
| Minimal runtime base | Prefer gcr.io/distroless/* (Debian-based, keyless-signed) — no shell/package manager; OR Chainguard/Wolfi images when nightly rebuilds, SBOM attestations, patch SLAs, or compliance matter | ||
| Non-root | USER nonroot:nonroot (distroless) or create dedicated user | ||
| Dependency-first copy | COPY package.json bun.lock ./ → install → COPY . . for layer caching |
| BuildKit secrets | RUN --mount=type=secret,id=token ... — never use ARG for secrets |
.dockerignore | .git, node_modules, .env*, *.log, dist/, coverage/, __pycache__/ |
For canonical Dockerfile templates, read references/docker.md §1.
§1.2 Image Security & Supply-Chain Baseline (STRICT)
CRITICAL/HIGH findings → block push. No exceptions. The 2026 baseline is ONE workflow,
not separate tips: minimal base image (§1.1) → SBOM generation (Syft / Docker Scout) →
vulnerability scan (Trivy or Grype) → sign + attest (Cosign/Sigstore) → digest-based
promote (§2.4). Read references/docker.md §4 for scan/SBOM/sign command examples, and
../dev-security/references/supply-chain-sbom.md for deeper SBOM/signing policy.
§1.3 Anti-Patterns
| Banned | Symptom | Fix | |
|---|---|---|---|
FROM node:latest | Irreproducible builds | Pinned version + digest | |
USER root in final stage | Attack surface | Non-root user | |
COPY . . as first instruction | Cache invalidation on every change | Dependency files first | |
ARG SECRET=xxx | Exposed in image history | BuildKit --mount=type=secret | |
| No scan before push | Vulnerable images in prod | Trivy/Scout CI gate |
apt-get install without cleanup | Bloated image | --no-install-recommends && rm -rf /var/lib/apt/lists/* |
---
§2 Deploy Pipeline
§2.1 Pipeline Stages (DEFAULT)
[dev-testing §5] lint → typecheck → test → contract → e2e
[dev-devops] build-image → scan → push-registry → deploy-staging → smoke → promote → deploy-prod
§2.2 GHA Reusable Workflows (DEFAULT)
# .github/workflows/ci.yml (caller)
jobs:
build:
uses: org/templates/.github/workflows/build-test.yml@v2
with:
service: payments
secrets: inherit
| Rule | Detail |
|---|---|
workflow_call | Central CI template, max 10-level nesting |
| Permissions | Caller cannot escalate; downgrade only |
| Environment | environment: production + required reviewers + prevent self-review |
| Promote | Digest-based (image@sha256:...), never mutable tags |
§2.3 Deploy Strategies (DEFAULT)
| Strategy | Tool | When | Risk |
|---|---|---|---|
| Rolling update | K8s Deployment | Stateless, low risk | Low |
| Blue-green | Argo Rollouts blueGreen: | Instant rollback, no DB migration | Medium |
| Canary | Argo Rollouts canary: steps: | Traffic % control, metric-based promote | Medium |
| Progressive | Flagger | Auto analysis + rollback, A/B testing | Medium-High |
| Feature flag | LaunchDarkly / Unleash | Code-level gradual rollout | Low |
§2.4 Rollback Rules (STRICT)
- Every deployment must be rollback-capable within 5 minutes
- Digest-based promote only — mutable tags are banned
- DB migrations: forward-only + backward-compatible (expand-contract pattern)
- Post-rollback: automatic Slack/PagerDuty notification
§2.5 Secret Management (STRICT)
| Source | Usage |
|---|---|
| GHA Secrets / Vault / AWS SM | CI pipeline secrets |
| External Secrets Operator | K8s → Vault/AWS SM sync |
.env files | Never committed — generated in CI |
| Rotation | 90-day cycle or immediate on incident |
§2.6 GitOps (DEFAULT)
- Actions = CI, ArgoCD = CD — separation of concerns
- Actions updates deploy repo (image digest PR/commit) → ArgoCD reconciles
- Self-heal: ArgoCD auto-reverts drift
- Environment protection: GitHub Environments for prod approval gate
---
§3 Kubernetes Basics
§3.1 Minimum Viable K8s (DEFAULT)
| Resource | Purpose | |
|---|---|---|
| Deployment | Pod template + replica management | |
| Service | Internal networking | |
| HTTPRoute (Gateway API) | External traffic routing — not Ingress | |
| ResourceQuota | Request/limit enforcement | |
| Probes | Liveness + readiness + startup |
| Namespace | Environment isolation (dev/staging/prod) |
§3.2 Gateway API (v1.6+, verified 2026-07-02)
Gateway API is the successor for new routing while Ingress remains GA but feature-frozen
(not planned for removal). v1.6.0 (2026-06) graduates TCPRoute and UDPRoute to GA. Role
separation: platform team owns GatewayClass + Gateway, app team owns HTTPRoute.
Progressive delivery pairs directly with it: Flagger supports Gateway API HTTPRoute canaries.
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: payments-route
spec:
parentRefs:
- name: shared-gateway
hostnames: ["payments.example.com"]
rules:
- matches:
- path: { type: PathPrefix, value: /api }
backendRefs:
- name: payments-svc
port: 8080
§3.3 Scaling (DEFAULT)
| Mechanism | Scope | Watch |
|---|---|---|
| HPA | CPU/memory + custom metrics | Don't combine with VPA on same metric |
| VPA | Request auto-tuning | Use Off mode for recommendations only |
| PDB | Disruption budget | minAvailable: 50% or maxUnavailable: 1 |
§3.4 Anti-Patterns
| Banned | Fix | |
|---|---|---|
Ingress (new projects) | Gateway API HTTPRoute | |
| No resource limits | Always set requests + limits | |
image: app:latest | SHA digest or pinned SemVer | |
| Single replica in prod | Minimum 2 + PDB | |
| Secrets in ConfigMap | K8s Secret + External Secrets Operator |
| Annotation-based routing | Gateway API native fields |
---
§4 Infrastructure as Code
§4.1 OpenTofu/Terraform Rules (DEFAULT)
| State | Remote backend required (S3+DynamoDB / TF Cloud / OpenTofu) |
| Encryption | OpenTofu native state/plan encryption via KMS |
| Blast radius | Separate state per app/layer/env |
| Modules | Purpose-built (vpc, iam, ecs-service), typed I/O |
| Apply | plan → PR review → apply; auto-apply staging only |
| Versions | Pin provider + module versions explicitly |
§4.2 Tool Selection (HEURISTIC)
| Tool | Best For | Status (verified 2026-07-02) |
|---|---|---|
| OpenTofu (HCL) | Open-source/licensing-neutral IaC default (MPL-2.0, Linux Foundation; v1.12.x) | ✅ Recommended for OSS neutrality |
| Terraform / HCP Terraform (BSL) | Vendor support or HashiCorp platform integration required | ✅ Active (BSL license — check competitive-use terms) |
| Pulumi (TS/Python) | Teams preferring programming languages | ✅ Active |
| AWS CDK | AWS-only infrastructure | ✅ Active (AWS only) |
| CDKTF | — | ❌ Deprecated 2025-12-10; repo archived/read-only, no further fixes |
§4.3 Anti-Patterns
| Banned | Fix | |
|---|---|---|
| Local state file | Remote backend required | |
| Manual console changes | All changes via code | |
| Hardcoded values | Variables + tfvars | |
| Monolithic main.tf | Modular decomposition | |
| CDKTF (new projects) | OpenTofu or Pulumi |
| Unpinned provider versions | Explicit version constraints |
---
§5 SRE Foundations
§5.1 SLO/SLI (DEFAULT)
| Availability | Success requests / total | 99.9% (28-day rolling) |
| Latency | p50/p95/p99 response time | p99 < 500ms |
| Error rate | 5xx / total | < 0.1% |
| Freshness | Data update delay | < 5min (pipelines) |
SLIs measure user experience, not infrastructure metrics. "CPU is fine ≠ users are fine."
Error budget = 1 − SLO (99.9% → 0.1% budget).
DORA 2025 (verified 2026-07-02): AI acts as an *amplifier* — returns depend on the
underlying sociotechnical system. For AI-agent-heavy delivery, invest in golden paths,
guardrails, observability, provenance, and review gates, not just faster merges.
§5.2 Error Budget Policy (DEFAULT)
| Budget State | Action |
|---|---|
| Normal (>50%) | Continue releases, routine monitoring |
| Accelerated burn (20–50%) | Heightened alerts, slow releases, reliability triage |
| Exhausted (≤0%) | Feature freeze — security/bugfix only; VP exception required |
Single incident consuming >20% → mandatory postmortem.
Two consecutive window misses → architecture review.
§5.3 Incident Response (DEFAULT)
- Detect → Triage (S1/S2/S3) → Stabilize → Fix → Postmortem
- Roles: IC, Primary Responder, Comms Lead, Scribe
- Mitigation first, diagnosis second during active incidents
- All S1/S2 → mandatory blameless postmortem within 5 business days
- Status updates: S1 every 15min, S2 every 30min
§5.4 Runbook Template (HEURISTIC)
## [Service] — [Symptom]
### Diagnosis
1. Logs: `kubectl logs -l app=<name> --tail=100`
2. Metrics: Grafana → [dashboard URL]
3. Dependencies: `curl -s http://<dep>/health | jq .`
### Emergency Mitigation
1. Rollback: `argocd app rollback <app>`
2. Traffic block: ...
### Root Fix
1. ...
### Escalation
- Owner: @team-sre
- PagerDuty: [policy]
§5.5 Anti-Patterns
| Banned | Fix | |
|---|---|---|
| Infrastructure-only SLIs | User-experience-based SLIs | |
| SLO without consequences | Error budget policy with freeze gate | |
| Too many SLIs (>5 per service) | 2-4 meaningful SLIs | |
| Page on every deviation | Burn-rate multi-window alerting | |
| Blame individuals | Blameless postmortem, system improvement |
| No error budget policy | Define 3-stage policy (normal/accelerated/exhausted) |
---
§6 Cross-References
dev-testing §5| Backend observability code patterns | dev-backend observability.md | OTel SDK setup, structured logging |
| Security hardening (app-layer) | dev-security | OWASP, auth, input validation |
| SBOM/signing depth | dev-security references/supply-chain-sbom.md | Supply-chain evidence policy beyond image scan gates |
| Architecture module boundaries | dev-architecture | Coupling taxonomy, barrel discipline |
| Scaffolding conventions | dev-scaffolding | File naming, project structure |
| Frontend build/bundle | dev-frontend | Vite/webpack config, SSR |
dev-devops owns: container builds, image security, deploy pipelines, K8s manifests, IaC modules, SRE/incident response, edge infra, ML infra.
dev-backend owns: application-layer observability code, API design, health check implementation.
Overlap: observability alerting rules (dev-devops §5) ↔ observability code instrumentation (dev-backend observability.md). Cross-ref both.
Pre-flight Checklist
Before submitting infrastructure changes:
- [ ] Dockerfile is multi-stage with distroless/slim final image and non-root user
- [ ] Image scanned (Trivy/Scout) with CRITICAL/HIGH gate — no unresolved findings
- [ ] SBOM generated and attestation signed (Cosign) for production images
- [ ] Deploy pipeline uses digest-based promotion, never mutable tags
- [ ] K8s manifests have resource requests/limits, probes, PDB, and use Gateway API (not Ingress)
- [ ] IaC uses remote state, pinned versions, and modular decomposition
- [ ] Secrets are managed through Vault/AWS SM/GHA Secrets — no
.envcommits, noARGsecrets - [ ] SLO/SLI defined with error budget policy and burn-rate alerting
- [ ] Rollback plan documented and tested — <5min rollback capability confirmed
- [ ] Runbook exists for critical failure scenarios