Kubernetes Security Skill

Harden Kubernetes clusters and workloads through RBAC, pod security, network isolation, secret management, and supply chain controls.

Instructions

Step 1: RBAC -- Least-Privilege Roles and Bindings

Grant the minimum permissions required. Prefer namespace-scoped Roles over ClusterRoles. Write exact verbs and resources in production -- even in dev clusters, because dev habits carry forward and dev manifests get promoted. Write exact verbs and resources every time.

# Good: namespace-scoped Role with specific verbs and resources
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: app-team
  name: deployment-reader
rules:
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]

# Bind the Role to a specific ServiceAccount, not a user or group wildcard
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: app-team
  name: deployment-reader-binding
subjects:
  - kind: ServiceAccount
    name: ci-deployer
    namespace: app-team
roleRef:
  kind: Role
  name: deployment-reader
  apiGroup: rbac.authorization.k8s.io

ServiceAccount best practices:

• Create dedicated ServiceAccounts per workload -- create dedicated ServiceAccounts per workload
• Set automountServiceAccountToken: false on pods that have no need for Kubernetes API access
• Regularly audit which ServiceAccounts have ClusterRole bindings

Step 2: PodSecurityStandards -- Baseline vs Restricted

Kubernetes PodSecurity admission replaces the deprecated PodSecurityPolicy. Apply labels at the namespace level. All containers must run as non-root with a read-only root filesystem unless there is a documented exception -- if an app claims it needs root, it usually just needs a writable /tmp; it usually just needs a writable /tmp, which an emptyDir volume solves.

# Enforce restricted profile, warn on baseline violations
apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/audit: restricted

SecurityContext for a restricted-compliant pod:

apiVersion: v1
kind: Pod
metadata:
  name: secure-app
spec:
  securityContext:
    runAsNonRoot: true
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: registry.example.com/app:v1.2.3@sha256:abc123
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        runAsUser: 1000
        runAsGroup: 1000
        capabilities:
          drop: ["ALL"]
      resources:
        limits:
          memory: "256Mi"
          cpu: "500m"
        requests:
          memory: "128Mi"
          cpu: "100m"

Key differences:

• Baseline -- blocks known privilege escalations (hostNetwork, privileged, hostPID) but allows running as root
• Restricted -- enforces non-root, drops all capabilities, requires seccomp profile, disallows privilege escalation

Step 3: Network Policies -- Default Deny and Allow-Lists

Start with a default-deny policy for both ingress and egress in every namespace. Apply this on day one, not later -- without network policies, lateral movement between compromised pods is trivial.

# Default deny all traffic in the namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

Then add specific allow rules:

# Allow frontend pods to reach backend on port 8080
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

# Allow DNS egress for all pods (required for service discovery)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-egress
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to: []
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

Step 4: Secret Management

Store secrets using Sealed Secrets or External Secrets Operator, environment variables from manifests, or checked-in YAML. Secrets exposed as env vars are visible in kubectl describe pod output, which makes them trivially discoverable after any pod compromise. Use one of these approaches instead:

Sealed Secrets -- encrypts secrets client-side so they are safe in Git:

# Encrypt a secret with kubeseal
kubectl create secret generic db-creds \
  --from-literal=password=supersecret \
  --dry-run=client -o yaml | \
  kubeseal --format yaml > sealed-db-creds.yaml

External Secrets Operator -- syncs secrets from external vaults:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: production
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: db-credentials
  data:
    - secretKey: password
      remoteRef:
        key: secret/data/production/db
        property: password

Use these alternatives instead:

• Mounting secrets as environment variables in the pod spec (visible in kubectl describe pod)
• Storing secrets in ConfigMaps
• Hardcoding credentials in container images or Dockerfiles

Step 5: Image Security

Containers should instead run as privileged or with elevated capabilities unless explicitly justified -- privileged mode grants full host access to an attacker if the pod is compromised. Use specific capabilities or debug containers instead.

Build minimal, non-root container images:

# Use distroless or minimal base images
FROM gcr.io/distroless/static-debian12:nonroot
COPY --chown=65532:65532 app /app
USER 65532:65532
ENTRYPOINT ["/app"]

Requirements:

• Non-root user: Always set USER in the Dockerfile and runAsNonRoot: true in the SecurityContext
• Read-only root filesystem: Use readOnlyRootFilesystem: true and mount writable volumes only where needed
• Distroless or scratch: No shell, no package manager -- reduces attack surface
• Pin image digests: Use image:tag@sha256:... to prevent tag mutation attacks
• Scan images: Run Trivy, Grype, or Snyk in CI before pushing to registry

Step 6: Supply Chain Security

Image signing with cosign:

# Sign an image after building
cosign sign --key cosign.key registry.example.com/app:v1.2.3@sha256:abc123

# Verify before deploying
cosign verify --key cosign.pub registry.example.com/app:v1.2.3@sha256:abc123

Admission controllers to enforce policy at deploy time:

• Kyverno or OPA Gatekeeper -- reject pods that violate security policies
• Sigstore Policy Controller -- verify image signatures before admission

Example Kyverno policy to require non-root containers:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-run-as-nonroot
spec:
  validationFailureAction: Enforce
  rules:
    - name: run-as-non-root
      match:
        any:
          - resources:
              kinds:
                - Pod
      validate:
        message: "Containers must run as non-root"
        pattern:
          spec:
            containers:
              - securityContext:
                  runAsNonRoot: true

Step 7: Detect Common Misconfigurations

Watch for these frequent security mistakes:

Error Handling

Error: Pod rejected by PodSecurity admission

Cause: Pod spec violates the namespace's PodSecurity level (e.g., missing runAsNonRoot, privileged: true). Solution: Check the admission warning message, then update the pod's SecurityContext to comply with the enforced level.

Error: NetworkPolicy blocking legitimate traffic

Cause: Default-deny is in place but the allow-list rule is missing or has incorrect label selectors. Solution: Verify pod labels match the NetworkPolicy podSelector and from/to selectors. Use kubectl describe networkpolicy to inspect rules.

Error: RBAC "access denied" errors in application logs

Cause: ServiceAccount lacks required permissions. Solution: Identify the API group, resource, and verb from the error message. Create or update a Role with the exact permissions needed -- list specific verbs and resources.

References

• Kubernetes Pod Security Standards
• Kubernetes Network Policies
• Kubernetes RBAC
• External Secrets Operator
• Sealed Secrets
• Cosign
• Kyverno