KUBERNETES: FROM BASIC TO ADVANCED

Module 6: Security

xdev.asia

🎯 Lesson Objective

Know how to use essential security tools: kube-bench for compliance checking, Trivy for vulnerability scanning, Falco for runtime detection, OPA/Gatekeeper for policy enforcement.

1. kube-bench — CIS Benchmark

kube-bench tests your Kubernetes cluster against CIS (Center for Internet Security) Kubernetes Benchmark — a widely recognized security hardening standard.

# Chạy kube-bench trên master node
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job-master.yaml
Xem kết quả
kubectl logs job/kube-bench-master
Chạy trên worker node
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/kube-bench/main/job-node.yaml
kubectl logs job/kube-bench-node

# Output example:
# [PASS] 1.1.1 Ensure that the API server pod specification file permissions are set to 600 or more restrictive
# [FAIL] 1.2.1 Ensure that the --anonymous-auth argument is set to false
# [WARN] 1.2.6 Ensure that the --kubelet-certificate-authority argument is set as appropriate
#
# == Summary ==
# 42 checks PASS
# 13 checks FAIL
# 11 checks WARN

# Remediation cho 1.2.1:
# Thêm vào kube-apiserver: --anonymous-auth=false

Priority fixes: starting with FAILs in sections 1 (API server) and 4 (kubelet).

2. Trivy — Vulnerability Scanning

Trivy is a comprehensive security scanner for containers, filesystems, Git repos, and Kubernetes clusters.

2.1 Scan Container Images

# Cài Trivy
curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin
Scan image
trivy image nginx:1.27
Chỉ hiển thị HIGH và CRITICAL
trivy image --severity HIGH,CRITICAL nginx:1.27
Scan với SBOM output
trivy image --format spdx-json -o nginx-sbom.json nginx:1.27
Fail build nếu có CRITICAL vulnerabilities
trivy image --exit-code 1 --severity CRITICAL nginx:1.27

# Output:
# nginx:1.27 (debian 12.8)
# ========================
# Total: 25 (HIGH: 8, CRITICAL: 2)
#
# ┌──────────────┬────────────────┬──────────┬────────┬───────────────┐
# │   Library    │ Vulnerability  │ Severity │ Status │ Fixed Version │
# ├──────────────┼────────────────┼──────────┼────────┼───────────────┤
# │ openssl      │ CVE-2024-5535  │ CRITICAL │ fixed  │ 3.0.14-1~deb12│
# └──────────────┴────────────────┴──────────┴────────┴───────────────┘

2.2 Scan Kubernetes Manifests

# Scan YAML manifests cho misconfiguration
trivy config ./k8s/
Output: tìm thấy container chạy root, no resource limits, etc.

2.3 Scan Running Cluster

# Scan toàn bộ cluster
trivy k8s --report all cluster
Scan chỉ workloads trong namespace
trivy k8s --namespace production cluster
Export kết quả
trivy k8s --format json -o cluster-report.json cluster

2.4 Trivy in CI/CD

# GitHub Actions
- name: Scan image
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: myapp:${{ github.sha }}
    format: sarif
    output: trivy-results.sarif
    severity: CRITICAL,HIGH
    exit-code: 1

name: Upload results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: trivy-results.sarif

3. Falco — Runtime Threat Detection

Falco uses eBPF to monitor system calls and detect suspicious behavior at runtime.

3.1 Install Falco

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
helm install falco falcosecurity/falco 

--namespace falco 

--create-namespace 

--set driver.kind=ebpf \          # dùng eBPF driver (không cần kernel module)
--set falcosidekick.enabled=true \ # forward alerts
--set falcosidekick.config.slack.webhookurl="https://hooks.slack.com/..."
kubectl get pods -n falco

3.2 Default Falco Rules__HTMLTAG_96___

# Falco detect các hành vi này theo default:
# - Shell spawned trong container
# - Sensitive file access (/etc/passwd, /etc/shadow, SSH keys)
# - Outbound network connections không mong muốn
# - Privilege escalation attempts
# - Container drift: binary được tạo sau khi container start
Xem alerts
kubectl logs -n falco -l app.kubernetes.io/name=falco -f
Alert example:
15:32:47.123456789: Warning Shell spawned in a container
(user=root container_id=abc123 container_name=nginx image=nginx:1.27
shell=sh parent=sh)

3.3 Custom Falco Rules

# /etc/falco/rules.d/custom-rules.yaml
- rule: Database Access from Unexpected Container
  desc: Detect database connections từ containers không phải backend
  condition: |
    evt.type = connect and
    fd.sport = 5432 and
    not container.image.repository contains "backend"
  output: |
    Unexpected DB connection (container=%container.name
    image=%container.image.repository user=%user.name)
  priority: WARNING

rule: Kubernetes Secret Read at Runtime
desc: Detect việc đọc mounted secrets
condition: |
open_read and
fd.name startswith /var/run/secrets/kubernetes.io and
not proc.name in (node, python, java, python3)
output: |
K8s secret read (proc=%proc.name file=%fd.name container=%container.id)
priority: NOTICE

3.4 Falco Sidekick — Alert Forwarding

# Falco Sidekick forward alerts đến nhiều destinations
helm install falco falcosecurity/falco \
  --set falcosidekick.enabled=true \
  --set falcosidekick.config.slack.webhookurl="$SLACK_WEBHOOK" \
  --set falcosidekick.config.pagerduty.routingkey="$PD_KEY" \
  --set falcosidekick.config.elasticsearch.hostport="http://elastic:9200"

4. OPA/Gatekeeper — Advanced Policy

# Cài Gatekeeper
helm repo add gatekeeper https://open-policy-agent.github.io/gatekeeper/charts
helm install gatekeeper gatekeeper/gatekeeper -n gatekeeper-system --create-namespace

# ConstraintTemplate: định nghĩa policy schema
apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        openAPIV3Schema:
          properties:
            labels:
              type: array
              items: string
  targets:
  - target: admission.k8s.gatekeeper.sh
    rego: |
      package k8srequiredlabels
      violation[{"msg": msg}] {
        required := input.parameters.labels[_]
        not input.review.object.metadata.labels[required]
        msg := sprintf("Missing required label: %v", [required])
      }
---
# Constraint: apply policy
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: ns-must-have-team
spec:
  match:
    kinds:
    - apiGroups: ["apps"]
      kinds: ["Deployment"]
    namespaces: ["production"]
  parameters:
    labels: ["team", "app", "environment"]

5. Security Pipeline

Developer commits code
        ↓
CI/CD Pipeline:
  1. Build image
  2. Trivy scan image (fail on CRITICAL)
  3. Generate SBOM
  4. Cosign sign image
  5. Push to registry
        ↓
Kubernetes Admission:
  6. Policy Controller verify image signature
  7. ValidatingAdmissionPolicy check (no :latest, resource limits)
  8. PSA check (Restricted level)
  9. OPA/Gatekeeper check (required labels, etc.)
        ↓
Runtime:
  10. Falco monitor behavior
  11. Trivy scan running workloads
  12. Alert → Slack/PagerDuty

Summary

• kube-bench: CIS benchmark compliance, fix priority FAIL items
• Trivy: scan images, manifests, and clusters — CI/CD integration
• Falco: eBPF runtime detection, custom rules, Sidekick alerting
• OPA/Gatekeeper: when mutating policies or complex rego logic is needed
• Security pipeline: shift-left (scan in CI) + runtime (Falco)