LESSON 50: CAPSTONE PROJECT — E-COMMERCE MICROSERVICES PLATFORM

Deploy Microservices On-Premises with Kubernetes HA

Part 12: Production Operations & Capstone Project

xdev.asia

🎯 LESSON OBJECTIVE__HTMLTAG_68___

• ✅ E-commerce microservices architecture design__HTMLTAG_71___
• ✅ Deploy the entire platform on K8s HA on-premises
• ✅ Apply all best practices from the previous 49 articles__HTMLTAG_75___
• ✅ Production-ready: security, observability, DR
• ✅ Performance testing and go-live

---

PART 1: SYSTEM ARCHITECTURE

graph TB
    GW["🌐 Istio Gateway
TLS + CORS"]

    GW --> US["👤 User Service
Go"]
    GW --> PS["📦 Product Service
Go"]
    GW --> OS["🛒 Order Service
Go"]

    OS --> IS["📊 Inventory Service
Go"]
    OS --> PAY["💳 Payment Service
Go"]
    PAY --> NS["📧 Notification Service
Node.js
email · SMS · push"]
    OS --> NS

    subgraph DATA["💾 Data Stores"]
        PG["🐘 PostgreSQL
HA Cluster"]
        RD["⚡ Redis
Sentinel"]
        RMQ["🐰 RabbitMQ
Quorum Queues"]
        KF["📡 Kafka
Events"]
    end

    US & PS & OS & IS & PAY --> PG
    US & PS & OS --> RD
    OS & PAY --> RMQ
    OS & PAY --> KF
    NS --> RMQ

    subgraph PLATFORM["🏗️ Platform Layer"]
        direction LR
        K8S["☸ K8s HA
3 masters + 4 workers"]
        CEPH["💿 Rook-Ceph"]
        ISTIO["🔗 Istio Mesh"]
        ARGO["🔄 ArgoCD GitOps"]
        OBS["📊 Prometheus + Loki + Tempo"]
        SEC["🛡️ Kyverno + Falco"]
        VEL["💼 Velero Backup/DR"]
    end

    style GW fill:#1d4ed8,stroke:#60a5fa,color:#fff
    style DATA fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style PLATFORM fill:#1e293b,stroke:#475569,color:#e2e8f0
    style NS fill:#7c3aed,stroke:#a78bfa,color:#fff

---

PART 2: MICROSERVICES DESIGN

Service	Language	Database	Async Events
User Service_	Go	PostgreSQL (users DB)	user.created, user.updated
Product Service	Go	PostgreSQL (products DB)	product.updated
Inventory Service_	Go	PostgreSQL (inventory DB)	stock.reserved, stock.released
Order Service	Go	PostgreSQL (orders DB)	order.created, order.completed
Payment Service_	Go	PostgreSQL (payments DB)	payment.processed, payment.failed
Notification Service_	Node.js	Redis (queue)	Consumes order/payment events

# GitOps repo structure:
ecommerce-gitops/
├── apps/
│   ├── user-service/
│   │   ├── base/
│   │   │   ├── deployment.yaml
│   │   │   ├── service.yaml
│   │   │   ├── hpa.yaml
│   │   │   ├── pdb.yaml
│   │   │   └── kustomization.yaml
│   │   └── overlays/
│   │       ├── staging/
│   │       └── production/
│   ├── product-service/
│   ├── order-service/
│   ├── payment-service/
│   ├── inventory-service/
│   └── notification-service/
├── infrastructure/
│   ├── namespaces/
│   ├── networking/
│   │   ├── istio-gateway.yaml
│   │   ├── virtualservices.yaml
│   │   └── network-policies.yaml
│   ├── databases/
│   │   ├── postgresql-cluster.yaml
│   │   ├── redis-sentinel.yaml
│   │   ├── rabbitmq-cluster.yaml
│   │   └── kafka-cluster.yaml
│   └── observability/
│       ├── servicemonitors.yaml
│       ├── prometheusrules.yaml
│       └── grafana-dashboards.yaml
├── argocd/
│   ├── applicationset.yaml
│   └── appproject.yaml
└── helmfile.yaml

---

PART 3: DEPLOYMENT STEPS

# Step 1: Prepare infrastructure (Bài 1-4)
# ✅ 7 nodes provisioned, OS hardened, HAProxy + keepalived

# Step 2: K8s HA cluster (Bài 5-10)
# ✅ 3 master + 4 worker, Cilium CNI, MetalLB, etcd backup

# Step 3: Storage (Bài 11-15)
# ✅ Rook-Ceph cluster, StorageClasses (block + filesystem)

# Step 4: Databases (Bài 16-23)
# ✅ PostgreSQL HA, Redis Sentinel, RabbitMQ cluster, Kafka

# Step 5: Service Mesh (Bài 24-27)
# ✅ Istio + Gateway, mTLS, AuthorizationPolicy

# Step 6: GitOps (Bài 28-31)
# ✅ ArgoCD, Helm charts, Vault secrets, CI/CD pipeline

# Step 7: Observability (Bài 32-35)
# ✅ Prometheus, Loki, Tempo, Grafana, SLO alerts

# Step 8: Security (Bài 36-39)
# ✅ RBAC, Kyverno, Falco, Harbor

# Step 9: Deploy services
kubectl apply -f argocd/applicationset.yaml
# ArgoCD auto-deploys all services from GitOps repo

# ArgoCD ApplicationSet (deploy all services):
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: ecommerce-services
  namespace: argocd
spec:
  generators:
    - git:
        repoURL: https://github.com/company/ecommerce-gitops
        revision: main
        directories:
          - path: apps/*
  template:
    metadata:
      name: '{{path.basename}}'
    spec:
      project: ecommerce
      source:
        repoURL: https://github.com/company/ecommerce-gitops
        targetRevision: main
        path: '{{path}}/overlays/production'
      destination:
        server: https://kubernetes.default.svc
        namespace: production
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

---

PART 4: TESTING & VALIDATION

# Integration test flow:
# 1. Create user → 2. Get products → 3. Place order
#    → 4. Check inventory reserved → 5. Process payment
#    → 6. Verify notification sent

# Load test:
k6 run ecommerce-load-test.js
# Target: 500 orders/min, P99 < 1s, errors < 0.1%

# Chaos test:
# Round 1: Kill order-service pod (verify auto-recovery)
# Round 2: Simulate payment timeout (verify retry + DLQ)
# Round 3: Database failover (verify < 5s downtime)
# Round 4: Node failure (verify pod rescheduling)

# Security validation:
# - Kyverno: deploy privileged pod → blocked ✅
# - Falco: kubectl exec → alert fired ✅
# - Harbor: push image with critical CVE → scan detected ✅
# - NetworkPolicy: cross-namespace access → blocked ✅

---

PART 5: CAPSTONE EVALUATION CHECKLIST

#	Requirement_	Points_	_Status
1	K8s HA cluster (3 masters, 4 workers)	10	☐
2	Rook-Ceph distributed storage	10	☐
3	PostgreSQL HA + automated backup	10	☐
4	Message queues (RabbitMQ/Kafka)	5	☐
5	Redis caching layer	5	☐
6	Istio service mesh + mTLS	10	☐
7	ArgoCD GitOps deployment	10	☐
8	CI/CD pipeline (build + scan + sign)	5	☐
9	Prometheus + Grafana monitoring	_5	☐
10_	Loki centralized logging	5	☐
11	Tempo distributed tracing	5	☐
12	SLO/Error budget alerts	5	☐
13	Security (RBAC + Kyverno + Falco)	5	☐
14	Velero backup + DR tested	5	☐
15	Load test passing (500 orders/min)	5	☐
	Total	100

---

💡 KEY TAKEAWAYS

1. Capstone: Integrate all 49 previous articles into one practical system
2. GitOps: Entire infrastructure + apps as code
3. Production-ready: Security, observability, reliability, DR
4. Testing: Integration + load + chaos + security validation
5. Operations: SLO monitoring, incident response, capacity planning

---

🎓 END OF COURSE

Congratulations on completing 50 lessons__HTMLTAG_386___ of the course "Deploy Microservices On-Premises with Kubernetes HA"!

You have mastered:

• ☑️ Infrastructure planning & Linux system tuning__HTMLTAG_392___
• ☑️ Kubernetes HA cluster setup & operations
• ☑️ Distributed storage (Rook-Ceph)
• ☑️ Database HA (PostgreSQL, Redis, RabbitMQ, Kafka)
• ☑️ Service Mesh (Istio) & API Gateway
• ☑️ GitOps (ArgoCD) & CI/CD & Secrets Management
• ☑️ Full observability stack (Prometheus, Loki, Tempo, Grafana)
• ☑️ Security hardening (RBAC, Kyverno, Falco, Harbor)
• ☑️ Deployment patterns & Auto-scaling
• ☑️ Disaster Recovery & Chaos Engineering
• ☑️ Production operations & Troubleshooting__HTMLTAG_412___

Keep learning, keep building! 🚀