Chuyển đến nội dung chính
BlogCoursesHL7Cert PrepRoadmapGameDomainAboutMCPNEW
Topics
DatabaseDevOpsKiến trúc hệ thốngLinuxProgrammingSecurity

LESSON 9: METALLB — LOADBALANCER FOR ON-PREMISES

Installing MetalLB provides LoadBalancer IP for services in the on-premises cluster, configures L2 mode and BGP mode, IPAddressPool, and exposes services to the external network.

🔒 DevSecOps — Lesson 9 LESSON 9: METALLB — LOADBALANCER FOR ON-PREMISES

Deploy Microservices On-Premises with Kubernetes HA

Part 2: Kubernetes HA Cluster with kubeadm__HTMLTAG_62___

xdev.asia

🎯 LESSON OBJECTIVE__HTMLTAG_68___

After completing this lesson, you will:

  • ✅ Understand the LoadBalancer problem in on-premises (no cloud LB)
  • ✅ Install MetalLB with Helm
  • ✅ Configure L2 mode with IPAddressPool
  • ✅ Understand BGP mode for large datacenter__HTMLTAG_79___
  • ✅ Expose services with type LoadBalancer

PART 1: LOADBALANCER PROBLEM ON ON-PREMISES

1.1. Cloud vs On-Premises


Cloud (AWS/GCP/Azure):
┌─────────┐     ┌──────────────────┐     ┌──────────────────┐
│  User   │────►│  Cloud LB (ELB)  │────►│  K8s Service     │
│         │     │  (tự động tạo)   │     │  type:LoadBalancer│
└─────────┘     └──────────────────┘     └──────────────────┘
                ✅ Tự động provisioning    ✅ External IP tự gán

On-Premises (KHÔNG có MetalLB):
┌─────────┐     ┌──────────────────┐     ┌──────────────────┐
│  User   │────►│  ???             │────►│  K8s Service     │
│         │     │  Không có LB!    │     │  type:LoadBalancer│
└─────────┘     └──────────────────┘     │  ⚠️ PENDING...   │
└──────────────────┘
❌ External IP = <pending>  ← Mãi mãi pending!


On-Premises (CÓ MetalLB):
┌─────────┐     ┌──────────────────┐     ┌──────────────────┐
│  User   │────►│  MetalLB         │────►│  K8s Service     │
│         │     │  (announce IP)   │     │  type:LoadBalancer│
└─────────┘     └──────────────────┘     │  ✅ 10.10.40.201  │
✅ MetalLB gán IP          └──────────────────┘

PART 2: METALLB INSTALL

2.1. Prerequisites

# MetalLB yêu cầu kube-proxy strictARP (đã cấu hình ở Bài 6):
# Verify:
kubectl get configmap kube-proxy -n kube-system -o yaml | grep strictARP
# strictARP: true  ← ✅ OK

Nếu đã xóa kube-proxy (dùng Cilium replacement):


→ Không cần verify, Cilium handle ARP

2.2. Install MetalLB using Helm

# Add MetalLB Helm repo:
helm repo add metallb https://metallb.universe.tf
helm repo update

Install MetalLB:


helm install metallb metallb/metallb 

--namespace metallb-system 

--create-namespace 

--set speaker.frr.enabled=true


Verify installation:


kubectl -n metallb-system get pods


NAME                                  READY   STATUS    RESTARTS   AGE


metallb-controller-xxxxx-xxxxx        1/1     Running   0          30s


metallb-speaker-xxxxx                 4/4     Running   0          30s  (DaemonSet)


metallb-speaker-xxxxx                 4/4     Running   0          30s


metallb-speaker-xxxxx                 4/4     Running   0          30s


...

PART 3: L2 MODE CONFIGURATION

3.1. IPAddressPool

# metallb-config.yaml:
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: external-pool
  namespace: metallb-system
spec:
  addresses:
    - 10.10.40.200-10.10.40.250         # 51 IPs cho services
  autoAssign: true                       # Tự động gán IP
  avoidBuggyIPs: true                    # Bỏ qua .0 và .255



apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: internal-pool
namespace: metallb-system
spec:
addresses:
- 10.10.20.200-10.10.20.220         # Internal services
autoAssign: false                      # Phải specify manually




apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: l2-advertisement
namespace: metallb-system
spec:
ipAddressPools:
- external-pool
- internal-pool
nodeSelectors:
- matchLabels:
node-role.kubernetes.io/worker: ""   # Chỉ announce từ worker nodes


kubectl apply -f metallb-config.yaml
# ipaddresspool.metallb.io/external-pool created
# ipaddresspool.metallb.io/internal-pool created
# l2advertisement.metallb.io/l2-advertisement created

3.2. How does L2 Mode work?


L2 Mode (ARP/NDP):
┌─────────────┐                    ┌─────────────────┐
│  Client     │  ARP: Who has     │  MetalLB        │
│  (external) │  10.10.40.201?    │  Speaker         │
│             │──────────────────►│  (trên worker1)  │
│             │                    │  "Tôi có!"       │
│             │◄──────────────────│  ARP Reply       │
│             │                    └─────────────────┘
│             │  Traffic:
│             │──────────────────►  worker1 → kube-proxy/Cilium → Pod
└─────────────┘

✅ Đơn giản, không cần router hỗ trợ
❌ Failover chậm hơn BGP (~10 giây)
❌ Một node handle tất cả traffic cho 1 IP (không true LB)

PART 4: TEST LOADBALANCER SERVICE

4.1. Deploy test application

# Deploy nginx test:
kubectl create deployment nginx-test --image=nginx:alpine --replicas=3

Expose với type LoadBalancer:


kubectl expose deployment nginx-test 

--port=80 

--target-port=80 

--type=LoadBalancer


Kiểm tra service:


kubectl get svc nginx-test


NAME         TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)        AGE


nginx-test   LoadBalancer   10.96.xxx.xx   10.10.40.200   80:3xxxx/TCP   10s


↑ MetalLB assigned! ✅

4.2. Test access from external

# Từ máy trong cùng network:
curl http://10.10.40.200
# 
# 
# Welcome to nginx!
# ...  ← OK! ✅

Kiểm tra MetalLB speaker IP assignment:


kubectl -n metallb-system get ipaddresspool


NAME            AUTO ASSIGN   AVOID BUGGY IPS   ADDRESSES


external-pool   true          true              10.10.40.200-10.10.40.250

4.3. Specify specific IP

# Service với IP cụ thể:
apiVersion: v1
kind: Service
metadata:
  name: my-app
  annotations:
    metallb.universe.tf/address-pool: internal-pool   # Chọn pool
spec:
  type: LoadBalancer
  loadBalancerIP: 10.10.20.210                         # IP cụ thể
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080

PART 5: BGP MODE (LARGE DATACENTER)

5.1. When to use BGP?

Criteria L2 Mode BGP Mode
Router request No Needs BGP-capable router__HTMLTAG_135___
Failover ~10 seconds__HTMLTAG_141___ ~3 seconds
True load balancing ❌ (1 node handle) ✅ ECMP across nodes__HTMLTAG_151___
Cross-subnet ❌ Same L2 segment__HTMLTAG_157___ ✅ Works across subnets__HTMLTAG_159___
Complexity Simple__HTMLTAG_165___ Need router config
Best for SMB, single rack Large DC, multi-rack

5.2. BGP Configuration Example

# metallb-bgp.yaml (chỉ dùng khi có BGP router):
apiVersion: metallb.io/v1beta2
kind: BGPPeer
metadata:
  name: tor-switch
  namespace: metallb-system
spec:
  myASN: 64512                          # K8s cluster ASN
  peerASN: 64501                        # ToR switch ASN
  peerAddress: 10.10.20.1               # Router IP
  holdTime: 90s
  keepaliveTime: 30s
  nodeSelectors:
    - matchLabels:
        node-role.kubernetes.io/worker: ""



apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
name: bgp-advertisement
namespace: metallb-system
spec:
ipAddressPools:
- external-pool
localPref: 100
communities:
- 64512:100

PART 6: IP SHARING AND ADVANCED FEATURES

6.1. Shared IP (many services use 1 IP)

# Nhiều services share cùng 1 external IP (khác port):
apiVersion: v1
kind: Service
metadata:
  name: web-http
  annotations:
    metallb.universe.tf/allow-shared-ip: "shared-web"
spec:
  type: LoadBalancer
  loadBalancerIP: 10.10.40.210
  selector:
    app: web
  ports:
    - port: 80
---
apiVersion: v1
kind: Service
metadata:
  name: web-https
  annotations:
    metallb.universe.tf/allow-shared-ip: "shared-web"
spec:
  type: LoadBalancer
  loadBalancerIP: 10.10.40.210          # Same IP!
  selector:
    app: web
  ports:
    - port: 443

6.2. Cleanup test

kubectl delete deployment nginx-test
kubectl delete svc nginx-test

💡 KEY TAKEAWAYS

  1. MetalLB resolves LoadBalancer for on-premises — no cloud provider needed__HTMLTAG_196___
  2. L2 Mode Simple, suitable for single rack/small cluster
  3. BGP Mode for large, multi-rack datacenters with ECMP true load balancing
  4. IPAddressPool manages IP ranges, can create many pools (external, internal)
  5. IP Sharing allows multiple services to share the same external IP
  6. strictARP: true in kube-proxy is a requirement for L2 mode

🎯 EXERCISE

Exercise 1: Install MetalLB

  • Install MetalLB with Helm
  • Create IPAddressPool with appropriate IP range lab
  • Deploy service type LoadBalancer, verify external IP

Exercise 2: Multiple pools__HTMLTAG_232___
  • Create 2 pools: external-pool and internal-pool
  • Deploy 2 services, each service uses a different pool
  • Test IP sharing with 2 services with the same IP port

📚 NEXT POST

In Lesson 10: etcd — Operations, Backup and Disaster Recovery, we will deep dive into etcd operations, backup strategies, and restore procedures.