1. はじめに
サーキットブレーカー (サーキット ブレーカー) は、実際のサーキット ブレーカー デバイスからインスピレーションを得た、マイクロサービス アーキテクチャにおける重要な設計パターンです。サーキット ブレーカーが過負荷を検出すると自動的に回路を遮断して電気システムを保護するのと同じように、ソフトウェアのサーキット ブレーカーは、問題が発生しているサービスへのリクエストを自動的に中断してシステム全体を保護します。
このパターンはどのような問題を解決しますか?
マイクロサービス アーキテクチャでは、サービスは相互に依存します。サービスがクラッシュしたとき (遅いか応答しない)、保護メカニズムがない場合:
[Service A] ---> [Service B (đang chết)] ---> timeout 30s
↓
Threads bị block
↓
Resource exhaustion
↓
Service A cũng chết (Cascading Failure)
サーキット ブレーカーは、次の方法でこのドミノ効果を防ぎます。 早く失敗する - 無駄に待つのではなく、すぐにエラーを返します。
2. なぜサーキットブレーカーが必要なのでしょうか?
2.1.カスケード障害の問題
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Gateway │────▶│ Order Svc │────▶│ Payment Svc │ ← Đang chết
└─────────────┘ └─────────────┘ └─────────────┘
│
▼
┌─────────────┐
│Inventory Svc│
└─────────────┘
決済サービスが停止した場合:
- Order Service が応答を待機する → スレッドがブロックされる
- Order Service へのリクエストが増加 → スレッド プールが枯渇する
- Order Service が新しいリクエストを処理できない → これも「停止」
- ゲートウェイのタイムアウト → ユーザー エクスペリエンスが悪い
2.2.サーキットブレーカーの利点
| 利点 | 説明 |
|---|---|
| フェイルファスト | タイムアウトを待たずに、ただちにエラーを返します。 |
| リソースを保護する | 無料のスレッド、接続 |
| 自動回復 | サービス再開時のセルフチェックと復元 |
| グレースフル デグラデーション | 完全なエラーの代わりにフォールバック応答を提供します |
| モニタリング | 依存関係の健全性に関するメトリクスを提供します |
3. 動作原理
3.1.サーキットブレーカーの 3 つの状態
┌──────────────────────────────────────┐
│ │
▼ │
┌─────────┐ failure ┌─────────┐ wait timeout ┌─────────────┐
│ CLOSED │───────────────▶│ OPEN │──────────────────▶│ HALF_OPEN │
│(Đóng) │ threshold │ (Mở) │ │ (Nửa mở) │
└─────────┘ └─────────┘ └─────────────┘
▲ ▲ │
│ │ │
│ success │ failure │
└──────────────────────────┴──────────────────────────────┘
CLOSED (閉) - 通常の状態
- すべてのリクエストが許可されます
- サーキットブレーカーはエラー率を監視します
- 故障率が閾値を超えた場合→OPENに切り替える
OPEN (オープン) - 保護ステータス
- すべてのリクエストは即座に拒否されます
- フォールバック応答または例外を返します
- 一定時間経過後(待機時間) → HALF_OPENに切り替わります
HALF_OPEN (ハーフオープン) - テストステータス
- テストのために一定数のリクエストの通過を許可する
- 成功した場合 → CLOSEDに戻る
- 失敗した場合→OPENに戻る
3.2.引き違い窓
Circuit Breaker はスライディング ウィンドウを使用して故障率を計算します。
カウントベースのスライディング ウィンドウ:
[Request 1: ✓] [Request 2: ✗] [Request 3: ✓] [Request 4: ✗] [Request 5: ✗]
↑
Failure rate = 3/5 = 60%
時間ベースのスライディング ウィンドウ:
|-------- 10 seconds --------|
| ✓ ✗ ✓ ✗ ✗ ✓ ✗ ✗ ✓ ✓ |
| Failure rate = 5/10 = 50% |
4. Spring Boot と Resilience4j を使用してインストールする
4.1. Resilience4j を選ぶ理由?
Hystrix (Netflix) は 2018 年から非推奨になりました。 レジリエンス4j 推奨されるライブラリは次のとおりです。
- 軽量、推移的な依存関係なし
- 関数型プログラミングを使用した Java 8 以降向けに設計
- Spring Bootとうまく統合
- リアクティブサポート (Project Reactor、RxJava)
4.2.依存関係
<!-- pom.xml --> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>2023.0.3</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement><dependencies> <!-- Spring Boot Starter --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency>
<!-- Resilience4j Circuit Breaker --> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId> </dependency> <!-- AOP cho annotations --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-aop</artifactId> </dependency> <!-- Actuator cho monitoring --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency>
</dependencies>
または Gradle を使用する場合:
// build.gradle ext { springCloudVersion = "2023.0.3" }dependencies { implementation 'org.springframework.boot:spring-boot-starter-web' implementation 'org.springframework.cloud:spring-cloud-starter-circuitbreaker-resilience4j' implementation 'org.springframework.boot:spring-boot-starter-aop' implementation 'org.springframework.boot:spring-boot-starter-actuator' }
dependencyManagement { imports { mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}" } }
5. 詳細な構成
5.1. application.yml による設定
# application.yml resilience4j: circuitbreaker: configs: # Cấu hình mặc định cho tất cả circuit breakers default: # Số lượng calls trong sliding window để tính failure rate slidingWindowSize: 10# Loại sliding window: COUNT_BASED hoặc TIME_BASED slidingWindowType: COUNT_BASED # Số calls tối thiểu trước khi tính failure rate minimumNumberOfCalls: 5 # Tỷ lệ lỗi (%) để chuyển sang OPEN failureRateThreshold: 50 # Tỷ lệ slow calls (%) để chuyển sang OPEN slowCallRateThreshold: 100 # Thời gian được coi là slow call (ms) slowCallDurationThreshold: 2000 # Thời gian ở trạng thái OPEN trước khi chuyển sang HALF_OPEN waitDurationInOpenState: 30s # Số calls được phép trong trạng thái HALF_OPEN permittedNumberOfCallsInHalfOpenState: 3 # Tự động chuyển từ OPEN sang HALF_OPEN automaticTransitionFromOpenToHalfOpenEnabled: true # Các exception được ghi nhận là failure recordExceptions: - java.io.IOException - java.net.ConnectException - java.util.concurrent.TimeoutException - org.springframework.web.client.HttpServerErrorException # Các exception KHÔNG được ghi nhận là failure ignoreExceptions: - com.example.BusinessException # Cấu hình cho từng circuit breaker cụ thể instances: # Circuit breaker cho Payment Service paymentService: baseConfig: default failureRateThreshold: 30 waitDurationInOpenState: 20s slidingWindowSize: 20 # Circuit breaker cho Inventory Service inventoryService: baseConfig: default failureRateThreshold: 60 slowCallDurationThreshold: 3000Actuator endpoints
management: endpoints: web: exposure: include: health,circuitbreakers,circuitbreakerevents health: circuitbreakers: enabled: true endpoint: health: show-details: always
5.2.重要なパラメータの説明
| パラメータ | 説明 | 推奨値 |
|---|---|---|
スライドウィンドウサイズ |
失敗率を計算するための呼び出し数 | 交通状況に応じて 10 ～ 100 |
スライドウィンドウタイプ |
COUNT_BASED または TIME_BASED | トラフィックが少ない場合は COUNT_BASED |
最小通話数 |
評価前の最小呼び出し数 | 5-10 |
失敗率しきい値 |
開路エラー% | 50%が一般的です |
OpenState での waitDuration |
再試行前のタイムアウト | 30代～60代 |
半分オープン状態で許可された通話数 |
HALF_OPENのテストコールの数 | 3-10 |
6. 実践例
6.1.プロジェクトの構造
order-service/
├── src/main/java/com/example/order/
│ ├── OrderServiceApplication.java
│ ├── config/
│ │ └── Resilience4jConfig.java
│ ├── controller/
│ │ └── OrderController.java
│ ├── service/
│ │ ├── OrderService.java
│ │ └── PaymentServiceClient.java
│ ├── dto/
│ │ ├── OrderRequest.java
│ │ ├── OrderResponse.java
│ │ └── PaymentResponse.java
│ └── exception/
│ └── ServiceUnavailableException.java
├── src/main/resources/
│ └── application.yml
└── pom.xml
6.2.サーキットブレーカーを備えた決済サービスクライアント
package com.example.order.service;import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker; import io.github.resilience4j.retry.annotation.Retry; import io.github.resilience4j.timelimiter.annotation.TimeLimiter; import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; import org.springframework.stereotype.Service; import org.springframework.web.client.RestTemplate;
import java.util.concurrent.CompletableFuture;
@Service @RequiredArgsConstructor @Slf4j public class PaymentServiceClient {
private final RestTemplate restTemplate; private static final String PAYMENT_SERVICE_URL = "http://payment-service:8080"; /** * Gọi Payment Service với Circuit Breaker bảo vệ * * Thứ tự xử lý: Retry -> CircuitBreaker -> TimeLimiter -> Bulkhead */ @CircuitBreaker(name = "paymentService", fallbackMethod = "processPaymentFallback") @Retry(name = "paymentService") @TimeLimiter(name = "paymentService") public CompletableFuture<PaymentResponse> processPayment(PaymentRequest request) { log.info("Calling Payment Service for order: {}", request.getOrderId()); return CompletableFuture.supplyAsync(() -> { PaymentResponse response = restTemplate.postForObject( PAYMENT_SERVICE_URL + "/api/payments", request, PaymentResponse.class ); log.info("Payment processed successfully: {}", response); return response; }); } /** * Fallback method khi Circuit Breaker OPEN hoặc có lỗi * * Phải có cùng parameters + thêm Exception/Throwable */ public CompletableFuture<PaymentResponse> processPaymentFallback( PaymentRequest request, Throwable throwable) { log.warn("Payment Service unavailable. Triggering fallback for order: {}. Error: {}", request.getOrderId(), throwable.getMessage()); // Option 1: Trả về response mặc định PaymentResponse fallbackResponse = PaymentResponse.builder() .orderId(request.getOrderId()) .status("PENDING") .message("Payment will be processed when service is available") .fallback(true) .build(); // Option 2: Có thể queue lại để xử lý sau // paymentQueue.add(request); return CompletableFuture.completedFuture(fallbackResponse); } /** * Ví dụ với synchronous call (không dùng TimeLimiter) */ @CircuitBreaker(name = "paymentService", fallbackMethod = "getPaymentStatusFallback") @Retry(name = "paymentService") public PaymentResponse getPaymentStatus(String paymentId) { log.info("Getting payment status for: {}", paymentId); return restTemplate.getForObject( PAYMENT_SERVICE_URL + "/api/payments/" + paymentId, PaymentResponse.class ); } public PaymentResponse getPaymentStatusFallback(String paymentId, Throwable throwable) { log.warn("Cannot get payment status for: {}. Error: {}", paymentId, throwable.getMessage()); return PaymentResponse.builder() .paymentId(paymentId) .status("UNKNOWN") .message("Service temporarily unavailable") .fallback(true) .build(); }
}
6.3.オーダーサービス
package com.example.order.service;import io.github.resilience4j.circuitbreaker.CircuitBreaker; import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry; import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; import org.springframework.stereotype.Service;
@Service @RequiredArgsConstructor @Slf4j public class OrderService {
private final PaymentServiceClient paymentClient; private final InventoryServiceClient inventoryClient; private final OrderRepository orderRepository; private final CircuitBreakerRegistry circuitBreakerRegistry; public OrderResponse createOrder(OrderRequest request) { log.info("Creating order: {}", request); // 1. Kiểm tra inventory InventoryResponse inventory = inventoryClient.checkInventory(request.getProductId()); if (!inventory.isAvailable()) { throw new InsufficientInventoryException("Product not available"); } // 2. Tạo order với status PENDING Order order = Order.builder() .customerId(request.getCustomerId()) .productId(request.getProductId()) .quantity(request.getQuantity()) .status(OrderStatus.PENDING) .build(); order = orderRepository.save(order); // 3. Xử lý payment (có Circuit Breaker bảo vệ) try { PaymentResponse payment = paymentClient.processPayment( PaymentRequest.builder() .orderId(order.getId()) .amount(request.getAmount()) .build() ).get(); // Blocking call if (payment.isFallback()) { // Payment đang trong queue, cần xử lý sau order.setStatus(OrderStatus.PAYMENT_PENDING); } else if ("SUCCESS".equals(payment.getStatus())) { order.setStatus(OrderStatus.CONFIRMED); } else { order.setStatus(OrderStatus.PAYMENT_FAILED); } } catch (Exception e) { log.error("Payment processing failed", e); order.setStatus(OrderStatus.PAYMENT_PENDING); } order = orderRepository.save(order); return OrderResponse.from(order); } /** * Kiểm tra trạng thái Circuit Breaker programmatically */ public CircuitBreakerStatus getCircuitBreakerStatus(String name) { CircuitBreaker circuitBreaker = circuitBreakerRegistry.circuitBreaker(name); CircuitBreaker.Metrics metrics = circuitBreaker.getMetrics(); return CircuitBreakerStatus.builder() .name(name) .state(circuitBreaker.getState().name()) .failureRate(metrics.getFailureRate()) .slowCallRate(metrics.getSlowCallRate()) .numberOfBufferedCalls(metrics.getNumberOfBufferedCalls()) .numberOfFailedCalls(metrics.getNumberOfFailedCalls()) .numberOfSuccessfulCalls(metrics.getNumberOfSuccessfulCalls()) .numberOfSlowCalls(metrics.getNumberOfSlowCalls()) .build(); }
}
6.4.プログラムによる構成 (YAML の代替)
package com.example.order.config;import io.github.resilience4j.circuitbreaker.CircuitBreaker; import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig; import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry; import io.github.resilience4j.common.circuitbreaker.configuration.CircuitBreakerConfigCustomizer; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration;
import java.io.IOException; import java.net.ConnectException; import java.time.Duration; import java.util.concurrent.TimeoutException;
@Configuration public class Resilience4jConfig {
@Bean public CircuitBreakerConfigCustomizer paymentServiceCustomizer() { return CircuitBreakerConfigCustomizer.of("paymentService", builder -> builder .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED) .slidingWindowSize(10) .minimumNumberOfCalls(5) .failureRateThreshold(50) .slowCallRateThreshold(80) .slowCallDurationThreshold(Duration.ofSeconds(2)) .waitDurationInOpenState(Duration.ofSeconds(30)) .permittedNumberOfCallsInHalfOpenState(3) .automaticTransitionFromOpenToHalfOpenEnabled(true) .recordExceptions( IOException.class, ConnectException.class, TimeoutException.class ) ); } /** * Tạo Circuit Breaker programmatically với custom config */ @Bean public CircuitBreaker customCircuitBreaker() { CircuitBreakerConfig config = CircuitBreakerConfig.custom() .slidingWindowSize(20) .failureRateThreshold(40) .waitDurationInOpenState(Duration.ofSeconds(60)) .permittedNumberOfCallsInHalfOpenState(5) .build(); CircuitBreakerRegistry registry = CircuitBreakerRegistry.of(config); CircuitBreaker circuitBreaker = registry.circuitBreaker("customService"); // Đăng ký event listeners circuitBreaker.getEventPublisher() .onStateTransition(event -> System.out.println("State transition: " + event.getStateTransition())) .onFailureRateExceeded(event -> System.out.println("Failure rate exceeded: " + event.getFailureRate())) .onCallNotPermitted(event -> System.out.println("Call not permitted")) .onError(event -> System.out.println("Error: " + event.getThrowable().getMessage())); return circuitBreaker; }
}
6.5. WebClient で使用する (リアクティブ)
package com.example.order.service;import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker; import io.github.resilience4j.reactor.circuitbreaker.operator.CircuitBreakerOperator; import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; import org.springframework.stereotype.Service; import org.springframework.web.reactive.function.client.WebClient; import reactor.core.publisher.Mono;
@Service @RequiredArgsConstructor @Slf4j public class PaymentServiceReactiveClient {
private final WebClient webClient; private final io.github.resilience4j.circuitbreaker.CircuitBreaker circuitBreaker; /** * Cách 1: Sử dụng annotation */ @CircuitBreaker(name = "paymentService", fallbackMethod = "processPaymentFallback") public Mono<PaymentResponse> processPayment(PaymentRequest request) { return webClient.post() .uri("/api/payments") .bodyValue(request) .retrieve() .bodyToMono(PaymentResponse.class) .doOnSuccess(response -> log.info("Payment successful: {}", response)) .doOnError(error -> log.error("Payment failed: {}", error.getMessage())); } public Mono<PaymentResponse> processPaymentFallback(PaymentRequest request, Throwable t) { log.warn("Fallback triggered for payment: {}", request.getOrderId()); return Mono.just(PaymentResponse.builder() .orderId(request.getOrderId()) .status("PENDING") .fallback(true) .build()); } /** * Cách 2: Sử dụng operator programmatically */ public Mono<PaymentResponse> processPaymentWithOperator(PaymentRequest request) { return webClient.post() .uri("/api/payments") .bodyValue(request) .retrieve() .bodyToMono(PaymentResponse.class) .transformDeferred(CircuitBreakerOperator.of(circuitBreaker)) .onErrorResume(throwable -> { log.warn("Circuit breaker triggered fallback"); return Mono.just(PaymentResponse.builder() .status("PENDING") .fallback(true) .build()); }); }
}
6.6.サーキットブレーカーステータスを備えた REST コントローラー
package com.example.order.controller;import io.github.resilience4j.circuitbreaker.CircuitBreaker; import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry; import lombok.RequiredArgsConstructor; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.*;
import java.util.HashMap; import java.util.Map;
@RestController @RequestMapping("/api/orders") @RequiredArgsConstructor public class OrderController {
private final OrderService orderService; private final CircuitBreakerRegistry circuitBreakerRegistry; @PostMapping public ResponseEntity<OrderResponse> createOrder(@RequestBody OrderRequest request) { OrderResponse response = orderService.createOrder(request); return ResponseEntity.ok(response); } /** * Endpoint kiểm tra trạng thái Circuit Breakers */ @GetMapping("/circuit-breakers/status") public ResponseEntity<Map<String, Object>> getCircuitBreakersStatus() { Map<String, Object> status = new HashMap<>(); circuitBreakerRegistry.getAllCircuitBreakers().forEach(cb -> { CircuitBreaker.Metrics metrics = cb.getMetrics(); Map<String, Object> cbStatus = new HashMap<>(); cbStatus.put("state", cb.getState().name()); cbStatus.put("failureRate", metrics.getFailureRate()); cbStatus.put("slowCallRate", metrics.getSlowCallRate()); cbStatus.put("bufferedCalls", metrics.getNumberOfBufferedCalls()); cbStatus.put("failedCalls", metrics.getNumberOfFailedCalls()); cbStatus.put("successfulCalls", metrics.getNumberOfSuccessfulCalls()); cbStatus.put("notPermittedCalls", metrics.getNumberOfNotPermittedCalls()); status.put(cb.getName(), cbStatus); }); return ResponseEntity.ok(status); } /** * Endpoint để manually transition Circuit Breaker * (Chỉ dùng cho testing/debugging) */ @PostMapping("/circuit-breakers/{name}/transition/{state}") public ResponseEntity<String> transitionCircuitBreaker( @PathVariable String name, @PathVariable String state) { CircuitBreaker circuitBreaker = circuitBreakerRegistry.circuitBreaker(name); switch (state.toUpperCase()) { case "OPEN": circuitBreaker.transitionToOpenState(); break; case "CLOSED": circuitBreaker.transitionToClosedState(); break; case "HALF_OPEN": circuitBreaker.transitionToHalfOpenState(); break; default: return ResponseEntity.badRequest().body("Invalid state"); } return ResponseEntity.ok("Transitioned to " + state); }
}
7. サーキットブレーカーはいつ適用する必要がありますか?
7.1.次の場合にはサーキットブレーカーを使用すべきです。
✅ 外部サービスに電話する
// Gọi Payment Gateway (Stripe, VNPay, MoMo) @CircuitBreaker(name = "paymentGateway") public PaymentResult processPayment(PaymentRequest request) { return paymentGatewayClient.charge(request); }
// Gọi SMS/Email Provider @CircuitBreaker(name = "notificationService") public void sendNotification(NotificationRequest request) { twilioClient.sendSMS(request); }
✅ マイクロサービス間の通信
// Order Service → Inventory Service @CircuitBreaker(name = "inventoryService") public InventoryStatus checkStock(String productId) { return inventoryClient.getStock(productId); }
// User Service → Auth Service @CircuitBreaker(name = "authService") public TokenInfo validateToken(String token) { return authClient.validate(token); }
✅ ネットワーク経由でデータベースにアクセス
// Database cluster có thể unavailable
@CircuitBreaker(name = "databaseOperation")
public List<Order> getOrderHistory(String customerId) {
return orderRepository.findByCustomerId(customerId);
}
✅ サードパーティ API を呼び出す
// Weather API, Exchange Rate API, Social Login
@CircuitBreaker(name = "weatherApi")
public WeatherData getCurrentWeather(String location) {
return weatherApiClient.fetch(location);
}
7.2.次の場合にはサーキットブレーカーを使用しないでください。
❌ 内部操作にはネットワーク呼び出しはありません
// Tính toán local - KHÔNG cần Circuit Breaker public BigDecimal calculateDiscount(Order order) { return order.getTotal().multiply(DISCOUNT_RATE); }
// Validate input - KHÔNG cần Circuit Breaker public boolean validateEmail(String email) { return EMAIL_PATTERN.matcher(email).matches(); }
❌ タイムアウトとリトライは十分です
// Nếu đã có cơ chế retry với exponential backoff
// và timeout hợp lý, có thể không cần thêm Circuit Breaker
❌ 重要な操作は失敗しない
// Ví dụ: Ghi log audit bắt buộc
// Không nên dùng Circuit Breaker vì không thể skip
7.3.意思決定マトリックス
| 状況 | サーキットブレーカー? | 理由 |
|---|---|---|
| 通話決済サービス | ✅ はい | 外部サービス、ダウンロード可能 |
| 内部キャッシュを呼び出す | ❌ いいえ | ローカル、ネットワーク遅延なし |
| データベースのプライマリを呼び出す | ⚠️検討してください | 設定に応じて、通常は接続プールが存在します。 |
| メッセージキューの呼び出し | ✅ はい | ネットワーク呼び出し、タイムアウトの可能性あり |
| ローカルファイルを読み取る | ❌ いいえ | ローカル I/O、不要 |
| OAuthプロバイダーを呼び出す | ✅ はい | 外部サービス、重要 |
| CPU 負荷の高いコンピューティング | ❌ いいえ | ネットワークの問題ではない |
7.4.他のパターンと組み合わせる
# Thứ tự áp dụng (từ ngoài vào trong): # Retry -> CircuitBreaker -> RateLimiter -> TimeLimiter -> Bulkheadresilience4j: retry: instances: paymentService: maxAttempts: 3 waitDuration: 1s retryExceptions: - java.io.IOException
circuitbreaker: instances: paymentService: failureRateThreshold: 50 waitDurationInOpenState: 30s
ratelimiter: instances: paymentService: limitForPeriod: 100 limitRefreshPeriod: 1s
timelimiter: instances: paymentService: timeoutDuration: 5s
bulkhead: instances: paymentService: maxConcurrentCalls: 10 maxWaitDuration: 500ms
8. 監視と可観測性
8.1.アクチュエータのエンドポイント
# application.yml
management:
endpoints:
web:
exposure:
include: health,circuitbreakers,circuitbreakerevents
health:
circuitbreakers:
enabled: true
endpoint:
health:
show-details: always
利用可能なエンドポイント:
# Xem tất cả circuit breakers GET /actuator/circuitbreakersXem events của circuit breaker cụ thể
GET /actuator/circuitbreakerevents GET /actuator/circuitbreakerevents/{name}
Health check bao gồm circuit breaker status
GET /actuator/health
応答例:
// GET /actuator/circuitbreakers
{
"circuitBreakers": {
"paymentService": {
"failureRate": "25.0%",
"slowCallRate": "10.0%",
"failureRateThreshold": "50.0%",
"slowCallRateThreshold": "100.0%",
"bufferedCalls": 20,
"failedCalls": 5,
"slowCalls": 2,
"slowFailedCalls": 1,
"notPermittedCalls": 0,
"state": "CLOSED"
}
}
}
8.2. Prometheus + Grafana の統合
<!-- Thêm dependency -->
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
# application.yml
management:
endpoints:
web:
exposure:
include: health,prometheus,circuitbreakers
prometheus:
metrics:
export:
enabled: true
利用可能なメトリクス:
# Số lượng calls theo trạng thái resilience4j_circuitbreaker_calls_total{name="paymentService",kind="successful"} 150 resilience4j_circuitbreaker_calls_total{name="paymentService",kind="failed"} 10 resilience4j_circuitbreaker_calls_total{name="paymentService",kind="not_permitted"} 5Trạng thái circuit breaker (0=CLOSED, 1=OPEN, 2=HALF_OPEN)
resilience4j_circuitbreaker_state{name="paymentService",state="closed"} 1
Failure rate
resilience4j_circuitbreaker_failure_rate{name="paymentService"} 6.25
Slow call rate
resilience4j_circuitbreaker_slow_call_rate{name="paymentService"} 2.5
8.3.カスタム イベント ハンドラー
package com.example.order.config;import io.github.resilience4j.circuitbreaker.CircuitBreaker; import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry; import io.github.resilience4j.circuitbreaker.event.*; import lombok.extern.slf4j.Slf4j; import org.springframework.context.annotation.Configuration;
import jakarta.annotation.PostConstruct;
@Configuration @Slf4j public class CircuitBreakerEventConfig {
private final CircuitBreakerRegistry registry; private final AlertService alertService; public CircuitBreakerEventConfig(CircuitBreakerRegistry registry, AlertService alertService) { this.registry = registry; this.alertService = alertService; } @PostConstruct public void registerEventConsumers() { registry.getAllCircuitBreakers().forEach(this::registerEventConsumer); // Đăng ký cho các circuit breakers được tạo sau registry.getEventPublisher() .onEntryAdded(event -> registerEventConsumer(event.getAddedEntry())); } private void registerEventConsumer(CircuitBreaker circuitBreaker) { circuitBreaker.getEventPublisher() .onStateTransition(this::handleStateTransition) .onFailureRateExceeded(this::handleFailureRateExceeded) .onCallNotPermitted(this::handleCallNotPermitted) .onError(this::handleError) .onSuccess(this::handleSuccess); } private void handleStateTransition(CircuitBreakerOnStateTransitionEvent event) { String message = String.format( "Circuit Breaker '%s' transitioned from %s to %s", event.getCircuitBreakerName(), event.getStateTransition().getFromState(), event.getStateTransition().getToState() ); log.warn(message); // Gửi alert khi chuyển sang OPEN if (event.getStateTransition().getToState() == CircuitBreaker.State.OPEN) { alertService.sendAlert( AlertLevel.HIGH, "Circuit Breaker OPENED", message ); } // Gửi notification khi recovery (về CLOSED) if (event.getStateTransition().getToState() == CircuitBreaker.State.CLOSED && event.getStateTransition().getFromState() == CircuitBreaker.State.HALF_OPEN) { alertService.sendNotification( "Circuit Breaker RECOVERED", message ); } } private void handleFailureRateExceeded(CircuitBreakerOnFailureRateExceededEvent event) { log.error("Failure rate exceeded for '{}': {}%", event.getCircuitBreakerName(), event.getFailureRate()); } private void handleCallNotPermitted(CircuitBreakerOnCallNotPermittedEvent event) { log.debug("Call not permitted for '{}'", event.getCircuitBreakerName()); } private void handleError(CircuitBreakerOnErrorEvent event) { log.error("Error in '{}': {}", event.getCircuitBreakerName(), event.getThrowable().getMessage()); } private void handleSuccess(CircuitBreakerOnSuccessEvent event) { log.trace("Successful call to '{}' in {}ms", event.getCircuitBreakerName(), event.getElapsedDuration().toMillis()); }
}
9. ベストプラクティス
9.1.フォールバック戦略の設計
/**
Các chiến lược Fallback phổ biến */ public class FallbackStrategies {
// 1. Trả về giá trị mặc định public ProductInfo getProductFallback(String productId, Throwable t) { return ProductInfo.builder() .id(productId) .name("Product information temporarily unavailable") .available(false) .build(); }
// 2. Trả về dữ liệu từ cache @Autowired private CacheManager cacheManager;
public ProductInfo getProductFromCacheFallback(String productId, Throwable t) { Cache cache = cacheManager.getCache("products"); ProductInfo cached = cache.get(productId, ProductInfo.class);
if (cached != null) { cached.setFromCache(true); return cached; } return getProductFallback(productId, t);}
// 3. Gọi service backup public ProductInfo getProductFromBackupServiceFallback(String productId, Throwable t) { try { return backupProductService.getProduct(productId); } catch (Exception e) { return getProductFromCacheFallback(productId, t); } }
// 4. Queue để xử lý sau (async fallback) @Autowired private RabbitTemplate rabbitTemplate;
public OrderResponse createOrderAsyncFallback(OrderRequest request, Throwable t) { // Lưu vào queue để xử lý sau rabbitTemplate.convertAndSend("order-retry-queue", request);
return OrderResponse.builder() .status("QUEUED") .message("Your order is being processed") .estimatedProcessingTime("5 minutes") .build();}
// 5. Graceful degradation - giảm tính năng public RecommendationResponse getRecommendationsFallback(String userId, Throwable t) { // Thay vì personalized recommendations, trả về popular items return RecommendationResponse.builder() .items(popularItemsCache.getTopItems(10)) .type("POPULAR") // Thay vì "PERSONALIZED" .message("Showing popular items") .build(); } }
9.2.チューニングパラメータ
# Development/Testing environment resilience4j: circuitbreaker: configs: development: slidingWindowSize: 5 # Window nhỏ để test nhanh minimumNumberOfCalls: 3 # Ít calls để trigger sớm failureRateThreshold: 50 waitDurationInOpenState: 10s # Recovery nhanh permittedNumberOfCallsInHalfOpenState: 2Production environment
resilience4j: circuitbreaker: configs: production: slidingWindowSize: 100 # Window lớn hơn, ổn định hơn minimumNumberOfCalls: 20 # Cần nhiều data points failureRateThreshold: 50 slowCallRateThreshold: 80 # Theo dõi slow calls slowCallDurationThreshold: 3s waitDurationInOpenState: 60s # Chờ lâu hơn permittedNumberOfCallsInHalfOpenState: 10
9.3.サーキットブレーカーのテスト
package com.example.order.service;import io.github.resilience4j.circuitbreaker.CircuitBreaker; import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.mock.mockito.MockBean;
import static org.assertj.core.api.Assertions.assertThat; import static org.mockito.ArgumentMatchers.any; import static org.mockito.Mockito.*;
@SpringBootTest class PaymentServiceClientTest {
@Autowired private PaymentServiceClient paymentClient; @Autowired private CircuitBreakerRegistry circuitBreakerRegistry; @MockBean private RestTemplate restTemplate; private CircuitBreaker circuitBreaker; @BeforeEach void setUp() { circuitBreaker = circuitBreakerRegistry.circuitBreaker("paymentService"); circuitBreaker.reset(); // Reset state before each test } @Test void shouldReturnSuccessfulResponse() { // Given PaymentResponse expectedResponse = PaymentResponse.builder() .status("SUCCESS") .build(); when(restTemplate.postForObject(any(), any(), any())) .thenReturn(expectedResponse); // When PaymentResponse response = paymentClient.processPayment( PaymentRequest.builder().orderId("123").build() ).join(); // Then assertThat(response.getStatus()).isEqualTo("SUCCESS"); assertThat(response.isFallback()).isFalse(); assertThat(circuitBreaker.getState()).isEqualTo(CircuitBreaker.State.CLOSED); } @Test void shouldOpenCircuitAfterFailures() { // Given - Circuit breaker có threshold 50%, window size 10 when(restTemplate.postForObject(any(), any(), any())) .thenThrow(new RuntimeException("Service unavailable")); // When - Gọi nhiều lần để trigger circuit breaker for (int i = 0; i < 10; i++) { try { paymentClient.processPayment( PaymentRequest.builder().orderId("123").build() ).join(); } catch (Exception ignored) { } } // Then assertThat(circuitBreaker.getState()).isEqualTo(CircuitBreaker.State.OPEN); } @Test void shouldReturnFallbackWhenCircuitOpen() { // Given circuitBreaker.transitionToOpenState(); // When PaymentResponse response = paymentClient.processPayment( PaymentRequest.builder().orderId("123").build() ).join(); // Then assertThat(response.isFallback()).isTrue(); assertThat(response.getStatus()).isEqualTo("PENDING"); verify(restTemplate, never()).postForObject(any(), any(), any()); } @Test void shouldTransitionToHalfOpenAfterWaitDuration() throws InterruptedException { // Given circuitBreaker.transitionToOpenState(); // When - Chờ hết waitDurationInOpenState (đã config là 10s trong test profile) Thread.sleep(11000); // Then assertThat(circuitBreaker.getState()).isEqualTo(CircuitBreaker.State.HALF_OPEN); } @Test void shouldCloseAfterSuccessfulCallsInHalfOpen() { // Given circuitBreaker.transitionToHalfOpenState(); when(restTemplate.postForObject(any(), any(), any())) .thenReturn(PaymentResponse.builder().status("SUCCESS").build()); // When - Số calls thành công >= permittedNumberOfCallsInHalfOpenState for (int i = 0; i < 3; i++) { paymentClient.processPayment( PaymentRequest.builder().orderId("123").build() ).join(); } // Then assertThat(circuitBreaker.getState()).isEqualTo(CircuitBreaker.State.CLOSED); }
}
9.4.避けるべきよくある間違い
/**
- ❌ SAI: Fallback method signature không đúng */ @CircuitBreaker(name = "service", fallbackMethod = "fallback") public String callService(String param) { return service.call(param); }
// ❌ Thiếu Throwable parameter public String fallback(String param) { // Sẽ không được gọi! return "default"; }
// ✅ ĐÚNG: Phải có Throwable/Exception public String fallback(String param, Throwable t) { return "default"; }
/**
- ❌ SAI: Catch exception trong method */ @CircuitBreaker(name = "service") public String callService() { try { return service.call(); } catch (Exception e) { return "error"; // Circuit Breaker không thấy failure! } }
// ✅ ĐÚNG: Để exception propagate @CircuitBreaker(name = "service", fallbackMethod = "fallback") public String callService() { return service.call(); // Throw exception nếu fail }
/**
❌ SAI: Self-invocation (gọi method trong cùng class) */ @Service public class MyService {
@CircuitBreaker(name = "service") public String callService() { return externalService.call(); }
public String doSomething() { return callService(); // Circuit Breaker không work! } }
// ✅ ĐÚNG: Gọi từ class khác hoặc inject self @Service public class MyService {
@Autowired private MyService self; // Hoặc inject từ ApplicationContext @CircuitBreaker(name = "service") public String callService() { return externalService.call(); } public String doSomething() { return self.callService(); // Đi qua proxy, Circuit Breaker work! }}
/**
- ❌ SAI: Không set timeout, leading to thread exhaustion */ @CircuitBreaker(name = "service") public String callSlowService() { return slowService.call(); // Có thể block 60s! }
// ✅ ĐÚNG: Kết hợp với TimeLimiter @CircuitBreaker(name = "service") @TimeLimiter(name = "service") // Timeout sau 5s public CompletableFuture<String> callSlowService() { return CompletableFuture.supplyAsync(() -> slowService.call()); }
10. 結論
10.1.概要
サーキット ブレーカーは、マイクロサービス アーキテクチャにおいて次の目的に不可欠なパターンです。
- 連鎖的な障害の防止 - ドミノ効果からシステムを保護する
- フェイルファスト - タイムアウトを待たずに、すぐにエラーを返します
- 自己修復 - 依存関係サービスが再び動作すると自動的に復元します
- グレースフル デグラデーション - 完全な失敗の代わりにフォールバックを提供する
10.2.導入チェックリスト
- [ ] 保護する必要がある外部依存関係を特定する
- [ ] Resilience4j の依存関係を追加します
- [ ] サーキットブレーカーのパラメータを適切に構成する
- [ ] すべてのサーキット ブレーカーにフォールバック メソッドを実装する
- [ ] Actuator + Prometheus によるモニタリングの追加
- [ ] 状態遷移のアラートを設定する
- [ ] サーキット ブレーカーの動作に関する単体テストを作成する
- [ ] 実稼働メトリクスに基づいたパラメータの調整
10.3.リソース
この記事は、Spring Boot Advanced シリーズ用に書かれたものです。回復力のあるマイクロサービス システムを構築したいバックエンド開発者および DevOps エンジニアに適しています。