Introduction
Observability includes 3 pillars: Logs, Metrics, Traces. Spring Boot 4 has built-in Actuator for health checks, Micrometer for metrics, and Micrometer Tracing for distributed tracing — helping to monitor and debug applications in production.
1. Spring Boot Actuator
1.1 Setup
// build.gradle.kts
implementation("org.springframework.boot:spring-boot-starter-actuator")
1.2 Configuring Endpoints
# application.yml
management:
endpoints:
web:
exposure:
include: health, info, metrics, prometheus, env, loggers
base-path: /actuator
endpoint:
health:
show-details: when_authorized
show-components: always
info:
env:
enabled: true
1.3 Important Endpoints
| Endpoints | Description |
|---|---|
| /actuator/health | Health check — UP/DOWN |
| /actuator/info | Application info |
| /actuator/metrics | List of metrics |
| /actuator/metrics/{name} | Specific metric details |
| /actuator/prometheus | Metrics Prometheus format |
| /actuator/loggers | View and change log level runtime |
1.4 Custom Health Indicator
@Component
public class DatabaseHealthIndicator implements HealthIndicator {
private final DataSource dataSource;
public DatabaseHealthIndicator(DataSource dataSource) {
this.dataSource = dataSource;
}
@Override
public Health health() {
try (Connection conn = dataSource.getConnection()) {
if (conn.isValid(2)) {
return Health.up()
.withDetail("database", "PostgreSQL")
.withDetail("connection", "valid")
.build();
}
} catch (SQLException e) {
return Health.down()
.withDetail("error", e.getMessage())
.build();
}
return Health.down().build();
}
}
1.5 Change Log Level Runtime
# Xem log level hiện tại
curl http://localhost:8080/actuator/loggers/com.example.myapp
# Thay đổi log level (không cần restart)
curl -X POST http://localhost:8080/actuator/loggers/com.example.myapp \
-H "Content-Type: application/json" \
-d '{"configuredLevel": "DEBUG"}'
2. Micrometer Metrics
2.1 Custom Metrics
@Service
public class OrderService {
private final Counter orderCounter;
private final Timer orderProcessingTimer;
private final AtomicInteger activeOrders;
public OrderService(MeterRegistry registry) {
this.orderCounter = Counter.builder("orders.created")
.description("Total orders created")
.tag("type", "web")
.register(registry);
this.orderProcessingTimer = Timer.builder("orders.processing.time")
.description("Order processing duration")
.register(registry);
this.activeOrders = registry.gauge("orders.active",
new AtomicInteger(0));
}
public OrderResponse createOrder(CreateOrderRequest request) {
return orderProcessingTimer.record(() -> {
activeOrders.incrementAndGet();
try {
OrderResponse result = processOrder(request);
orderCounter.increment();
return result;
} finally {
activeOrders.decrementAndGet();
}
});
}
}
2.2 @Timed Annotation
@RestController
@RequestMapping("/api/v1/products")
public class ProductController {
@Timed(value = "api.products.get",
description = "Time to get product",
percentiles = {0.5, 0.95, 0.99})
@GetMapping("/{id}")
public ResponseEntity<ProductResponse> getProduct(@PathVariable Long id) {
return ResponseEntity.ok(productService.getProduct(id));
}
}
2.3 Prometheus Integration
// build.gradle.kts
implementation("io.micrometer:micrometer-registry-prometheus")
# application.yml
management:
prometheus:
metrics:
export:
enabled: true
Access http://localhost:8080/actuator/prometheus → Prometheus metrics:
# HELP orders_created_total Total orders created
# TYPE orders_created_total counter
orders_created_total{type="web"} 142.0
# HELP orders_processing_time_seconds Order processing duration
orders_processing_time_seconds_count 142
orders_processing_time_seconds_sum 28.456
3. Structured Logging
3.1 Configure JSON Logging
# application.yml
logging:
structured:
format:
console: ecs # Elastic Common Schema
level:
com.example: DEBUG
org.springframework.web: INFO
3.2 Contextual Logging
@Service
public class OrderService {
private static final Logger log = LoggerFactory.getLogger(OrderService.class);
public OrderResponse createOrder(CreateOrderRequest request) {
log.info("Creating order for user: {}, items: {}",
request.userId(), request.items().size());
try {
Order order = processOrder(request);
log.info("Order created successfully: orderId={}, total={}",
order.getId(), order.getTotalAmount());
return OrderResponse.from(order);
} catch (Exception e) {
log.error("Failed to create order for user: {}",
request.userId(), e);
throw e;
}
}
}
4. Distributed Tracing
4.1 Setup Micrometer Tracing
// build.gradle.kts
implementation("io.micrometer:micrometer-tracing-bridge-otel")
implementation("io.opentelemetry:opentelemetry-exporter-otlp")
# application.yml
management:
tracing:
sampling:
probability: 1.0 # 100% sampling (dev), production: 0.1
otlp:
tracing:
endpoint: http://localhost:4318/v1/traces
4.2 Custom Spans
@Service
public class PaymentService {
private final ObservationRegistry observationRegistry;
public PaymentService(ObservationRegistry observationRegistry) {
this.observationRegistry = observationRegistry;
}
public PaymentResult processPayment(PaymentRequest request) {
return Observation.createNotStarted("payment.process", observationRegistry)
.lowCardinalityKeyValue("payment.method", request.method())
.observe(() -> {
// Logic xử lý payment
return doProcessPayment(request);
});
}
}
4.3 Observability Architecture
┌─────────────────────────────────────────────┐
│ Spring Boot Application │
│ ┌──────────┐ ┌──────────┐ ┌──────────────┐ │
│ │ Logs │ │ Metrics │ │ Traces │ │
│ │ Logback │ │Micrometer│ │ OTel/Brave │ │
│ └────┬─────┘ └────┬─────┘ └──────┬───────┘ │
└───────┼────────────┼───────────────┼─────────┘
│ │ │
▼ ▼ ▼
┌────────┐ ┌──────────┐ ┌───────────┐
│ Loki │ │Prometheus│ │ Tempo │
└───┬────┘ └────┬─────┘ └─────┬─────┘
│ │ │
└────────────┼───────────────┘
▼
┌──────────┐
│ Grafana │ ← Dashboard tổng hợp
└──────────┘
5. Actuator security in Production
@Bean
public SecurityFilterChain actuatorSecurity(HttpSecurity http) throws Exception {
http
.securityMatcher("/actuator/**")
.authorizeHttpRequests(auth -> auth
.requestMatchers("/actuator/health").permitAll()
.requestMatchers("/actuator/info").permitAll()
.requestMatchers("/actuator/**").hasRole("ADMIN"))
.httpBasic(Customizer.withDefaults());
return http.build();
}
Summary
- Actuator provides health checks, metrics, loggers endpoints — can change runtime log level without restart
- Micrometer: Counter, Timer, Gauge for custom metrics, export to Prometheus
- Structured logging (JSON/ECS format) + Distributed Tracing (OpenTelemetry) creates a complete observability stack
- Secure Actuator endpoints in production: only expose health/info publicly, the rest require the ADMIN role
Exercises
- Configure Actuator: expose health, metrics, prometheus endpoints. Create custom health indicators for external API dependencies
- Implement custom metrics: count orders, measure request processing time with Timer, display active users with Gauge
- Setup structured logging and distributed tracing: connect to Prometheus + Grafana (docker-compose)