Recent Posts
Archives

Posts Tagged ‘CircuitBreaker’

PostHeaderIcon Building Resilient Architectures: Patterns That Survive Failure

How to design systems that gracefully degrade, recover quickly, and scale under pressure.

1) Patterns for Graceful Degradation

When dependencies fail, your system should still provide partial service. Examples:

  • Show cached product data if the pricing service is down.
  • Allow “read-only” mode if writes are failing.
  • Provide degraded image quality if the CDN is unavailable.

2) Circuit Breakers

Prevent cascading failures with Resilience4j or Hystrix:

@CircuitBreaker(name = "inventoryService", fallbackMethod = "fallbackInventory")
public Inventory getInventory(String productId) {
    return restTemplate.getForObject("/inventory/" + productId, Inventory.class);
}

public Inventory fallbackInventory(String productId, Throwable t) {
    return new Inventory(productId, 0);
}

3) Retries with Backoff

Retries should be bounded and spaced out:

@Retry(name = "paymentService", fallbackMethod = "fallbackPayment")
public PaymentResponse processPayment(PaymentRequest req) {
    return restTemplate.postForObject("/pay", req, PaymentResponse.class);
}

RetryConfig config = RetryConfig.custom()
    .maxAttempts(3)
    .waitDuration(Duration.ofMillis(200))
    .intervalFunction(IntervalFunction.ofExponentialBackoff(200, 2.0, 0.5)) // jitter
    .build();

4) Scaling Microservices in Kubernetes/ECS

Scaling is not just replicas—it’s smart policies:

  • Kubernetes HPA: Scale pods based on CPU or custom metrics (e.g., p95 latency).
    kubectl autoscale deployment api --cpu-percent=70 --min=3 --max=10
  • ECS: Use Service Auto Scaling with CloudWatch alarms on queue depth.
  • Pre-warm caches: Scale up before big events (e.g., Black Friday).

PostHeaderIcon [DevoxxPL2019] Resilience Patterns in Microservices: Beyond Hystrix

Lecturer

Tomasz Skowroński contributes to resilience-focused libraries and speaks on fault tolerance in distributed systems.

Abstract

This overview introduces resilience patterns for microservices, transitioning from Hystrix to Resilience4j. It explains time limiters, rate limiters, bulkheads, retries, and circuit breakers, using analogies and code to demonstrate configurations and executions. It appraises integration with existing code, order of application, and higher-layer usages, while considering effects on system stability and developer explicitness.

Introducing Resilience: Patterns for Fault Tolerance

Resilience ensures responsiveness amid failures, vital in distributed setups. Tomasz analogizes to Dunkirk evacuation, where limited boats mirror API calls—use responsibly.

Hystrix, Netflix’s library, implemented circuit breakers but ceased development. Resilience4j succeeds, embracing Java 8+ functional styles sans annotations or AOP.

Analytically, this shift favors lightweight, composable resilience over monolithic commands. Implications: easier adoption in diverse stacks, reducing overhead.

Time and Rate Limiters: Controlling Execution Durations and Frequencies

Time limiters enforce timeouts on futures or suppliers, preventing indefinite waits. Configure via builders:

TimeLimiterConfig config = TimeLimiterConfig.custom()
    .timeoutDuration(Duration.ofMillis(500))
    .build();
TimeLimiter timeLimiter = TimeLimiter.of(config);
Callable<String> callable = TimeLimiter.decorateFutureSupplier(timeLimiter, () -> CompletableFuture.supplyAsync(this::slowMethod));

This decorates calls, throwing on timeouts.

Rate limiters restrict invocations per period, using permissions:

RateLimiterConfig config = RateLimiterConfig.custom()
    .limitForPeriod(50)
    .limitRefreshPeriod(Duration.ofMinutes(1))
    .timeoutDuration(Duration.ofSeconds(3))
    .build();
RateLimiter rateLimiter = RateLimiter.of("backend", config);
Runnable restrictedCall = RateLimiter.decorateRunnable(rateLimiter, this::backendMethod);

Analytically, parameters like refresh periods balance throughput and protection. Implications: prevents overloads, though misconfigurations cause premature failures.

Bulkheads and Retries: Isolating and Recovering from Failures

Bulkheads isolate via thread pools or semaphores, limiting concurrent calls:

BulkheadConfig config = BulkheadConfig.custom()
    .maxConcurrentCalls(100)
    .maxWaitDuration(Duration.ofMillis(10))
    .build();
Bulkhead bulkhead = Bulkhead.of("backend", config);
Supplier<String> decorated = Bulkhead.decorateSupplier(bulkhead, this::backendMethod);

Retries attempt failed calls, configurable for attempts and waits.

Analytically, exponential backoffs mitigate thundering herds. Implications: boosts reliability, but excessive retries amplify loads.

Circuit Breakers: Caching Failures for Protection

Circuit breakers track successes/failures in buffers, opening on thresholds to block calls, periodically probing recovery.

Configurations define states: closed (allow), open (block), half-open (test).

Analytically, sliding windows maintain recent histories. Implications: shields backends during outages, explicit via decorators.

Strategic Application: Ordering, Layers, and Myths

Order matters: circuit breakers before retries avoid futile attempts. Apply at gateways or clouds for broader protection.

Myths: not all patterns always; failure-fast over safe sans explicit fallbacks.

Implications: explicitness via decorators clarifies intent, fostering robust designs.

Links: