WAGERBABE DOCS
All Stories
2-2-2-circuit-breaker-optic-oddsDoneEpic 2.2

Story 2.2.2: Circuit Breaker for Optic Odds

Status: done

Tasks

  • Task 1: Implement CircuitBreaker class (AC: #1, #2, #3, #4)
    • 1.1: Create `CircuitState` enum (CLOSED, OPEN, HALF_OPEN)
    • 1.2: Initialize CircuitBreaker with configurable thresholds (`failure_threshold=3`, `failure_window=30s`, `recovery_timeout=60s`)
    • 1.3: Implement `call()` method with state machine logic (check OPEN → evaluate recovery timeout, execute if CLOSED/HALF_OPEN)
    • 1.4: Implement `_on_failure()` method (track count within window, reset if expired, transition to OPEN at threshold, trigger alert)
    • 1.5: Implement `_on_success()` method (close circuit if HALF_OPEN, reset failure count)
    • 1.6: Implement CircuitOpenError exception class
    • 1.7: Add type hints and docstrings
  • Task 2: Integrate CircuitBreaker with Optic Odds service layer (AC: #2, #4)
    • 2.1: Create global circuit breaker instance for Optic Odds
    • 2.2: Wrap all Optic Odds API calls with `circuit_breaker.call()`
    • 2.3: Handle CircuitOpenError → fallback to stale cache
    • 2.4: Preserve existing error handling for other exceptions
  • Task 3: Implement health check endpoint (AC: #5)
    • 3.1: Create GET `/health/optic-odds` endpoint
    • 3.2: Return circuit state and metrics (state, failure_count, last_failure_time, opened_at, healthy)
    • 3.3: Add endpoint to API documentation
    • 3.4: Include in existing `/health` aggregated endpoint
  • Task 4: Implement alerting on circuit open (AC: #6)
    • 4.1: Create `send_alert()` utility function
    • 4.2: Integrate with notification system (Slack webhook or email)
    • 4.3: Include failure context in alert payload (failure count, last error message, timestamp)
    • 4.4: Test alert delivery in staging environment
  • Task 5: Add comprehensive testing (AC: #1, #2, #3, #4, #5, #6)
    • 5.1: Unit tests for CircuitBreaker state transitions
    • 5.2: Test failure threshold and time window logic
    • 5.3: Test recovery timeout and HALF-OPEN behavior
    • 5.4: Test health endpoint returns correct state
    • 5.5: Integration tests with mock Optic Odds failures
    • 5.6: Test alert triggering on circuit open
  • Task 6: Add monitoring and metrics (AC: #2, #5)
    • 6.1: Log circuit state transitions (INFO level)
    • 6.2: Expose metrics via /health/optic-odds endpoint
    • 6.3: Health check includes circuit state and metrics

Progress

Tasks6/6
Acceptance Criteria0
Total Tasks6

Navigation

View Epic 2.2All Stories