All Stories
2-2-4-hot-spot-protectionDoneEpic 2.2
Story 2.2.4: Hot Spot Protection
Status: done
Tasks
- Task 1: Implement HotSpotProtector class with per-fixture rate limiting (AC: #1, #2, #3)
- 1.1: Create `HotSpotProtector` class with fixture-scoped request tracking
- 1.2: Implement sliding window rate counter (50 calls/min per fixture)
- 1.3: Add request queue with 100-request capacity per fixture
- 1.4: Implement queue overflow handling with 429 response and retry-after calculation
- 1.5: Add async acquire/release methods for request permit management
- 1.6: Write unit tests for rate limiting edge cases (burst, sustained, overflow)
- Task 2: Pre-emptive cache warming for major events (AC: #4) - DEFERRED
- 2.1: Create event classification system (major vs regular events)
- 2.2: Implement cache warming scheduler (15 min before game start for major events)
- 2.3: Add background job to pre-fetch odds for identified major events
- 2.4: Configure major event triggers (Super Bowl, Championship games, playoffs)
- 2.5: Write tests for cache warming execution and timing
- Task 3: Real-time dashboard for per-fixture API usage monitoring (AC: #5)
- 3.1: Create `/health/hot-spots` endpoint returning per-fixture call counts
- 3.2: Expose queue depths and overflow counts per fixture
- 3.3: Add Prometheus metrics for hot spot detection (fixture_api_calls_total, fixture_queue_depth) - DEFERRED
- 3.4: Create dashboard visualization showing top 10 fixtures by API usage
- 3.5: Add alerting when any fixture exceeds 40 calls/min (80% threshold)
- 3.6: Write integration tests for metrics endpoint
- Task 4: Integration with existing Optic Odds service layer (AC: #1, #2, #3)
- 4.1: Wrap all fixture-specific Optic Odds calls with HotSpotProtector.acquire()
- 4.2: Add fixture_id extraction from request context
- 4.3: Handle RateLimitExceeded exceptions in API error middleware
- 4.4: Update API documentation with 429 response format
- 4.5: Write end-to-end tests simulating hot spot scenarios
- Task 5: Performance validation and load testing (AC: #1, #2, #5) - DEFERRED
- 5.1: Load test with 100 concurrent requests to single fixture (unit test)
- 5.2: Verify queue behavior under sustained overload
- 5.3: Measure queue wait times and retry-after accuracy
- 5.4: Test cache warming reduces hot spot pressure
- 5.5: Validate metrics accuracy under load
Progress
Tasks3/5
Acceptance Criteria0
Total Tasks5