madbase/docs/REDIS_IMPLEMENTATION.md

MadBase Redis Integration Implementation Plan

Current Status Analysis

✅ Already Implemented

1. Redis in State Pillar - Redis is already added to docker-compose.pillar-database.yml
2. HAProxy Configuration - Port 6379 routing is configured in autobase-haproxy.cfg
3. Config Support - redis_url field exists in common/src/config.rs
4. L1 Cache Ready - moka is already in gateway/Cargo.toml

🔨 Needs Implementation

Phase 1: Core Redis Client (Common Crate)

File: common/src/cache.rs

• Create Redis client wrapper with connection pooling
• Implement L1/L2 cache abstraction layer
• Add distributed locking primitives
• Add session management utilities

Phase 2: Application Layer Integration

Gateway (Proxy/Worker)

• File: gateway/src/cache_layer.rs
  • L1 cache for project configs (moka)
  • L2 cache for shared state (Redis)
  • Cache warming strategies
  • Cache invalidation logic

Auth Module

• File: auth/src/session.rs
  • Shared auth sessions across proxies
  • Session tokens in Redis
  • Multi-proxy logout support

Phase 3: Features

Realtime Presence

• File: realtime/src/presence.rs
  • Track online users across workers
  • Channel presence management
  • Heartbeat mechanism with Redis pub/sub

Distributed Locking

• File: common/src/locking.rs
  • Redlock implementation
  • Migration coordination
  • Background job synchronization

Rate Limiting

• File: gateway/src/rate_limit.rs
  • Sliding window rate limiting
  • Distributed counters
  • IP-based and user-based limits

Phase 4: Updates to Existing Files

Templates

• Update db-node.yaml to include Redis service definition
• Update all templates that include PostgreSQL to also include Redis

Documentation

• Update AUTOBASE.md to reflect "State Node" concept
• Create CACHING_STRATEGY.md with architecture details
• Update NODE_TEMPLATES.md with Redis information

Tests

• Integration tests for cache layers
• Failover tests for Redis HA
• Performance benchmarks

Implementation Order

1. Common Cache Layer (Priority 1)

   • Redis client with pooling
   • Cache abstraction (L1/L2)
   • Basic operations (get/set/delete)

2. Auth Sessions (Priority 1)

   • Shared session store
   • Multi-proxy support

3. Presence Tracking (Priority 2)

   • User online status
   • Channel presence

4. Distributed Locking (Priority 2)

   • Migration coordination
   • Background job locks

5. Rate Limiting (Priority 3)

   • Distributed rate limiting
   • Sliding windows

6. Documentation & Tests (Priority 4)

   • Update all docs
   • Add comprehensive tests