Vlad Durnea
cffdf8af86
Some checks failed
CI/CD Pipeline / unit-tests (push) Failing after 1m16s
CI/CD Pipeline / integration-tests (push) Failing after 2m32s
CI/CD Pipeline / lint (push) Successful in 5m22s
CI/CD Pipeline / e2e-tests (push) Has been skipped
CI/CD Pipeline / build (push) Has been skipped
98 lines
2.6 KiB
Markdown
98 lines
2.6 KiB
Markdown
# MadBase Redis Integration Implementation Plan
|
|
|
|
## Current Status Analysis
|
|
|
|
### ✅ Already Implemented
|
|
1. **Redis in State Pillar** - Redis is already added to docker-compose.pillar-database.yml
|
|
2. **HAProxy Configuration** - Port 6379 routing is configured in autobase-haproxy.cfg
|
|
3. **Config Support** - redis_url field exists in common/src/config.rs
|
|
4. **L1 Cache Ready** - moka is already in gateway/Cargo.toml
|
|
|
|
### 🔨 Needs Implementation
|
|
|
|
## Phase 1: Core Redis Client (Common Crate)
|
|
|
|
### File: common/src/cache.rs
|
|
- Create Redis client wrapper with connection pooling
|
|
- Implement L1/L2 cache abstraction layer
|
|
- Add distributed locking primitives
|
|
- Add session management utilities
|
|
|
|
## Phase 2: Application Layer Integration
|
|
|
|
### Gateway (Proxy/Worker)
|
|
- File: gateway/src/cache_layer.rs
|
|
- L1 cache for project configs (moka)
|
|
- L2 cache for shared state (Redis)
|
|
- Cache warming strategies
|
|
- Cache invalidation logic
|
|
|
|
### Auth Module
|
|
- File: auth/src/session.rs
|
|
- Shared auth sessions across proxies
|
|
- Session tokens in Redis
|
|
- Multi-proxy logout support
|
|
|
|
## Phase 3: Features
|
|
|
|
### Realtime Presence
|
|
- File: realtime/src/presence.rs
|
|
- Track online users across workers
|
|
- Channel presence management
|
|
- Heartbeat mechanism with Redis pub/sub
|
|
|
|
### Distributed Locking
|
|
- File: common/src/locking.rs
|
|
- Redlock implementation
|
|
- Migration coordination
|
|
- Background job synchronization
|
|
|
|
### Rate Limiting
|
|
- File: gateway/src/rate_limit.rs
|
|
- Sliding window rate limiting
|
|
- Distributed counters
|
|
- IP-based and user-based limits
|
|
|
|
## Phase 4: Updates to Existing Files
|
|
|
|
### Templates
|
|
- Update db-node.yaml to include Redis service definition
|
|
- Update all templates that include PostgreSQL to also include Redis
|
|
|
|
### Documentation
|
|
- Update AUTOBASE.md to reflect "State Node" concept
|
|
- Create CACHING_STRATEGY.md with architecture details
|
|
- Update NODE_TEMPLATES.md with Redis information
|
|
|
|
### Tests
|
|
- Integration tests for cache layers
|
|
- Failover tests for Redis HA
|
|
- Performance benchmarks
|
|
|
|
## Implementation Order
|
|
|
|
1. **Common Cache Layer** (Priority 1)
|
|
- Redis client with pooling
|
|
- Cache abstraction (L1/L2)
|
|
- Basic operations (get/set/delete)
|
|
|
|
2. **Auth Sessions** (Priority 1)
|
|
- Shared session store
|
|
- Multi-proxy support
|
|
|
|
3. **Presence Tracking** (Priority 2)
|
|
- User online status
|
|
- Channel presence
|
|
|
|
4. **Distributed Locking** (Priority 2)
|
|
- Migration coordination
|
|
- Background job locks
|
|
|
|
5. **Rate Limiting** (Priority 3)
|
|
- Distributed rate limiting
|
|
- Sliding windows
|
|
|
|
6. **Documentation & Tests** (Priority 4)
|
|
- Update all docs
|
|
- Add comprehensive tests
|