added initial roadmap and implementation

2026-03-11 22:23:16 +02:00
parent 39b97a6db5
commit c0792f2e1d
62 changed files with 12410 additions and 1 deletions
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -0,0 +1,146 @@
+# MadBase Development Roadmap
+
+This document outlines the development plan for **MadBase**, a high-performance, resource-efficient, Supabase-compatible API layer written in Rust. The roadmap is derived from the requirements specified in [SPECIFICATIONS.md](./SPECIFICATIONS.md).
+
+## Phase 1: Foundation & Core APIs (MVP)
+**Goal:** Establish the single-binary architecture and deliver functional Auth and Data APIs for a single project context.
+
+### 1.1 Project Scaffolding & Architecture
+- [x] Initialize Rust workspace with modular crate structure (`gateway`, `auth`, `data_api`, `common`, `control_plane`).
+- [x] Implement configuration management (Environment variables + .env).
+- [x] Set up basic HTTP server (Axum/Actix) acting as the **Gateway**.
+- [x] Implement connection pooling for PostgreSQL (SQLx or similar).
+- [x] Create `docker-compose.yml` for dev database (compatible with Podman).
+
+### 1.2 Authentication Service (`/auth/v1`)
+- [x] Implement User model & schema (compatible with GoTrue/Supabase).
+- [x] **Sign Up**: Email/password registration with Argon2 hashing.
+- [x] **Sign In**: Email/password login returning JWTs.
+- [x] **Token Management**:
+    - [x] Issue Access Tokens (JWT) with required claims (`sub`, `role`, `iss`, `iat`, `exp`) and optional (`aud`, `email`).
+    - [x] Issue Refresh Tokens and implement rotation logic.
+- [x] **Session**: `/user` endpoint to retrieve current session.
+
+### 1.3 Data API (PostgREST-lite) (`/rest/v1`)
+- [x] **Query Parser**: Parse URL parameters for filtering, ordering, and pagination.
+    - [x] Filters: `eq`, `neq`, `lt`, `gt`, `in`, `is`.
+    - [x] Ordering: `order=col.asc|desc`.
+    - [x] Pagination: `limit`, `offset`.
+- [x] **CRUD Operations**:
+    - [x] `GET`: Select rows (basic `select=*`).
+    - [x] `POST`: Insert rows.
+    - [x] `PATCH`: Update rows.
+    - [x] `DELETE`: Delete rows.
+- [x] **RPC**: `POST /rpc/<function>` support for calling Postgres functions.
+- [x] **RLS Enforcement**:
+    - [x] Implement transaction wrapping.
+    - [x] Inject claims via `SET LOCAL request.jwt.claims`.
+    - [x] Switch roles (`anon` vs `authenticated` vs `service_role`).
+
+### 1.9 Podman Compose Deployment
+Single `docker-compose.yml` (compatible with `podman-compose`) deploys:
+- [x] **PostgreSQL**: Database for Auth and Data storage.
+- [x] **MinIO**: Object storage for file uploads.
+- [x] **Control Plane DB**: Stores project-specific config and secrets.
+
+---
+
+## Phase 2: Realtime & Storage
+**Goal:** Enable real-time data subscriptions and object storage capabilities.
+
+### 2.1 Realtime Service (`/realtime/v1`)
+- [x] **WebSocket Server**: Implement using `axum` + `tungstenite`.
+- [x] **Replication Consumer**:
+    - [x] Connect to Postgres via LISTEN/NOTIFY (fallback path).
+    - [ ] Connect to Postgres replication slot (`pgoutput`) via `tokio-postgres` or `sqlx` (Defer to Phase 5: Advanced Realtime).
+    - [x] Broadcast row changes (INSERT/UPDATE/DELETE) to connected clients.
+- [x] **Subscription Management**:
+    - [x] Handle `Join` messages to subscribe to specific tables/rows.
+    - [x] Filter events based on client subscriptions.
+
+### 2.2 Storage Service (`/storage/v1`)
+- [x] **S3 Proxy**:
+    - [x] List Buckets (`GET /bucket`).
+    - [x] List Objects (`GET /object/:bucket_id`).
+    - [x] Upload/Download (`POST/GET /object/:bucket_id/:filename`).
+- [x] **Permissions**:
+    - [x] RLS-like policies for buckets/objects (storage.buckets, storage.objects tables).
+    - [x] Public vs Private buckets.
+
+## Phase 3: Control Plane & Management
+**Goal**: Build the administrative layer to manage projects and configurations.
+
+### 3.1 Project Management (`/v1/projects`)
+- [x] **Projects Table**: Store project metadata (name, owner, status).
+- [x] **Provisioning**: (Mocked for MVP) Simulate creating resources for a new project.
+- [x] **API Keys**: Generate and validate Service Keys (anon/service_role).
+
+### 3.2 Secrets Management (`/v1/secrets`)
+- [x] **JWT Generation**: Automatically generate secure JWT secrets and keys for new projects.
+
+- [x] **Project Resolution**:
+    - [x] Resolve project context via `x-project-ref` header.
+- [x] **Dynamic Configuration**:
+    - [x] Load project-specific config (DB URL, JWT secret, API keys) from Control Plane DB.
+- [x] **Isolation**: Ensure strict separation of connections and caches between projects.
+
+---
+
+## Phase 4: Admin UI & Observability
+**Goal:** Provide a management interface and production-grade monitoring.
+
+### 4.1 Admin API (`/admin/v1`)
+- [x] **Project Management**: Create, Update, Soft-delete projects.
+- [x] **User Management**: Admin-level user CRUD.
+- [x] **Config Management**: Key rotation and setting updates.
+
+### 4.2 Management UI
+- [x] **Dashboard**: React/Web-based UI for managing projects.
+- [x] **Features**:
+    - [x] DB Connection tester.
+    - [x] Storage bucket browser (Basic).
+    - [x] Realtime connection stats (Basic).
+    - [x] Logs viewer (Basic).
+
+### 4.3 Observability Stack
+- [x] **Metrics**: Expose Prometheus-compatible metrics (Request latency, DB pool stats, Active WS connections).
+- [x] **Logs**: Structured JSON logging with correlation IDs.
+- [x] **Infrastructure**:
+    - [x] Configure **VictoriaMetrics** for metric storage.
+    - [x] Configure **Loki** for log aggregation.
+    - [x] Configure **Grafana** with pre-built dashboards.
+- [x] **Docker Compose**: Finalize the all-in-one `docker-compose.yml`.
+
+---
+
+## Phase 5: Polish, Security & Extensions
+**Goal:** Harden the system for production use and expand compatibility.
+
+### 5.1 Advanced Features
+- [x] **Auth**: OAuth provider integration (Google, GitHub, etc.).
+- [ ] **Data API**:
+    - [x] Basic column selection (`?select=col1,col2`).
+    - [x] Nested selects (joins) (`?select=col,relation(col)`).
+    - [x] Complex boolean logic (`or`, `and`).
+    - [x] Bulk operations optimization (Bulk Insert).
+- [x] **Realtime**: Resume from LSN/ID support for reliability (via History Table).
+
+### 5.2 Security & Performance
+- [x] **Hardening**:
+    - [x] Rate limiting (per IP/Project).
+    - [x] CORS configuration.
+    - [x] Input validation strictness.
+- [x] **Performance**:
+    - [x] Query caching where appropriate.
+    - [x] WS fanout optimization.
+- [x] **Testing**:
+    - [x] Integration tests using the official `@supabase/supabase-js` client.
+    - [ ] Load testing.
+
+---
+
+## Milestone Summary
+1.  **MVP**: Auth + Data API (Phase 1).
+2.  **Beta**: + Realtime + Storage (Phase 2).
+3.  **RC**: + Functions + Multi-tenancy (Phase 3).
+4.  **v1.0**: + Admin UI + Observability + Production Ready (Phase 4 & 5).