35 lines
2.0 KiB
Markdown
35 lines
2.0 KiB
Markdown
# Implement Missing Phase 2 Features
|
|
|
|
I will implement the remaining features for Phase 2: **Advanced Replication** (Realtime) and **Resumable Uploads** (Storage).
|
|
|
|
## 1. Advanced Realtime Replication (`pgoutput`)
|
|
**Goal**: Replace the `LISTEN/NOTIFY` fallback with robust logical replication using the `pgoutput` protocol.
|
|
|
|
- **Dependencies**: Add `pgoutput` crate and enable `replication` feature for `tokio-postgres`.
|
|
- **Implementation**:
|
|
- Update `realtime/src/replication.rs` to connect to Postgres in **replication mode**.
|
|
- Create a replication slot (`madbase_slot`) and start streaming from publication (`madbase_pub`).
|
|
- Use `pgoutput::Decoder` to parse binary replication messages (`Relation`, `Insert`, `Update`, `Delete`).
|
|
- Maintain an in-memory cache of `Relation` metadata (schema, table, columns) to map relation IDs to names.
|
|
- Construct `PostgresPayload` from change events and broadcast to WebSocket clients.
|
|
|
|
## 2. Resumable Uploads (TUS Protocol)
|
|
**Goal**: Implement the TUS protocol for reliable large file uploads in the Storage service.
|
|
|
|
- **Dependencies**: Add `base64` to `storage/Cargo.toml`.
|
|
- **New Module**: Create `storage/src/tus.rs`.
|
|
- **Endpoints**:
|
|
- `POST /storage/v1/upload/resumable`: Initialize upload. Creates a local tracking file.
|
|
- `PATCH /storage/v1/upload/resumable/:id`: Append data chunk to the local file.
|
|
- `HEAD /storage/v1/upload/resumable/:id`: Return current upload offset.
|
|
- **Completion Logic**:
|
|
- When `offset == size`, stream the complete file to S3.
|
|
- Insert metadata into `storage.objects`.
|
|
- Clean up local temporary files.
|
|
|
|
## Execution Steps
|
|
1. **Update Dependencies**: Modify `realtime/Cargo.toml` and `storage/Cargo.toml`.
|
|
2. **Implement Realtime Replication**: Rewrite `realtime/src/replication.rs` with `pgoutput` logic.
|
|
3. **Implement TUS Handlers**: Create `storage/src/tus.rs` and register routes in `storage/src/lib.rs`.
|
|
4. **Verify**: Ensure compilation and check for basic logic correctness.
|