madapes
Repositories
7
 Projects Packages

@aurora/runtime-sharding (0.3.3)

Published 2025-12-27 10:44:45 +00:00 by vlad

Installation

@aurora:registry=
npm install @aurora/runtime-sharding@0.3.3
"@aurora/runtime-sharding": "0.3.3"

About this package

@aurora/runtime-sharding

Sharding, replication, and failover for Aurora multi-tenant deployments.

Overview

Aurora's sharding package provides:

  • Tenant-based sharding with consistent hashing
  • Shard Registry using NATS Key-Value store
  • Automatic shard assignment and routing
  • Replica support for high availability
  • Automatic failover
  • Shard rebalancing

Installation

bun add @aurora/runtime-sharding

Consistent Hashing

Shard Calculator

import { hashToShard } from "@aurora/runtime-sharding";

const shardId = hashToShard("tenant-123", 10);
console.log(shardId); // 7

// Same tenant always maps to same shard
const shardId2 = hashToShard("tenant-123", 10);
console.log(shardId); // 7

Hash Function

Uses SHA-256 for stable, deterministic hashing:

import { sha256 } from "@aurora/runtime-sharding";

const hash = await sha256("tenant-123");
console.log(hash); // hex string

Shard Registry

Creating Registry

import { ShardRegistry } from "@aurora/runtime-sharding";

const registry = new ShardRegistry({
  natsUrl: "nats://localhost:4222",
  bucket: "aurora-shards",  // NATS KV bucket
});

Node Registration

const node = await registry.register({
  id: "node-1",
  address: "http://localhost:3000",
  shards: ["shard-0", "shard-1", "shard-2"],
  role: "primary",  // primary, replica, or proxy
});

console.log(node.id); // node-1

Shard Assignment

// Assign shard to node
await registry.assignShard("shard-0", {
  nodeId: "node-1",
  role: "primary",
});

// Get shard owner
const assignment = await registry.getShardOwner("shard-0");
console.log(assignment);
// { nodeId: "node-1", role: "primary" }

// List all assignments
const assignments = await registry.listAssignments();

Replicas

// Add replica
await registry.addReplica("shard-0", {
  nodeId: "node-2",
  role: "replica",
});

// Get replicas
const replicas = await registry.getReplicas("shard-0");
// [{ nodeId: "node-2", role: "replica" }]

Shard Router

Creating Router

import { ShardRouter } from "@aurora/runtime-sharding";

const router = new ShardRouter({
  registry,
  localNodeId: "node-1",
  rpcClient,  // For remote shard calls
  timeout: 5000,
});

Routing Requests

const result = await router.execute("shard-0", async () => {
  return await database.findOrder(orderId);
});

// Or route by tenant
const result = await router.executeByTenant("tenant-123", async () => {
  return await database.findOrder(orderId);
});

Local vs Remote

const isLocal = router.isShardLocal("shard-0");

if (isLocal) {
  // Handle locally
  const result = await database.findOrder(orderId);
} else {
  // Forward to owner node
  const result = await router.forward("shard-0", "getOrder", { orderId });
}

Failover Coordinator

Creating Coordinator

import { FailoverCoordinator } from "@aurora/runtime-sharding";

const coordinator = new FailoverCoordinator({
  registry,
  checkInterval: 5000,  // 5 seconds
  heartbeatTimeout: 15000,  // 15 seconds
});

Monitoring Nodes

coordinator.onNodeDown(async (nodeId, shards) => {
  console.log(`Node ${nodeId} down, shards: ${shards.join(", ")}`);
  
  // Promote replicas
  for (const shardId of shards) {
    const replica = await registry.getReplica(shardId);
    if (replica) {
      await coordinator.promoteReplica(shardId, replica.nodeId);
    }
  }
});

coordinator.onNodeUp(async (nodeId) => {
  console.log(`Node ${nodeId} is back up`);
});

Starting Coordinator

await coordinator.start();

Shard Rebalancer

Creating Rebalancer

import { ShardRebalancer } from "@aurora/runtime-sharding";

const rebalancer = new ShardRebalancer({
  registry,
  targetDistribution: {
    // Node ID -> desired shard count
    "node-1": 3,
    "node-2": 3,
    "node-3": 4,
  },
});

Rebalancing

const plan = await rebalancer.calculatePlan();

console.log("Move shards:", plan.moves);
// [
//   { from: "node-1", to: "node-2", shard: "shard-5" },
//   { from: "node-1", to: "node-3", shard: "shard-9" }
// ]

// Execute plan
for (const move of plan.moves) {
  await rebalancer.moveShard(move.shard, move.from, move.to);
}

Usage Example

Full Setup

import { ShardRegistry, ShardRouter, FailoverCoordinator } from "@aurora/runtime-sharding";
import { Client as RpcClient } from "@aurora/runtime-rpc";

// Create registry
const registry = new ShardRegistry({ natsUrl: "nats://localhost:4222" });

// Register node
const node = await registry.register({
  id: "node-1",
  address: "http://localhost:3000",
  shards: ["shard-0", "shard-1", "shard-2"],
  role: "primary",
});

// Create router
const rpcClient = new RpcClient({ natsUrl: "nats://localhost:4222" });
const router = new ShardRouter({
  registry,
  localNodeId: node.id,
  rpcClient,
});

// Start failover coordinator
const coordinator = new FailoverCoordinator({ registry });
await coordinator.start();

// Handle requests
app.get("/api/v1/:tenantId/orders/:orderId", async (req) => {
  return router.executeByTenant(req.params.tenantId, async () => {
    return await database.findOrder(req.params.orderId);
  });
});

Shard Roles

Primary

  • Handles all reads and writes for assigned shards
  • Source of truth for shard data
  • Replicates to replicas

Replica

  • Read-only copy of primary shard
  • Can be promoted to primary on failover
  • Receives replication stream from primary

Proxy

  • Routes requests to correct nodes
  • No shard ownership
  • Used for load balancing

Metrics

import { Metrics } from "@aurora/runtime-observability";

const metrics = new Metrics({ prefix: "aurora" });

// Shard assignment metrics
const shardAssignments = metrics.gauge("shard_assignments", "Number of shards per node");
shardAssignments.set({ node_id: "node-1" }, 3);

// RPC metrics
const rpcRequests = metrics.counter("rpc_requests_total", "RPC requests");
rpcRequests.inc({ shard_id: "shard-0", method: "getOrder" });

// Failover metrics
const failoverEvents = metrics.counter("failover_events_total", "Failover events");
failoverEvents.inc({ shard_id: "shard-0", from: "node-1", to: "node-2" });

License

MIT

Dependencies

Dependencies

ID Version
@aurora/runtime-effect 0.3.3
nats ^2.15.1
Details
npm
2025-12-27 10:44:45 +00:00
9
latest
14 KiB
Assets (1)
runtime-sharding-0.3.3.tgz 14 KiB
Versions (3) View all
0.3.3 2025-12-27
0.3.2 2025-12-27
0.3.1 2025-12-27