Replication & Sync

Overview

In sovseal, the replication and synchronization architecture is engineered for speed and privacy. By employing a local-first model, reads are executed locally with 0 RTT latency, and writes return immediately upon local commit. Ciphertext replication to the edge happens asynchronously in the background.

sequenceDiagram
    participant Agent as AI Agent / Client
    participant Local as Local LanceDB (On-Device)
    participant Queue as Outbox Replication Queue
    participant Edge as Deno Edge Function (Frankfurt)
    participant Storage as Arweave / Supabase

    Agent->>Local: store_memory({content})
    Note over Local: Encrypt active_context via AES-256-GCM
    Local-->>Agent: Returns Success (3.8ms p50)
    Local->>Queue: Push encrypted snapshot
    loop Background Replication (Write-Behind)
        Queue->>Edge: POST /v2-agent-state (Bearer token + Ciphertext)
        Note over Edge: Validate SHA-256 project/key hash
        Edge->>Storage: Store snapshot & write-behind metadata
        Edge-->>Queue: Ack (200 OK)
    end

Write-Behind Queue & Backpressure

When an agent invokes store_memory, the client encrypts the payload on-device and writes to the local LanceDB instance. The operation immediately returns to the agent (taking only 3.8 ms at p50). Concurrently, a background worker pushes the encrypted snapshot into the Outbox Replication Queue.

Backpressure Control

To prevent memory leaks and uncontrolled growth of the outbox during offline periods or network degradation:

• Queue Cap: The default queue capacity is set to 1,000 pending snapshots.
• Throttling: If the queue exceeds capacity, new writes are throttled, introducing artificial latency to align agent execution with the system's ability to sync.
• Fail-Closed Gate: If the queue reaches 5,000 items, the client fails closed, returning a replicate_queue_full error to protect local system memory from exhaustion.

Retry Policy & Exponential Backoff

Replication to the edge endpoint employs a resilient, jittered retry mechanism to handle network instability.

Base Interval: 1.5 seconds
Multiplier: 2.0 (Exponential)
Max Backoff: 300 seconds (5 minutes)
Jitter: ± 20% random offset

The background synchronizer retries up to 10 times for transient errors (e.g., HTTP 502/503, connection drops). If all retries are exhausted, the snapshot remains in the local queue and is flagged as failed_replication. The worker pauses replication until a network state change is detected or a manual sync is requested.

Conflict Resolution

Since agents can run concurrently across multiple devices (e.g., Cursor, Claude Desktop, and CLI clients), conflicts in the chronological lineage can occur.

Lineage Chain Verification:
snapshot_id = sha256(canonicalize(payload) || parent_snapshot_id)

Conflicts are resolved client-side during the sync pull loop using these strict rules:

1. Last-Writer-Wins (LWW): If two concurrent snapshots fork from the same parent_snapshot, the snapshot with the higher client-set timestamp is preferred.
2. Deterministic Tie-Breaking: If the timestamps are identical, the snapshot with the lexicographically smaller client_payload_hash is selected.
3. Rollback vs. Fork: If the client pulls a snapshot that represents a fork of its current parent lineage, it performs a rollback (an append-only pointer update to the common ancestor) and replays local edits to merge the branches.

Monitoring Replication Lag

Developers can monitor the health of the sync engine directly from the SDK:

import { MemoryClient } from "@sovseal/sdk";

const client = new MemoryClient({ apiKey: "sov_live_..." });

// Get active synchronizer status
const status = await client.getSyncStatus();
console.log(`Pending sync items: ${status.pendingCount}`);
console.log(`Replication lag: ${status.lagMs} ms`);
console.log(`Synchronizer State: ${status.state}`); // 'idle' | 'syncing' | 'offline' | 'error'

Graceful Degradation (Offline Mode)

sovseal is designed to work offline:

• Read Continuity: Calls to recall_memory perform local vector searches on LanceDB and are entirely network-independent.
• Write Buffer: Calls to store_memory continue to accumulate in the local queue.
• Silent Resync: When the network comes back online, the SDK automatically detects connectivity and flushes the outbox queue in sequential order, re-establishing state continuity.