Sync engine

Bolt ships a native sync engine — not a plugin, a polling loop, or a third-party replication layer. This page is how it works under the hood: the live-query Machine, guest connect and extendPrefix, host-only advance, and the shared host core that carries them. For the read and write API apps are authored against, see Live data.

The mental model: your queries are live

Think of every live query as a standing question the server keeps answered for you — there is no second database in the browser. Your app reads the answers it declared, writes through ordinary collection commands, and the engine corrects every answer the moment anyone’s commit touches it.

        SERVER DATABASE (Postgres)                     THIS BROWSER
        ┌────────────────────────────────┐            ┌────────────────────────────┐
        │ collections                    │            │ the sync Machine —         │
        │ policies · write contracts     │            │ versioned prefixes this    │
        │ approvals · audit              │            │ tab registered, plus an    │
        │ SyncChange capture             │            │ in-memory write overlay    │
        └────────────────────────────────┘            └────────────────────────────┘
              ▲                                            ▲
              │  client mutation pipeline                  │
              └────────────────────────────────────────────┘

        sync.connect registers prefixes; sync.extendPrefix grows them;
        the host calls sync.advance after a commit and pumps apply
        frames over the standing stream

Two flows keep the answers honest. Down: every commit is pushed as one apply frame — the prefix deltas, resets, and write settlements it produced, together. Up: your writes travel as ordinary collection commands and are committed, or refused, by the server, which stays the authority.

On Colony , the server side of this runs in your tenant runtime behind a proxied HTTP transport; the engine itself is a Bolt concern and works the same on any host.

How answers stay current

The server is never polled on a timer. Every committed mutation captures SyncChange facts (link-and-route values, not an id-only wake) and its write outcome in the same transaction. There is no changelog cursor, outbox, or digest of held ids. A commit wakes plans indexed under a changed collection, and the push begins:

  someone commits a mutation
        │  SyncChange facts + write outcome in the SAME transaction
        ▼
  COMMIT  ──►  the host awaits the lane; the guest sync.advance-s
               affected prefixes against the commit's change list
                                    │
                                    ▼
        keyed delta  ──►  removeIds + put at a final index
                           — or a reset that re-registers
                                    │
                                    ▼
        event: apply   { updates, resets, outcomes }   one reducer event:
        the Machine applies prefix deltas, settles the writes, the
        UI updates — nothing re-runs by hand

• Every delta carries a version fence : it applies only when the client’s retained version equals the update’s fromVersion and toVersion is the next integer. Otherwise the link restarts.
• A reset is always legal: the browser drops the prefix and re-registers. A plan-key or authority mismatch, a broken prefix, or a byte/row ceiling is a reset — not a full-answer patch on the wake.
• Every answer is filtered by the reader’s policy scope in SQL before it is sent, and columns the subject may not read are masked by the same rule a direct read goes through — a client can never receive a row its policy denies, and a row it already holds outlives a revoked grant only until the next policy-relevant commit.
• An idle stream costs nothing, because nothing is on a timer: the connection is quiet until a commit pushes through it.

Connect, reconnect, and many tabs

  FIRST CONNECT                    RECONNECT / RESET
  ─────────────                    ─────────────────
  sync.connect registers every     the same command re-registers
  live prefix and returns rows     prefixes the browser dropped
  plus the plan the host files
        │                          │
  sync.extendPrefix grows the      a missed wake closes the
  viewer's loaded prefix           stream; there is no changelog
  without bumping version          to replay
        │                          │
  the stream carries apply         a frame that does not continue
  frames: updates, resets,         every fromVersion restarts
  outcomes                         the link

• sync.connect carries each query’s key, input, and requestedPrefix, plus detached keys and pending write ids. It resolves each requested prefix and returns rows plus the plan the host will file. Reconnect and reset use the same command.
• sync.extendPrefix grows a viewer’s loaded prefix without bumping version. A missed wake closes the stream; there is no changelog to replay.
• One EventSource per browser profile is shared across tabs and workspaces. The host files one plan per planKey; several connections may attach as viewers with different loadedPrefix lengths.
• A reconnect re-resolves. Per-scope registry state is O(live plans); nothing durable is stored in the browser.

What happens over the wire

Public Colony URLs are dedicated sync routes. The browser posts /__bolt/sync/connect and /__bolt/sync/extend ; the host holds the stream open at /__bolt/sync/stream.

A frame that does not continue every retained fromVersion is a protocol error and restarts the link. A reset is always legal. The full protocol is described in the public Bolt sync-engine documentation .

The shared host core: registry and pump

Subscription state lives in the host registry. SyncRegistry and SyncConnectionLane are one executable core in bolt-protocol, consumed directly by Colony in the cloud and bolt-server when self-hosted. Hosts provide connection, scope, guest-call, and disconnect adapters; they do not reimplement filing, invalidation, ordering, or emission:

• The registry files opaque guest facts per planKey — version, prefix keys, retained bytes, authority fingerprint, dependencies, and routing constraints. Several connections may attach as viewers with different loadedPrefix lengths. The host never resolves a subject, evaluates a predicate, or constructs a delta.
• The pump serializes connect, extendPrefix, and committed on one lane per (tenant, environment, releaseId). The host awaits lane acceptance; an unwritable sink or a guest failure closes the uncertain connections. There is no changelog to replay a missed wake.

Related guides

• Live data — the live queries and collection writes authored against this engine
• Bolt framework — where the sync engine sits in the platform
• Policies — scope what every answer may contain