Stores
A store is where the server keeps sync data: every row (as JSON), the ordered log of changes, the
devices, and who belongs to which space. The SQL stores keep it in tables of their own, prefixed
ferry_, in your database.
import { createSyncServer, nodePostgres, postgresStore } from 'ferrysync/server';
import { Pool } from 'pg';
import { schema } from './schema';
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
export const syncServer = createSyncServer({
schema,
store: postgresStore(nodePostgres(pool)),
auth: async (request) => (await getSession(request))?.userId ?? null,
});| Store | For |
|---|---|
memoryStore() | Tests and demos: in memory, lost on restart |
sqliteStore(driver, options?) | One server process: a file next to the app |
postgresStore(driver, options?) | Production with several instances, serverless |
d1Store(env.DB, options?) | Cloudflare D1, inside one Durable Object |
A Durable Object's own SQLite uses sqliteStore with the durableObjectSqlite driver (below).
A driver connects a store to your database library. Drivers are a few lines over the library's own objects: Ferry imports none of them, and pass yours as they are.
| Driver | Wraps | Store | within takes |
|---|---|---|---|
nodeSqlite(db) | node:sqlite's DatabaseSync | sqliteStore | no |
bunSqlite(db) | bun:sqlite's Database | sqliteStore | no |
durableObjectSqlite(ctx.storage) | a Durable Object's ctx.storage | sqliteStore | no |
betterSqlite3(db) | better-sqlite3's Database | sqliteStore | no |
nodePostgres(pool) | pg's Pool | postgresStore | a pool client inside BEGIN |
postgresJs(sql) | postgres.js's sql | postgresStore | the sql of sql.begin |
pglite(db) | @electric-sql/pglite | postgresStore | its transaction's tx |
prismaDriver(prisma, { dialect }) | a Prisma client | either | an interactive transaction's tx |
drizzleDriver(db, { dialect, sql }) | a Drizzle database and its sql tag | either | a Drizzle tx |
within is how a write from your backend joins a
transaction of yours.
Memory
import { createSyncServer, memoryStore } from 'ferrysync/server';
import { schema } from './schema';
export const syncServer = createSyncServer({
schema,
store: memoryStore(),
auth: () => 'demo-user',
});Everything is lost when the process stops, and nothing is shared between processes. Use it in tests (see Testing) and demos.
SQLite
import { betterSqlite3, bunSqlite, nodeSqlite, sqliteStore } from 'ferrysync/server';
import Database from 'better-sqlite3';
import { Database as BunDatabase } from 'bun:sqlite';
import { DatabaseSync } from 'node:sqlite';
const onNode = sqliteStore(nodeSqlite(new DatabaseSync('sync.db')));
const onBun = sqliteStore(bunSqlite(new BunDatabase('sync.db')));
const withBetterSqlite3 = sqliteStore(betterSqlite3(new Database('sync.db')));- One write at a time: each takes SQLite's write lock up front (
BEGIN IMMEDIATE), so changes get their order with no conflicts. Reads run beside them. - One process per file. Another process on the same file waits for the lock, and needs a
busy_timeoutto wait at all. - SQLite 3.38 or later (
json_each), as every current Node, Bun and better-sqlite3 ships.
Postgres
import { PGlite } from '@electric-sql/pglite';
import { nodePostgres, pglite, postgresJs, postgresStore } from 'ferrysync/server';
import { Pool } from 'pg';
import postgres from 'postgres';
const url = process.env.DATABASE_URL ?? '';
const withPg = postgresStore(nodePostgres(new Pool({ connectionString: url })));
const withPostgresJs = postgresStore(postgresJs(postgres(url)));
const withPglite = postgresStore(pglite(new PGlite('./sync-data')));- Writes run in
READ COMMITTEDand lock only what they touch, so many instances write at once; each change still gets one place in the order. Reads see one snapshot (REPEATABLE READ). - Neon, Supabase, RDS, Cloud SQL: any Postgres works, directly or through a pooler in transaction
mode (PgBouncer, Supavisor). Behind such a pooler, give postgres.js
prepare: false. - PGlite has one connection and runs its transactions one at a time: fine for tests, local-first desktop apps and demos.
D1
import { d1Store } from 'ferrysync/server';
declare const env: { DB: D1Database };
const store = d1Store(env.DB);- D1 has no transactions to read inside, only
batch(). Sod1Storetakes the binding itself, not a driver, keeps each transaction's writes in memory and commits them as one batch. - It must run inside one Durable Object per database, the one writer: see
Cloudflare Workers. A second writer
makes requests fail with
D1ConflictError(nothing is written, and devices retry), never corrupts the log. - Send the Worker's requests to that object with
durableObjectCoordinator(env.SYNC).fetch(request): one object per D1 database, because sync ids are one gapless sequence per database. - Same tables as the SQLite store. No
within, nomirror. Its options areprefix,migrateandcreateId(the ids of its commit tokens).
Durable Object SQLite
import {
type DurableObjectSqliteStorage,
durableObjectSqlite,
sqliteStore,
} from 'ferrysync/server';
declare const ctx: { storage: DurableObjectSqliteStorage };
const store = sqliteStore(durableObjectSqlite(ctx.storage));- The store lives inside the object, in its own SQLite storage, with the SQLite store's tables and migrations. Transactions run one at a time.
- It cannot join a transaction of yours (
within). One object per sync authority: see Durable Objects.
ORMs: Prisma and Drizzle
When your backend has an ORM, give the store its client: sync then runs on the ORM's connections, and can join its transactions.
import { prismaDriver, postgresStore } from 'ferrysync/server';
import { PrismaClient } from '@prisma/client';
const prisma = new PrismaClient();
export const store = postgresStore(prismaDriver(prisma, { dialect: 'postgres' }));import { drizzleDriver, postgresStore } from 'ferrysync/server';
import { sql } from 'drizzle-orm';
import { drizzle } from 'drizzle-orm/node-postgres';
import { Pool } from 'pg';
const db = drizzle(new Pool({ connectionString: process.env.DATABASE_URL }));
export const store = postgresStore(drizzleDriver(db, { dialect: 'postgres', sql }));dialectsays which database is behind the ORM:'postgres'or'sqlite', withpostgresStoreorsqliteStoreto match.- Prisma:
prismaDriver(prisma, { dialect, timeout })passestimeout(andmaxWait) to$transaction. Prisma's default is 5 seconds; a first download of a large account reads every row in one transaction, so raise it if yours are large. - Drizzle on SQLite needs one connection (better-sqlite3, bun:sqlite, a local libsql file): the driver begins and commits transactions itself.
Options
import { nodePostgres, postgresStore } from 'ferrysync/server';
import { Pool } from 'pg';
export const store = postgresStore(nodePostgres(new Pool()), {
prefix: 'sync_',
migrate: false,
});| Option | Default | What it does |
|---|---|---|
prefix | ferry_ | Starts the name of every table and index: 1 to 40 lowercase letters, digits or _ |
migrate | true | Creates the tables on first use, and upgrades them after a release that changes them |
mirror | none | Copies tables of the schema into tables of yours: see Mirrors |
store.ready() resolves once the tables are ready (created, upgraded or checked). Every other call
waits for it, so calling it is optional: call it at startup to fail early.
Migrations
With migrate: true (the default) the store creates its tables the first time it is used, and
upgrades them when a new release of Ferry changes them. Nothing to run.
When your migrations are the only thing allowed to change the database (Prisma Migrate, drizzle-kit,
a SQL folder), put the store's tables in them, and open the store with migrate: false:
// scripts/ferry-migration.ts: writes the statements into a migration of yours
import { syncTablesSql } from 'ferrysync/server';
import { writeFileSync } from 'node:fs';
const statements = syncTablesSql({ dialect: 'postgres', prefix: 'ferry_' });
writeFileSync('migrations/0007_ferry/migration.sql', `${statements.join(';\n')};\n`);- Each statement creates something only if it is missing, so running them again on tables they made is safe.
- With
migrate: false, the store checks at first use that the tables are at the version this release needs, and fails with a message that says what to run when they are not. - A release of Ferry that changes these tables says so in its notes, with what to add to your
migrations. Until you add it, a store with
migrate: falserefuses to start. - Version 2 adds a nullable
updated_atcolumn (and an index on it) toferry_push_results, which retention prunes by.syncTablesSqlcreates it. For tables a version 1 script made, add it withALTER TABLE ferry_push_results ADD COLUMN updated_at bigint(INTEGERon SQLite) andCREATE INDEX ferry_push_results_by_time ON ferry_push_results (updated_at). Rows without a time are kept. A store withmigrate: truedoes this itself. - Version 3 adds a nullable
updated_atcolumn (and an index on it) toferry_clients: when each device last sent a change, which retention forgets devices by.syncTablesSqlcreates it. For tables a version 2 script made, addALTER TABLE ferry_clients ADD COLUMN updated_at bigint(INTEGERon SQLite) andCREATE INDEX ferry_clients_by_time ON ferry_clients (updated_at), then setschema_versionto3inferry_meta. Rows without a time get one at the first prune, so no device is forgotten right after the upgrade. A store withmigrate: truedoes this itself.
The tables are ferry_state, ferry_clients, ferry_memberships, ferry_records, ferry_log,
ferry_push_results and ferry_meta.
Gotchas
- Never write to the
ferry_tables yourself. Change synced data throughsyncServer.write, so devices hear about it. - postgres.js: no
transform. The store reads columns by their names in the database, so pass an instance without a transform that renames them. - No string may hold the character U+0000, which Postgres cannot store. The server refuses such
a row as
invalidon every store, so an app moving from SQLite to Postgres sees no difference. - PGlite and other one-connection databases with
within: callstore.ready()before your own transaction holds the connection, sinceready()needs a transaction of its own.
Hooks
Hooks put your backend's rules into sync, per table: validate refuses writes, onWrite writes more in the same transaction, and afterCommit reacts once a change is saved.
Adopting Ferry in an existing backend
An existing app has its own tables, ids, ORM and migrations. Ferry fits in beside them: it keeps its own tables in your database, mirrors synced tables into yours, and takes over one feature at a time.