Realtime and scaling

Long pollingCopy Anchor Link

Each online device keeps one wait request open on the server. The server answers it as soon as something the user can see changes; the device then pulls the change and opens the next wait.

It is on by default, and needs nothing on the server: fetch answers wait like any endpoint. It works through every framework, proxy and serverless platform, because it is ordinary requests.

A device that turns realtime off (http({ url, realtime: false })) syncs only after its own writes and when the app asks: see the client's realtime.

How a change wakes a waitCopy Anchor Link

When a transaction commits, the server publishes a notice: the spaces whose rows changed, and the users whose memberships changed. Every held wait hears it, and those it concerns answer changed at once, from the notice alone: it already says what changed and up to which sync id, so waking a thousand waits costs the database nothing. Each wait reads the database once, when it arrives, to see whether there is already something to pull.

import type { SyncNotice } from 'ferrysync/server';

const notice: SyncNotice = {
  lastSyncId: 1204, // the log's last id after the commit
  spaces: ['project:p1'], // whose rows changed
  userIds: [], // whose memberships changed
};

Notices are hints, not data. A lost one costs time: the wait finds the change at its next look, at most 15 seconds later. A late or repeated one costs nothing: a wait skips notices of changes it has already seen. Neither ever costs a change.

Several instancesCopy Anchor Link

Notices travel in memory by default, inside one process. With several instances behind a load balancer, a change applied by one instance wakes only the waits that instance holds; the others find it within 15 seconds.

To make it instant everywhere, pass a broadcast that every instance shares:

import type { SyncNotice } from 'ferrysync/server';

interface Broadcast {
  /** Sends a notice to every instance, this one included. */
  publish(notice: SyncNotice): void | Promise<void>;
  /** Hears the notices of every instance. Returns the unsubscribe. */
  subscribe(listener: (notice: SyncNotice) => void): () => void;
}

A failed publish goes to onError and never fails the write that committed.

On Cloudflare, with D1 or a Durable Object's own SQLite, every request of a database goes through one object, so the default memoryBroadcast() already wakes every wait at once: no broadcast to set up. WebSockets instead of long polling are not built.

RedisCopy Anchor Link

Redis pub/sub, with ioredis. A subscribed connection cannot publish, so it takes two:

import type { Broadcast, SyncNotice } from 'ferrysync/server';
import Redis from 'ioredis';

export const redisBroadcast = (url: string, channel = 'ferry:notices'): Broadcast => {
  const publisher = new Redis(url);
  const subscriber = new Redis(url);
  const listeners = new Set<(notice: SyncNotice) => void>();

  subscriber.subscribe(channel).catch((error: unknown) => console.error(error));
  subscriber.on('message', (from: string, message: string) => {
    if (from !== channel) {
      return;
    }
    const notice: SyncNotice = JSON.parse(message);
    for (const listener of listeners) {
      listener(notice);
    }
  });

  return {
    publish: async (notice) => {
      await publisher.publish(channel, JSON.stringify(notice));
    },
    subscribe: (listener) => {
      listeners.add(listener);
      return () => {
        listeners.delete(listener);
      };
    },
  };
};

Then give every instance's server the same channel: createSyncServer({ …, broadcast: redisBroadcast(process.env.REDIS_URL) }).

Postgres LISTEN/NOTIFYCopy Anchor Link

When the store is Postgres, the database can carry the notices: no other service. One connection of its own listens; the pool publishes.

import type { Broadcast, SyncNotice } from 'ferrysync/server';
import { Client, type Pool } from 'pg';

export const postgresBroadcast = async (
  pool: Pool,
  connectionString: string,
  channel = 'ferry_notices',
): Promise<Broadcast> => {
  const listeners = new Set<(notice: SyncNotice) => void>();
  const listening = new Client({ connectionString });
  listening.on('notification', (message) => {
    if (message.channel !== channel || message.payload === undefined) {
      return;
    }
    const notice: SyncNotice = JSON.parse(message.payload);
    for (const listener of listeners) {
      listener(notice);
    }
  });
  await listening.connect();
  await listening.query(`LISTEN ${channel}`);

  return {
    publish: async (notice) => {
      await pool.query('SELECT pg_notify($1, $2)', [channel, JSON.stringify(notice)]);
    },
    subscribe: (listener) => {
      listeners.add(listener);
      return () => {
        listeners.delete(listener);
      };
    },
  };
};

Then pass it with the store's pool: createSyncServer({ …, store: postgresStore(nodePostgres(pool)), broadcast: await postgresBroadcast(pool, url) }).

• The listening connection must be direct, not through a pooler in transaction mode, which does not keep LISTEN across transactions.
• A notice must fit in 8000 bytes, Postgres's limit for a payload. One commit that changes hundreds of spaces makes a larger one: its publish fails (to onError), and the waits find the change at their next look.
• Reconnect when the listening connection drops (listening.on('error', …)): until then, waits fall back to their 15-second looks.

Open requestsCopy Anchor Link

Every online device holds one request open most of the time, so a server with 2,000 devices online holds about 2,000 requests. Each costs little memory and no CPU while it waits. Plan for it:

• File descriptors: raise the process's limit (ulimit -n) on a server with many devices.
• Timeouts: everything between the device and the server (proxies, load balancers, the platform) must allow a request that sends nothing for 30 seconds. See Deploy.
• Serverless: a held request is a running invocation. Check how your platform bills one.
• Per user: an instance holds at most maxHeldWaitsPerUser waits for one user (default 10, a positive integer). When an eleventh arrives, the oldest is answered timeout and its device waits again, so a user's many tabs and devices cannot pile up connections. Raise it only if one user really has more than ten devices online on one instance.
• Watch them: with the server's metrics option, ferry_waits_held is how many requests an instance holds right now (add them up across instances), ferry_waits_total counts each wait by how it ended (changed or timeout, and whether it was held at all), and ferry_waits_evicted_total counts the waits the per-user limit ended. A timeout also includes a device that went away and a close(). See Metrics.

GotchasCopy Anchor Link

• More instances do not mean more writes per second. They hold more open requests and run more of each push side by side, but every push to one database takes its turn at the end, where it gets its sync ids: that turn is what keeps every device's view in order. On the benchmark laptop one Postgres takes about 1,600 pushes a second that way. Past that, give each tenant a database of its own.
• Realtime is a nudge to pull, not a stream of data. The data always comes from pulls, in log order, so devices converge even when notices are lost.
• close() ends held waits: they answer at once, and devices open their next one, on another instance after a deploy.
• A broadcast must deliver to its own instance too. Both snippets do: Redis and Postgres deliver a message to every subscriber, the publisher's included.
• A broadcast must pass each notice on as it was published. A held wait answers from it without reading the database, so a notice with the wrong lastSyncId or spaces wakes the wrong devices, or none, until their 15-second look. Sending it as JSON, as both snippets do, keeps it whole.