Retention

import { createSyncServer, memoryStore } from 'ferrysync/server';

import { schema } from './schema';

export const syncServer = createSyncServer({
  schema,
  store: memoryStore(),
  auth: async (request) => (await getSession(request))?.userId ?? null,
  retention: { days: 30 }, // default 90; `false` keeps everything
});

What it dropsCopy Anchor Link

Only history. Every row's current state stays, whatever its age: the log's entries are the changes that led there, which devices need only to catch up from where they left off.

It also forgets the answers the server stored to pushes (ferry_push_results) once they are older than days. The server keeps them so that a device that never heard the answer can resend its transaction and get the same one. After days, a resend of that transaction gets forgotten instead: the change is not applied twice, and the device works out what became of it from the server's data once it has caught up.

• If the server's rows show the change (every row it creates or edits is there, as it left it), the change is confirmed.
• Otherwise the device rolls it back, as for any refused write, and onRejected gets the code 'forgotten': the server refused it, someone changed those rows since, or it was a delete, which the device cannot tell from a row it no longer sees.

Either way the device ends with what the server has. Only what your app hears can differ from the first answer, so choose days above how long your users' devices stay offline. Devices on a release from before forgotten existed get applied instead, and see such a change as accepted until their next pull brings the server's rows.

And it forgets the devices that stopped sending changes: see the next section.

Devices that stop syncingCopy Anchor Link

Every app install that has sent a change has one small row on the server (ferry_clients): which user it belongs to, and how many of its changes the server has handled, so that a change sent twice is applied once. An app that is deleted never says goodbye, so without pruning these rows would pile up forever.

So the server forgets a device that has sent no change for clients.days: 365 days by default, or days if you keep the log longer. Its rows on the server are gone, and so are the answers stored for it. Nothing on the device is touched.

import { createSyncServer, memoryStore } from 'ferrysync/server';

import { schema } from './schema';

export const syncServer = createSyncServer({
  schema,
  store: memoryStore(),
  auth: async (request) => (await getSession(request))?.userId ?? null,
  // Keep the log 90 days, and remember a device for two years after its last change.
  retention: { days: 90, clients: { days: 730 } },
});

When a forgotten device comes backCopy Anchor Link

The app on that phone still works, and may hold changes it never sent. This is what happens when it syncs again:

1. The device sends its waiting changes. The server does not know it any more, so it answers forgottenClient ("I don't remember you") and applies nothing.

   You should see: nothing in the app. The device keeps its changes and keeps showing them.

2. The device downloads what changed while it was away, as on any sync (or everything, if it was away longer than days).

3. It checks its waiting changes against what it downloaded, oldest first. A change that already shows in the server's data reached the server before it was forgotten (its answer was lost on the way back): it is confirmed, and not sent again.

4. From the first change the data does not show, the device takes a new identity (a new client id) and sends the rest again, numbered from 1, as a new install would.

   You should see: the changes on the user's other devices, once. The device's status goes back to idle with nothing pending.

Your app has nothing new to handle. A change the device knows the server had already handled (it had an answer for a later one) is never sent again: it is confirmed from the data, or rolled back with the code 'forgotten', as above.

When it runsCopy Anchor Link

After a request, at most once an hour per server instance, the server drops the entries and the push answers older than days, and forgets the devices quiet for longer than clients.days. It runs in the background of that request: through waitUntil when the server has it, so serverless platforms keep the function alive for it.

No cron job, no scheduled function: a server that gets requests prunes, and one that gets none has nothing new to prune. To run it yourself (a script, a test), call syncServer.protocol.pruneClients(before) with an epoch ms.

To see it work, turn on metrics: ferry_log_entries shrinks after a prune, and ferry_retention_pruned_total adds up the entries each prune dropped.

A device behind the logCopy Anchor Link

A device that was offline longer than the retention finds its place in the log gone. The server tells it so, and it downloads everything again, as on its first sync:

• its pending changes are kept, and sent afterwards;
• the user sees nothing but the time a full download takes.

So choose days above how long your users' devices stay offline: a phone in a drawer, a laptop over the holidays. The default, 90, covers most apps.

OptionsCopy Anchor Link

GotchasCopy Anchor Link

• Each instance prunes on its own clock, at most hourly. With several, the log is pruned about as often as each instance gets requests. That is fine: pruning twice drops nothing more.
• Retention is not deletion. A deleted row stays in the store as a tombstone, so a late update from an offline device cannot bring it back. Pruning the log does not remove tombstones. To erase a user's data for good, see Deleting users.
• A failed prune goes to onError, fails no request, and is tried again an hour later.
• A change whose answer was lost right before the device went quiet is the one case the server cannot settle once it has forgotten the device. If the server applied it, and someone changed those rows afterwards, its effects no longer show, and the device cannot tell it from a change that never arrived. It sends it again, and it lands as a late write does: last writer wins, and a create can bring back a row someone deleted. The same goes for a device whose very first push was answered and lost. It takes a lost answer, then more than clients.days of silence, then someone editing the same rows, so keep clients.days well above how long your devices stay away.
• A device that only reads is forgotten too, since only pushes count. That costs it nothing: it keeps pulling, and its next change takes the steps above.
• After upgrading to this release (store version 3), devices the server knew before have no time yet. The first prune gives them its own, so none is forgotten until clients.days after it.
• Apps on an older release do not know forgottenClient: they get conflict, take a new client id and send all their waiting changes again, without checking the data first.
• A store of your own forgets devices only if it has pruneClients (see stores); without it the server keeps every device.