Choose and set up a store

A store keeps every scope your application syncs. Use a SQLite file to keep sync state between runs, memory in tests, or your own storage where your application keeps everything else.

Keep sync state in SQLite

import sqlite3

from calendaring_sync import Replica, SQLiteStore

connection = sqlite3.connect("sync.db")
connection.execute("PRAGMA journal_mode = WAL")
connection.close()

store = SQLiteStore("sync.db", timeout=10.0)
replica = Replica(store)

Set the file’s journal mode to WAL once, before you open it. With WAL, each edit and each push writes several times faster, which counts when your application records edits one at a time. SQLiteStore says what else WAL changes, and why it doesn’t suit a network filesystem. Several processes can share the file, as Sync from several threads or processes shows.

Retry a busy store

import time

from calendaring_sync import StoreBusyError

for _ in range(3):
    try:
        adapter.sync()
        break
    except StoreBusyError:
        time.sleep(1)

timeout sets how long SQLiteStore waits for another process before it gives up.

Back up and restore the store

Back up the store file together with everything your application derived from the change feed, such as your cursor and a mirrored table, as of the same moment. Copy the file while no process has it open, or use sqlite3.Connection.backup(), which copies a consistent snapshot of a file in use. A plain copy of a file in use can miss the changes WAL keeps in a file beside it.

Restoring an older copy loses nothing on the server. The next sync asks the server for every change since the copy’s sync token, or lists the scope in full where the server no longer accepts it. An edit the copy still holds as pending, which reached the server after the copy was made, is matched to the server’s copy before anything is sent. A create whose id the server picks has nothing to match by, so it goes out again. Where logical keys are unique, the server refuses it, and it becomes a conflict for you to resolve; elsewhere, the server can end up holding it twice.

The restored feed continues from the copy, so it numbers new entries again from there, and a cursor saved after the copy was made skips entries without an error. If you restore the store alone, rebuild what you derived from the feed from records(), and set your cursor to feed_head(), as Read the change feed does after a trim.

Keep sync state in memory

MemoryStore suits tests, as Test code that uses calendaring-sync shows.

Keep sync state in your own storage

To keep sync state in a file of your own, give MemoryStore a MemoryKeeper. The store loads its data from the keeper once, when it opens, and hands it to the keeper each time a write commits:

import pickle
from pathlib import Path

from calendaring_sync import MemoryData, MemoryKeeper, MemoryStore, Replica, StoreError


class FileKeeper(MemoryKeeper):
    """Keeps a memory store's data in one file, replaced whole on each save."""

    def __init__(self, path: Path) -> None:
        self.path = path

    def load(self) -> MemoryData | None:
        if not self.path.exists():
            return None
        data: MemoryData = pickle.loads(self.path.read_bytes())
        return data

    def save(self, data: MemoryData) -> None:
        # Written beside the file and moved over it, so a failure halfway keeps the last save.
        written = self.path.with_name(self.path.name + ".new")
        written.write_bytes(pickle.dumps(data))
        written.replace(self.path)

    def error(self, exc: Exception) -> StoreError | None:
        return StoreError(str(exc)) if isinstance(exc, OSError) else None


replica = Replica(MemoryStore(keeper=FileKeeper(Path("sync.pickle"))))

Only load a file your application wrote: unpickling a file runs code from it. Every lookup scans the records in memory, so for a large store use SQLite.

For a database that looks records up itself, implement Storage and StorageTransaction, and pass the storage to StateStore, which does everything but the reading and writing.