Introduction to Sync

This document is a brief introduction to how Sync is implemented in desktop Firefox.

General, Historical, Anatomy of a Sync Engine

This section describes how Sync used to work - and indeed, how much of it still does. While we discuss how this is slowly changing, this context is valuable.

For any datatype which syncs, there tends to be 3 parts:

Store

The sync store interfaces with the actual Firefox desktop store. For example, in the passwords engine, the “store” is that layer that talks to Services.logins

Tracker

The tracker is what knows that something should be synced. For example, when the user creates or updates a password, it is the tracker that knows we should sync now, and what particular password(s) should be updated.

This is typically done via “observer” notifications - Services.logins, places etc all send specific notifications when certain events happen (and indeed, some of these were added for Sync’s benefit)

Engine

The engine ties it all together. It works with the store and tracker and tracks its own metadata (eg, the timestamp of the passwords on the server, so it knows how to grab just changed records and how to pass them off to the store so the actual underlying storage can be updated.

All of the above parts were typically in the services/sync/modules/engines directory directory and decoupled from the data they were syncing.

The Future of Desktop-Specific Sync Engines

The system described above reflects the fact that Sync was “bolted on” to Desktop Firefox relatively late - eg, the Sync store is decoupled from the actual store. This has causes a number of problems - particularly around the tracker and the metadata used by the engine, and the fact that changes to the backing store would often forget that Sync existed.

Over the last few years, the Sync team has come to the conclusion that Sync support must be integrated much closer to the store itself. For example, Services.logins should track when something has changed that would cause an item to be synced. It should also track the metadata for the store so that if (say) a corrupt database is recovered by creating a new, empty one, the metadata should also vanish so Sync knows something bad has happened and can recover.

However, this is a slow process - currently the bookmarks, history and passwords legacy engines have been improved so more responsibility is taken by the stores. In all cases, for implementation reasons, the Sync implementation still has a store, but it tends to be a thin wrapper around the actual underlying store.

The Future of Cross-Platform Sync Engines

There are a number of Sync engines implemented in Rust and which live in the application-services repository. While these were often done for mobile platforms, the longer term hope is that they can be reused on Desktop. The Sync engines in the tree has more details on these.

While no existing engines have been replaced with Rust implemented engines, the webext-storage engine is implemented in Rust via application-services, so doesn’t tend to use any of the infrastructure described above.

Hopefully over time we will find more Rust-implemented engines in Desktop.