Remote Settings

The remote-settings.js module offers the ability to fetch remote settings that are kept in sync with Mozilla servers.

Usage

The get() method returns the list of entries for a specific key. Each entry can have arbitrary attributes, and can only be modified on the server.

const { RemoteSettings } = ChromeUtils.import("resource://services-settings/remote-settings.js", {});

const data = await RemoteSettings("a-key").get();

/*
  data == [
    {label: "Yahoo",  enabled: true,  weight: 10, id: "d0782d8d", last_modified: 1522764475905},
    {label: "Google", enabled: true,  weight: 20, id: "8883955f", last_modified: 1521539068414},
    {label: "Ecosia", enabled: false, weight: 5,  id: "337c865d", last_modified: 1520527480321},
  ]
*/

for(const entry of data) {
  // Do something with entry...
  // await InternalAPI.load(entry.id, entry.label, entry.weight);
});

Note

The id and last_modified (timestamp) attributes are assigned by the server.

Empty local database

On new user profiles or for recently added use-cases, the local database will be empty until a synchronization with the server happens. Synchronizations are managed internally, and can sometimes be triggered minutes after browser starts.

By default, if .get() is called before the local database had the chance to be synchronized, and if no initial data was provided (see below), then the settings will be pulled from the server in order to avoid returning an empty list. In that case, the first call to .get() will thus take longer than the following ones.

This behaviour can be disabled using the syncIfEmpty option.

Important

If the implicit synchronization fails (e.g network is not available) then errors are silent and an empty list is returned. Uptake Telemetry status is sent though.

Options

  • filters, order: The list can optionally be filtered or ordered:

    const subset = await RemoteSettings("a-key").get({
    filters: {
        "property": "value"
    },
    order: "-weight"
    });
    
  • syncIfEmpty: implicit synchronization if local data is empty (default: true). Set it to false if your use-case can tolerate an empty list until the first synchronization happens.

    await RemoteSettings("a-key").get({ syncIfEmpty: false });
    

Events

The on() function registers handlers to be triggered when events occur.

The sync event allows to be notified when the remote settings are changed on the server side. Your handler is given an event object that contains a data attribute that has information about the changes:

  • current: current list of entries (after changes were applied);
  • created, updated, deleted: list of entries that were created/updated/deleted respectively.
RemoteSettings("a-key").on("sync", event => {
  const { data: { current } } = event;
  for(const entry of current) {
    // Do something with entry...
    // await InternalAPI.reload(entry.id, entry.label, entry.weight);
  }
});

Note

Currently, the synchronization of remote settings is triggered via push notifications, and also by its own timer every 24H (see the preference services.settings.poll_interval ).

File attachments

When an entry has a file attached to it, it has an attachment attribute, which contains the file related information (url, hash, size, mimetype, etc.). Remote files are not downloaded automatically.

const data = await RemoteSettings("a-key").get();

data.filter(d => d.attachment)
    .forEach(async ({ attachment: { url, filename, size } }) => {
      if (size < OS.freeDiskSpace) {
        // Planned feature, see Bug 1501214
        await downloadLocally(url, filename);
      }
    });

Initial data

It is possible to package a dump of the server records that will be loaded into the local database when no synchronization has happened yet.

The JSON dump will serve as the default dataset for .get(), instead of doing a round-trip to pull the latest data. It will also reduce the amount of data to be downloaded on the first synchronization.

  1. Place the JSON dump of the server records in the services/settings/dumps/main/ folder
  2. Add the filename to the FINAL_TARGET_FILES list in services/settings/dumps/main/moz.build

Now, when RemoteSettings("some-key").get() is called from an empty profile, the some-key.json file is going to be loaded before the results are returned.

Note

JSON dumps are not shipped on Android to minimize the installer size.

Targets and A/B testing

In order to deliver settings to subsets of the population, you can set targets on entries (platform, language, channel, version range, preferences values, samples, etc.) when editing records on the server.

From the client API standpoint, this is completely transparent: the .get() method — as well as the event data — will always filter the entries on which the target matches.

Note

The remote settings targets follow the same approach as the Normandy recipe client (ie. JEXL filter expressions),

Uptake Telemetry

Some uptake telemetry is collected in order to monitor how remote settings are propagated.

It is submitted to a single keyed histogram whose id is UPTAKE_REMOTE_CONTENT_RESULT_1 and the keys are prefixed with main/ (eg. main/a-key in the above example).

Create new remote settings

Staff members can create new kinds of remote settings, following this documentation.

It basically consists in:

  1. Choosing a key (eg. search-providers)
  2. Assigning collaborators to editors and reviewers groups
  3. (optional) Define a JSONSchema to validate entries
  4. (optional) Allow attachments on entries

And once done:

  1. Create, modify or delete entries and let reviewers approve the changes
  2. Wait for Firefox to pick-up the changes for your settings key

Global Notifications

The polling for changes process sends two notifications that observers can register to:

  • remote-settings:changes-poll-start: Polling for changes is starting. triggered either by the scheduled timer or a push broadcast.
  • remote-settings:changes-poll-end: Polling for changes has ended
const observer = {
  observe(aSubject, aTopic, aData) {
    Services.obs.removeObserver(this, "remote-settings:changes-poll-start");

    const { expectedTimestamp } = JSON.parse(aData);
    console.log("Polling started", expectedTimestamp ? "from push broadcast" : "by scheduled trigger");
  },
};
Services.obs.addObserver(observer, "remote-settings:changes-poll-start");

Advanced Options

filterFunc: custom filtering function

By default, the entries returned by .get() are filtered based on the JEXL expression result from the filter_expression field. The filterFunc option allows to execute a custom filter (async) function, that should return the record (modified or not) if kept or a falsy value if filtered out.

RemoteSettings("a-collection", {
  filterFunc: (record, environment) => {
    const { enabled, ...entry } = record;
    return enabled ? entry : null;
  }
});

Debugging and manual testing

Remote Settings Dev Tools

The Remote Settings Dev Tools extension provides some tooling to inspect synchronization statuses, to change the remote server or to switch to preview mode in order to sign-off pending changes. More information on the dedicated repository.

Trigger a synchronization manually

The synchronization of every known remote settings clients can be triggered manually with pollChanges():

await RemoteSettings.pollChanges()

The synchronization of a single client can be forced with the .sync() method:

await RemoteSettings("a-key").sync();

Important

The above methods are only relevant during development or debugging and should never be called in production code.

Inspect local data

The internal IndexedDB of Remote Settings can be accessed via the Storage Inspector in the browser toolbox.

For example, the local data of the "key" collection can be accessed in the remote-settings database at Browser Toolbox > Storage > IndexedDB > chrome, in the records store.

Unit Tests

As a foreword, we would like to underline the fact that your tests should not test Remote Settings itself. Your tests should assume Remote Settings works, and should only run assertions on the integration part. For example, if you see yourself mocking the server responses, your tests may go over their responsability.

If your code relies on the "sync" event, you are likely to be interested in faking this event and make sure your code runs as expected. If it relies on .get(), you will probably want to insert some fake local data.

Simulate "sync" events

You can forge a payload that contains the events attributes as described above, and emit it :)

const payload = {
  current: [{ id: "", age: 43 }],
  created: [],
  updated: [{ old: { id: "abc", age: 42 }, new: { id: "abc", age: 43 }}],
  deleted: [],
};

await RemoteSettings("a-key").emit("sync", { "data": payload });

Manipulate local data

A handle on the local collection can be obtained with openCollection().

const collection = await RemoteSettings("a-key").openCollection();

And records can be created manually (as if they were synchronized from the server):

const record = await collection.create({
  domain: "website.com",
  usernameSelector: "#login-account",
  passwordSelector: "#pass-signin",
}, { synced: true });

In order to bypass the potential target filtering of RemoteSettings("key").get(), the low-level listing of records can be obtained with collection.list():

const subset = await collection.list({
  filters: {
    "property": "value"
  }
});

The local data can be flushed with clear():

await collection.clear()

For further documentation in collection API, checkout the kinto.js library, which is in charge of the IndexedDB interactions behind-the-scenes.

Misc

We host more documentation on https://remote-settings.readthedocs.io/, on how to run a server locally, manage attachments, or use the REST API etc.

About blocklists

Addons, certificates, plugins, and GFX blocklists were the first use-cases of remote settings, and thus have some specificities.

For example, they leverage advanced customization options (bucket, content-signature certificate, target filtering etc.), and in order to be able to inspect and manipulate their data, the client instances must first be explicitly initialized.

const BlocklistClients = ChromeUtils.import("resource://services-common/blocklist-clients.js", {});

BlocklistClients.initialize();

Then, in order to access a specific client instance, the bucket must be specified:

const collection = await RemoteSettings("addons", { bucketName: "blocklists" }).openCollection();

And in the storage inspector, the IndexedDB internal store will be prefixed with blocklists instead of main (eg. blocklists/addons).