“sync” ping

This is an aggregated format that contains information about each sync that occurred during a timeframe. It is submitted every 12 hours, and on browser shutdown, but only if the syncs property would not be empty. The ping does not contain the environment block, nor the clientId or profileGroupId.

Each item in the syncs property is generated after a sync is completed, for both successful and failed syncs, and contains measurements pertaining to sync performance and error information.

A JSON-schema document describing the exact format of the ping’s payload property can be found at services/sync/tests/unit/sync_ping_schema.json.

Structure:

{
  version: 4,
  type: "sync",
  ... common ping data
  payload: {
    version: 1,
    os : { ... }, // os data from the current telemetry environment. OS specific, but typically includes name, version and locale.
    discarded: <integer count> // Number of syncs discarded -- left out if zero.
    why: <string>, // Why did we submit the ping? Either "shutdown", "schedule", or "idchanged".
    uid: <string>, // Hashed FxA unique ID, or string of 32 zeros. If this changes between syncs, the payload is submitted.
    deviceID: <string>, // Hashed FxA Device ID, hex string of 64 characters, not included if the user is not logged in. If this changes between syncs, the payload is submitted.
    sessionStartDate: <ISO date>, // Hourly precision, ISO date in local time
    // Array of recorded syncs. The ping is not submitted if this would be empty
    syncs: [{
      when: <integer milliseconds since epoch>,
      took: <integer duration in milliseconds>,
      didLogin: <bool>, // Optional, is this the first sync after login? Excluded if we don't know.
      why: <string>, // Optional, why the sync occurred, excluded if we don't know.

      // Optional, excluded if there was no error.
      failureReason: {
        name: <string>, // "httperror", "networkerror", "shutdownerror", etc.
        code: <integer>, // Only present for "httperror" and "networkerror".
        error: <string>, // Only present for "othererror" and "unexpectederror".
        from: <string>, // Optional, and only present for "autherror".
      },

      // Optional, excluded if we couldn't get a valid uid or local device id
      devices: [{
        os: <string>, // OS string as reported by Services.appinfo.OS, if known
        version: <string>, // Firefox version, as reported by Services.appinfo.version if known
        id: <string>, // Hashed FxA device id for device
        type: <string>, // broad device "type", as reported by fxa ("mobile", "tv", etc).
        syncID: <string>, // Hashed Sync device id for device, if the user is a sync user.
      }],

      // Internal sync status information. Omitted if it would be empty.
      status: {
        sync: <string>, // The value of the Status.sync property, unless it indicates success.
        service: <string>, // The value of the Status.service property, unless it indicates success.
      },
      // Information about each engine's sync.
      engines: [
        {
          name: <string>, // "bookmarks", "tabs", etc.
          took: <integer duration in milliseconds>, // Optional, values of 0 are omitted.

          status: <string>, // The value of Status.engines, if it holds a non-success value.

          // Optional, excluded if all items would be 0. A missing item indicates a value of 0.
          incoming: {
            applied: <integer>, // Number of records applied
            succeeded: <integer>, // Number of records that applied without error
            failed: <integer>, // Number of records that failed to apply
            failedReasons: [
              name: <string> // Error message trying to apply the record
              count: <integer> // count of how many times this error occurred
            ],
          },

          // Optional, excluded if it would be empty. Records that would be
          // empty (e.g. 0 sent and 0 failed) are omitted.
          outgoing: [
            {
              sent: <integer>, // Number of outgoing records sent. Zero values are omitted.
              failed: <integer>, // Number that failed to send. Zero values are omitted.
              failedReasons: [
              name: <string> // Error message trying to apply the record
              count: <integer> // count of how many times this error occurred
            ],
            }
          ],
          // Optional, excluded if there were no errors
          failureReason: { ... }, // Same as above.

          // Timings and counts for detailed steps that the engine reported
          // as part of its sync. Optional; omitted if the engine didn't
          // report any extra steps.
          steps: {
            name: <string>, // The step name.
            took: <integer duration in milliseconds>, // Omitted if 0.
            // Optional, extra named counts (e.g., number of items handled
            // in this step). Omitted if the engine didn't report extra
            // counts.
            counts: [
              {
                name: <string>, // The counter name.
                count: <integer>, // The counter value.
              },
            ],
          },

          // Optional, excluded if it would be empty or if the engine cannot
          // or did not run validation on itself.
          validation: {
            // Optional validator version, default of 0.
            version: <integer>,
            checked: <integer>,
            took: <non-monotonic integer duration in milliseconds>,
            // Entries with a count of 0 are excluded, the array is excluded if no problems are found.
            problems: [
              {
                name: <string>, // The problem identified.
                count: <integer>, // Number of times it occurred.
              }
            ],
            // Format is same as above, this is only included if we tried and failed
            // to run validation, and if it's present, all other fields in this object are optional.
            failureReason: { ... },
          }
        }
      ],
      // Information about any storage migrations that have occurred. Omitted if it would be empty.
      migrations: [
        // See the section on the `migrations` array for detailed documentation on what may appear here.
        {
          type: <string identifier>,
          // per-type data
        }
      ]
    }],
    // The "node type" as reported by the token server. This will not change
    // from sync to sync, so is reported once per ping. Optional because it
    // will not appear if the token server omits this information, but in
    // general, we will expect all "new" pings to have it.
    syncNodeType: <string>,
    events: [
      event_array // See events below.
    ],
    histograms: { ... } // See histograms below
  }
}

info

discarded

The ping may only contain a certain number of entries in the "syncs" array, currently 500 (it is determined by the "services.sync.telemetry.maxPayloadCount" preference). Entries beyond this are discarded, and recorded in the discarded count.

syncs.took

These values should be monotonic. If we can’t get a monotonic timestamp, -1 will be reported on the payload, and the values will be omitted from the engines. Additionally, the value will be omitted from an engine if it would be 0 (either due to timer inaccuracy or finishing instantaneously).

uid

This property containing a hash of the FxA account identifier, which is a 32 character hexadecimal string. In the case that we are unable to authenticate with FxA and have never authenticated in the past, it will be a placeholder string consisting of 32 repeated 0 characters.

syncs.why

One of the following values:

  • startup: This is the first sync triggered after browser startup.

  • schedule: This is a sync triggered because it has been too long since the last sync.

  • score: This sync is triggered by a high score value one of sync’s trackers, indicating that many changes have occurred since the last sync.

  • user: The user manually triggered the sync.

  • tabs: The user opened the synced tabs sidebar, which triggers a sync.

syncs.status

The engine.status, payload.status.sync, and payload.status.service properties are sync error codes, which are listed in services/sync/modules/constants.js, and success values are not reported.

syncs.failureReason

Stores error information, if any is present. Always contains the “name” property, which identifies the type of error it is. The types can be.

  • httperror: Indicates that we received an HTTP error response code, but are unable to be more specific about the error. Contains the following properties:

    • code: Integer HTTP status code.

  • nserror: Indicates that an exception with the provided error code caused sync to fail.

    • code: The nsresult error code (integer).

  • shutdownerror: Indicates that the sync failed because we shut down before completion.

  • autherror: Indicates an unrecoverable authentication error.

    • from: Where the authentication error occurred, one of the following values: tokenserver, fxaccounts, or hawkclient.

  • othererror: Indicates that it is a sync error code that we are unable to give more specific information on. As with the syncStatus property, it is a sync error code, which are listed in services/sync/modules/constants.js.

    • error: String identifying which error was present.

  • unexpectederror: Indicates that some other error caused sync to fail, typically an uncaught exception.

    • error: The message provided by the error.

  • sqlerror: Indicates that we received a mozIStorageError from a database query.

    • code: Value of the error.result property, one of the constants listed here.

syncs.engine.name

Third-party engines are not reported, so only the following values are allowed: addons, bookmarks, clients, forms, history, passwords, prefs, and tabs.

syncs.engine.validation.problems

For engines that can run validation on themselves, an array of objects describing validation errors that have occurred. Items that would have a count of 0 are excluded. Each engine will have its own set of items that it might put in the name field, but there are a finite number. See BookmarkProblemData.getSummary in services/sync/modules/bookmark_validator.js for an example.

syncs.devices

The list of remote devices associated with this account, as reported by the clients collection. The ID of each device is hashed using the same algorithm as the local id.

Events in the “sync” ping

The sync ping includes events in the same format as they are included in the main ping, see Events.

All events submitted as part of the sync ping which already include the “extra” object (the 6th parameter of the event array described in the event telemetry documentation) may also include a “serverTime” parameter, which the most recent unix timestamp sent from the sync server (as a string). This arrives in the X-Weave-Timestamp HTTP header, and may be omitted in cases where the client has not yet made a request to the server, or doesn’t have it for any other reason. It is included to improve flow analysis across multiple clients.

Every event recorded in this ping will have a category of sync. The following events are defined, categorized by the event method.

Histograms in the “sync” ping

The sync ping includes histograms relating to measurements of password manager usage. These histograms are duplicated in the main ping. Histograms are only included in a ping if they have been set by the pwmgr code. Currently, the histograms that can be included are:

PWMGR_BLOCKLIST_NUM_SITES PWMGR_FORM_AUTOFILL_RESULT PWMGR_LOGIN_LAST_USED_DAYS PWMGR_LOGIN_PAGE_SAFETY PWMGR_NUM_PASSWORDS_PER_HOSTNAME PWMGR_NUM_SAVED_PASSWORDS PWMGR_PROMPT_REMEMBER_ACTION PWMGR_PROMPT_UPDATE_ACTION PWMGR_SAVING_ENABLED

Histograms are objects with the following 6 properties: - min - minimum bucket size - max - maximum bucket size - histogram_type - counts - array representing contents of the buckets in the histogram - ranges - array with calculated bucket sizes

sendcommand

Records that Sync wrote a remote “command” to another client. These commands cause that other client to take some action, such as resetting Sync on that client, or opening a new URL.

  • object: The specific command being written.

  • value: Not used (ie, null)

  • extra: An object with the following attributes:

    • deviceID: A GUID which identifies the device the command is being sent to.

    • flowID: A GUID which uniquely identifies this command invocation. This GUID

      is the same for every device the tab is sent to.

    • streamID: A GUID which uniquely identifies this command invocation’s

      specific target. This GUID is unique for every device the tab is sent to (new in Firefox 79).

    • serverTime: (optional) Most recent server timestamp, as described above.

processcommand

Records that Sync processed a remote “command” previously sent by another client. This is logically the “other end” of sendcommand.

  • object: The specific command being processed.

  • value: Not used (ie, null)

  • extra: An object with the following attributes:

    • flowID: A GUID which uniquely identifies this command invocation. The value

      for this GUID will be the same as the flowID sent to the client via sendcommand.

    • streamID: A GUID which uniquely identifies this command invocation’s

      specific target. The value for this GUID will be the same as the streamID sent to the client via sendcommand (new in Firefox 79).

    • reason: A string value of either "poll", "push", or "push-missed"

      representing an explanation for why the command is being processed.

    • serverTime: (optional) Most recent server timestamp, as described above.

The migrations Array

The application-services developers are in the process of oxidizing parts of firefox sync and the related data storage code, which typically requires migrating the old storage into a new database and/or format.

When a migration like this occurs, a record is reported in this list the next time the sync ping is submitted.

Because the format of each data store may be drastically different, we are not attempting to come up with a generic representation here, and currently planning on allowing each migration record to vary independently (at least for now). These records will be distinctly identified by their "type" field.

They should only appear once per migration (that is, we’d rather fail to report a record than report them multiple times).

migrations.type: "webext-storage"

This indicates a migration was performed from the legacy kinto-based extension-storage database into the new webext-storage rust implementation.

It contains the following fields:

  • type: Always the string "webext-storage".

  • entries: The number of entries/preferences in the source (legacy) database, including ones we failed to read. See below for information on the distinction between entries and extensions in this record.

  • entriesSuccessful: The number of entries/preferences (see below) which we have successfully migrated into the destination database..

  • extensions: The number of distinct extensions which have at least one preference in the source (legacy) database.

  • extensionsSuccessful: The number of distinct extensions which have at least one preference in the destination (migrated) database.

  • openFailure: A boolean flag that is true if we hit a read error prior to . This likely indicates complete corruption, or a bug in an underlying library like rusqlite.

Note: “entries” vs “extensions”

The webext-storage migration record detailed above contains counts for both:

  • The number of “entries” detected vs successfully migrated.

  • The number of “extensions” detected vs successfully migrated.

This may seem redundant, but these refer to different (but related) things. The distinction here has to do with the way the two databases store extension-storage data:

  • The legacy database stores one row for each (extension_id, preference_name, preference_value) triple. These are referred to as entries.

  • Conversely, the new database stores one row per extension, which is a pair containing both the extension_id, as well as a dictionary holding all preference data, and so are equivalent to extensions.

(The description above is a somewhat simplified view of things, as it ignores a number values each database stores which is irrelevant for migration)

That is, entries represent each individual preference setting, and extensions represent the collected set of preferences for a given extension.

Counts for are both of these are present as it’s likely that the disparity would point to different kinds of issues with the migration code.