Activity Stream Pings

The Activity Stream system add-on sends various types of pings to the backend (HTTPS POST) Onyx server :

• a health ping that reports whether or not a user has a custom about:home or about:newtab page

• a session ping that describes the ending of an Activity Stream session (a new tab is closed or refreshed), and

• an event ping that records specific data about individual user interactions while interacting with Activity Stream

• an impression_stats ping that records data about Pocket impressions and user interactions

Schema definitions/validations that can be used for tests can be found in system-addon/test/schemas/pings.js.

Example Activity Stream health log

{
  "addon_version": "20180710100040",
  "client_id": "374dc4d8-0cb2-4ac5-a3cf-c5a9bc3c602e",
  "locale": "en-US",
  "version": "62.0a1",
  "release_channel": "nightly",
  "event": "AS_ENABLED",
  "value": 10,

  // These fields are generated on the server
  "date": "2016-03-07",
  "ip": "10.192.171.13",
  "ua": "python-requests/2.9.1",
  "receive_at": 1457396660000
}

Example Activity Stream session Log

{
  // These fields are sent from the client
  "action": "activity_stream_session",
  "addon_version": "20180710100040",
  "client_id": "374dc4d8-0cb2-4ac5-a3cf-c5a9bc3c602e",
  "locale": "en-US",
  "page": ["about:newtab" | "about:home" | "about:welcome" | "unknown"],
  "session_duration": 1635,
  "session_id": "{12dasd-213asda-213dkakj}",
  "profile_creation_date": 14786,
  "user_prefs": 7

  // These fields are generated on the server
  "date": "2016-03-07",
  "ip": "10.192.171.13",
  "ua": "python-requests/2.9.1",
  "receive_at": 1457396660000
}

Example Activity Stream user_event Log

{
  "action": "activity_stream_user_event",
  "action_position": "3",
  "addon_version": "20180710100040",
  "client_id": "374dc4d8-0cb2-4ac5-a3cf-c5a9bc3c602e",
  "event": "click or scroll or search or delete",
  "locale": "en-US",
  "page": ["about:newtab" | "about:home" | "about:welcome" | "unknown"],
  "source": "top sites, or bookmarks, or...",
  "session_id": "{12dasd-213asda-213dkakj}",
  "recommender_type": "pocket-trending",
  "metadata_source": "MetadataService or Local or TippyTopProvider",
  "user_prefs": 7

  // These fields are generated on the server
  "ip": "10.192.171.13",
  "ua": "python-requests/2.9.1",
  "receive_at": 1457396660000,
  "date": "2016-03-07",
}

Example Activity Stream impression_stats Logs

{
  "impression_id": "{005deed0-e3e4-4c02-a041-17405fd703f6}",
  "addon_version": "20180710100040",
  "locale": "en-US",
  "source": "pocket",
  "page": ["about:newtab" | "about:home" | "about:welcome" | "unknown"]
  "tiles": [{"id": 10000}, {"id": 10001}, {"id": 10002}]
  "user_prefs": 7
}

{
  "impression_id": "{005deed0-e3e4-4c02-a041-17405fd703f6}",
  "addon_version": "20180710100040",
  "locale": "en-US",
  "source": "pocket",
  "page": "unknown",
  "user_prefs": 7,

  // "pos" is the 0-based index to record the tile's position in the Pocket section.
  // "shim" is a base64 encoded shim attached to spocs, unique to the impression from the Ad server.
  "tiles": [{"id": 10000, "pos": 0, "shim": "enuYa1j73z3RzxgTexHNxYPC/b,9JT6w5KB0CRKYEU+"}],

  // A 0-based index to record which tile in the "tiles" list that the user just interacted with.
  "click|block|pocket": 0
}

Example Activity Stream Router Pings

{
  "client_id": "n/a",
  "action": ["snippets_user_event" | "onboarding_user_event"],
  "impression_id": "{005deed0-e3e4-4c02-a041-17405fd703f6}",
  "source": "pocket",
  "addon_version": "20180710100040",
  "locale": "en-US",
  "source": "NEWTAB_FOOTER_BAR",
  "message_id": "some_snippet_id",
  "event": "IMPRESSION",
  "event_context": "{\"view\":\"application_menu\"}"
}

Where:

• :one: Firefox data

• :two: HTTP protocol data

• :three: server augmented data

• :four: User preferences encoding table

Note: the following session-related fields are not yet implemented in the system-addon, but will likely be added in future versions:

{
  "total_bookmarks": 19,
  "total_history_size": 9,
  "highlights_size": 20
}

Encoding and decoding of user_prefs

This encoding mapping was defined in system-addon/lib/TelemetryFeed.jsm

Each item above could be combined with other items through bitwise OR (|) operation.

Examples:

• Everything is on, user_prefs = 1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 | 256 = 511

• Everything is off, user_prefs = 0

• Only show search and Top Stories, user_prefs = 1 | 4 = 5

• Everything except Highlights, user_prefs = 1 | 2 | 4 | 16 | 32 | 64 | 128 | 256 = 503

Likewise, one can use bitwise AND (&) for decoding.

• Check if everything is shown, user_prefs & (1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 | 256) or user_prefs == 511

• Check if everything is off, user_prefs == 0

• Check if search is shown, user_prefs & 1

• Check if both Top Sites and Top Stories are shown, (user_prefs & 2) && (user_prefs & 4), or (user_prefs & (2 | 4)) == (2 | 4)