Runtime Metric Definition Subsystem: JOG

I’m Sorry

Why is it called JOG? Because it’s concerned with… run… time.

The normal mechanism for registering metrics, for reasons as varied from ease-of-impl to performance, happens at compile time. However, this doesn’t support use cases like

  • Artifact Builds (Where only the JavaScript of Firefox Desktop is repackaged at build time, so there is no compile environment)

  • Dynamic Telemetry (A theorized system for instrumenting Firefox Desktop without shipping code)

  • Web Extensions (Or at least the kind that can’t or won’t use the Glean JS SDK)

Thus we need a subsystem that supports the runtime registration of metrics. We call it JOG and it was implemented in bug 1698184.

JavaScript Only

Metrics in C++ and Rust are identified by identifiers which we can’t swap out at runtime. Thus, in order for changes to metrics to be visible to instrumented systems in C++ or Rust, you must compile.

JavaScript, on the other hand, we supply instances to on-demand. It not only supports the specific use cases driving this project, it’s the only environment that can benefit from runtime metric definition in Firefox Desktop.

Design

The original design was done as part of bug 1662863. Things have mostly just been refined from there.

Architecture

We silo as much of the subsystem as we can into the jog module located in toolkit/components/glean/bindings/jog/. This includes the metrics construction factory and storage for metrics instances and their names and ids.

Unfortunately, so that the metrics instances can be accessed by FFI, the Rust metrics instances created by the jog crate are stored within the fog crate.

Speaking of FFI, the jog crate is using cbindgen to be accessible to C++.

If necessary or pleasant, it is probably possible to do away with the C++ storage, moving the category set and metrics id map to Rust and moving information over FFI as needed.

Test methods are run from nsIFOG (so we can use them in JS in xpcshell) to static JOG:: functions.

Build Integration

If JOG detects we’re an artifact build (by checking MOZ_ARTIFACT_BUILDS), it generates jogfile.json and ensures it is placed in GreD (next to the firefox binary).

jogfile.json includes only the metric and ping information necessary to register them at runtime. (It doesn’t know about tags or descriptions, just the shapes and names of things)

This file is read the first time JS tries to get a metric category from the Glean global or a ping from the GleanPings global.

Yes, this is on the main thread. Yes, this is synchronous. Yes, this is file I/O.

Since this is a developer build, we think this is worth it to support Artifact Builds.

If we’re wrong about this and there are additional conditions we should place JOG under, please contact us.

If things get weird, delete objdir/dist/bin/jogfile.json

Sometimes, metrics or pings you’ve added may not appear when you run Firefox. For these and other odd cases, the solution is the same: delete jogfile.json from the dist/bin directory of your objdir, then try again.

This shouldn’t happen if you keep your artifact and non-artifact objdirs segregated (as is good practice).

If, despite doing things properly you still see this or something else odd, then that’s a bug. Please file it in Toolkit :: Telemetry