Task Kinds

This section lists and documents the available task kinds.

build

Builds are tasks that produce an installer or other output that can be run by users or automated tests. This is more restrictive than most definitions of “build” in a Mozilla context: it does not include tasks that run build-like actions for static analysis or to produce instrumented artifacts.

build-signing

Many builds must be signed. The build-signing task takes the unsigned build kind artifacts and passes them through signingscriptworker to a signing server and returns signed results.

artifact-build

This kind performs an artifact build: one based on precompiled binaries discovered via the TaskCluster index. This task verifies that such builds continue to work correctly.

hazard

Hazard builds are similar to “regular’ builds, but use a compiler extension to extract a bunch of data from the build and then analyze that data looking for hazardous behaviors.

l10n

The l10n kind takes the last published nightly build, and generates localized builds from it. You can read more about how to trigger these on the wiki.

nightly-l10n

The nightly l10n kind repacks a specific nightly build (from the same source code) in order to provide localized versions of the same source.

nightly-l10n-signing

The nightly l10n signing kind takes artifacts from the nightly-l10n kind and passes them to signing servers to have their contents signed appropriately, based on an appropriate signing format. One signing job is created for each nightly-l10n job (usually chunked).

source-test

Source-tests are tasks that run directly from the Gecko source. This can include linting, unit tests, source-code analysis, or measurement work. While source-test tasks run from a source checkout, it is still possible for them to depend on a build artifact, though often they do not.

upload-symbols

Upload-symbols tasks run after builds and upload the symbols files generated by build tasks to Socorro for later use in crash analysis.

upload-generated-sources

Upload-generated-sources tasks run after builds and upload source files that were generated as part of the build process to an s3 bucket for later use in links from crash reports or when debugging shipped builds.

valgrind

Valgrind tasks produce builds instrumented by valgrind.

searchfox

Searchfox builds generate C++ index data for Searchfox.

static-analysis

Static analysis builds use the compiler to perform some detailed analysis of the source code while building. The useful output from these tasks are their build logs, and while they produce a binary, they do not upload it as an artifact.

toolchain

Toolchain builds create the compiler toolchains used to build Firefox. These will eventually be dependencies of the builds themselves, but for the moment are run manually via try pushes and the results uploaded to tooltool.

spidermonkey

Spidermonkey tasks check out the full gecko source tree, then compile only the spidermonkey portion. Each task runs specific tests after the build.

test

The desktop-test kind defines tests for builds. Its tests.yml defines the full suite of desktop tests and their particulars, leaving it to the transforms to determine how those particulars apply to the various platforms.

The process of generating tests goes like this, based on a set of YAML files named in kind.yml:

  • For each build task, determine the related test platforms based on the build platform. For example, a Windows 2010 build might be tested on Windows 7 and Windows 10. Each test platform specifies “test sets” indicating which tests to run. This is configured in the file named test-platforms.yml.
  • Each test set is expanded to a list of tests to run. This is configured in the file named by test-sets.yml. A platform may specify several test sets, in which case the union of those sets is used.
  • Each named test is looked up in the file named by tests.yml to find a test description. This test description indicates what the test does, how it is reported to treeherder, and how to perform the test, all in a platform-independent fashion.
  • Each test description is converted into one or more tasks. This is performed by a sequence of transforms defined in the transforms key in kind.yml. See Transforms: for more information on these transforms.
  • The resulting tasks become a part of the task graph.

Important

This process generates all test jobs, regardless of tree or try syntax. It is up to a later stages of the task-graph generation (the target set and optimization) to select the tests that will actually be performed.

docker-image

Tasks of the docker-image kind build the Docker images in which other Docker tasks run.

The tasks to generate each docker image have predictable labels: build-docker-image-<name>.

Docker images are built from subdirectories of taskcluster/docker, using docker build. There is currently no capability for one Docker image to depend on another in-tree docker image, without uploading the latter to a Docker repository

The task definition used to create the image-building tasks is given in image.yml in the kind directory, and is interpreted as a YAML Template.

balrog

Balrog is the Mozilla Update Server. Jobs of this kind are submitting information which assists in telling Firefox that an update is available for the related job.

balrog-l10n

Balrog is the Mozilla Update Server. Jobs of this kind are submitting information which assists in telling Firefox that an update is available for the localized job involved.

beetmover

Beetmover, takes specific artifacts, “Beets”, and pushes them to a location outside of Taskcluster’s task artifacts, (archive.mozilla.org as one place) and in the process determines the final location and a “pretty” name (versioned product name)

beetmover-l10n

Beetmover L10n, takes specific artifacts, “Beets”, and pushes them to a location outside of Taskcluster’s task artifacts, (archive.mozilla.org as one place) and in the process determines the final location and a “pretty” name (versioned product name) This separate kind uses logic specific to localized artifacts, such as including the language in the final artifact names.

beetmover-repackage

Beetmover-repackage is beetmover but for tasks that need an intermediate step between signing and packaging, such as OSX. For more details see the definitions of the Beetmover kind above and the repackage kind below.

beetmover-cdns

Beetmover-cdns publishes promoted releases to CDNs. This is part of release promotion.

beetmover-source

Beetmover-source publishes release source. This is part of release promotion.

checksums-signing

Checksums-signing take as input the checksums file generated by beetmover tasks and sign it via the signing scriptworkers. Returns the same file signed and additional detached signature.

beetmover-checksums

Beetmover, takes specific artifact checksums and pushes it to a location outside of Taskcluster’s task artifacts (archive.mozilla.org as one place) and in the process determines the final location and “pretty” names it (version product name)

google-play-strings

Download strings to display on Google Play from https://l10n.mozilla-community.org/stores_l10n/. Artifact is then used by push-apk.

push-apk-breakpoint

Decides whether or not APKs should be published onto Google Play Store. Jobs of this kind depend on all the signed multi-locales (aka “multi”) APKs for a given release, in order to make the decision.

push-apk

PushApk publishes Android packages onto Google Play Store. Jobs of this kind take all the signed multi-locales (aka “multi”) APKs for a given release and upload them all at once. They also depend on the breakpoint.

release-balrog-publishing

Schedule a release to go live in Balrog.

release-binary-transparency

Binary transparency creates a publicly verifiable log of binary shas for downstream release auditing. https://wiki.mozilla.org/Security/Binary_Transparency

release-snap

Generate an installer using Ubuntu’s Snap format.

release-notify-push

Notify when a release has been pushed to CDNs.

release-notify-ship

Notify when a release has been shipped.

release-secondary-notify-ship

Notify when an RC release has been shipped to the beta channel.

release-notify-promote

Notify when a release has been promoted.

release-bouncer-sub

Submits bouncer updates for releases.

release-mark-as-shipped

Marks releases as shipped in Ship-It.

release-bouncer-aliases

Update Bouncer’s (download.mozilla.org) “latest” aliases.

release-bouncer-check

Checks Bouncer (download.mozilla.org) uptake.

release-generate-checksums

Generate the per-release checksums along with the summaries and upload it to S3.

release-final-verify

Verifies the contents and package of release update MARs.

release-secondary-final-verify

Verifies the contents and package of release update MARs for RC releases.

release-secondary-balrog-publishing

Schedule an RC release to go live in Balrog. Usually this will happen on the beta channel, to a smaller audience, before the RC goes live on the release channel.

release-update-verify

Verifies the contents and package of release update MARs.

release-secondary-update-verify

Verifies the contents and package of release update MARs.

release-updates-builder

Top level Balrog blob submission & patcher/update verify config updates.

release-version-bump

Bumps to the next version.

release-source

Generates source for the release

release-source-signing

Signs source for the release

release-partner-repack

Generates customized versions of releases for partners.

repackage

Repackage tasks take a signed output and package them up into something suitable for shipping to our users. For example, on OSX we return a tarball as the signed output and this task would package that up as an Apple Disk Image (.dmg)

repackage-l10n

Repackage-L10n is a `Repackage` task split up to be suitable for use after l10n repacks.

repackage-signing

Repackage-signing take the repackaged installers (windows) and update packaging (with the signed internal bits) and signs them.

repo-update

Repo-Update tasks are tasks that perform some action on the project repo itself, in order to update its state in some way.

partials

Partials takes the complete.mar files produced in previous tasks and generates partial updates between previous nightly releases and the new one. Requires a release_history in the parameters. See mach release-history if doing this manually.

partials-signing

Partials-signing takes the partial updates produced in Partials and signs them.

post-balrog-dummy

Dummy tasks to consolidate balrog dependencies to avoid taskcluster limits on number of dependencies per task.

post-beetmover-dummy

Dummy tasks to consolidate beetmover dependencies to avoid taskcluster limits on number of dependencies per task.

post-beetmover-checksums-dummy

Dummy tasks to consolidate beetmover-checksums dependencies to avoid taskcluster limits on number of dependencies per task.

packages

Tasks used to build packages for use in docker images.

diffoscope

Tasks used to compare pairs of Firefox builds using https://diffoscope.org/. As of writing, this is mainly meant to be used in try builds, by editing taskcluster/ci/diffoscope/kind.yml for your needs.