Task Generation

Many selectors (including chooser, coverage and fuzzy) source their available tasks directly from the taskgraph module by building the taskgraph locally. This means that the list of available tasks will never be stale. While this is very powerful, it comes with a large enough performance cost to get annoying (around twenty seconds).

The result of the taskgraph generation will be cached, so this penalty will only be incurred whenever a file in the /taskcluster directory is modified. Unfortunately this directory changes pretty frequently, so developers can expect to rebuild the cache each time they pull in mozilla-central. Developers who regularly work on /taskcluster can expect to rebuild even more frequently.

Configuring Watchman

It’s possible to bypass this penalty completely by using the file watching service watchman. If you use the fsmonitor mercurial extension, you already have watchman installed.

Note

If you aren’t using fsmonitor but end up installng watchman anyway, you might as well enable it for a faster Mercurial experience.

Otherwise, install watchman. If using Linux you’ll likely run into the inotify limits outlined on that page due to the size of mozilla-central. You can read this page for more information on how to bump the limits permanently.

Next run the following commands:

$ cd path/to/mozilla-central
$ watchman watch .
$ watchman -j < tools/tryselect/watchman.json

You should see output like:

{
    "triggerid": "rebuild-taskgraph-cache",
    "disposition": "created",
    "version": "20200920.192359.0"
}

That’s it. Now anytime a file under /taskcluster is modified (either by your editor, or by updating version control), the taskgraph cache will be rebuilt in the background, allowing you to skip the wait the next time you run mach try.

Note

Watchman triggers are persistent and don’t need to be added more than once. See Managing Triggers for how to remove a trigger.

You can test that everything is working by running these commands:

$ statedir=`mach python -c "from mach.util import get_state_dir; print(get_state_dir(specific_to_topsrcdir=True))"`
$ rm -rf $statedir/cache/taskgraph
$ touch taskcluster/mach_commands.py
# wait a minute for generation to trigger and finish
$ ls $statedir/cache/taskgraph

If the target_task_set file exists, you are good to go. If not you can look at the watchman log to see if there were any errors. This typically lives somewhere like /usr/local/var/run/watchman/$USER-state/log. In this case please file a bug under Firefox Build System :: Try and include the relevant portion of the log.

Running Watchman on Startup

Watchman is both a client and a service all in one. When running a watchman command, the client binary will start the service in the background if it isn’t running. This means on reboot the service won’t be running and you’ll need to start the service each time by invoking the client binary (e.g by running watchman version).

If you’d like this to happen automatically, you can use your favourite platform specific way of running commands at startup (crontab, rc.local, etc.). Watchman stores separate state for each user, so be sure you run the command as the user that set up the triggers.

Setting up a systemd Service

If systemd is an option you can create a service:

[Unit]
Description=Watchman for %i
After=network.target

[Service]
Type=simple
User=%i
ExecStart=/usr/local/bin/watchman --log-level 1 watch-list -f
ExecStop=/usr/local/bin/watchman shutdown-server

[Install]
WantedBy=multi-user.target

Save this to a file called /etc/systemd/system/watchman@.service. Then run:

$ sudo systemctl enable watchman@$USER.service
$ sudo systemctl start watchman@$USER.service

The next time you reboot, the watchman service should start automatically.

Managing Triggers

When adding a trigger watchman writes it to disk. Typically it’ll be a path similar to /usr/local/var/run/watchman/$USER-state/state. While editing that file by hand would work, the watchman binary provides an interface for managing your triggers.

To see all directories you are currently watching:

$ watchman watch-list

To view triggers that are active in a specified watch:

$ watchman trigger-list <path>

To delete a trigger from a specified watch:

$ watchman trigger-del <path> <name>

In the above two examples, replace <path> with the path of the watch, presumably mozilla-central. Using . works as well if that is already your working directory. For more information on managing triggers and a reference of other commands, see the official docs.