mozbuild package



mozbuild.android_version_code module

mozbuild.android_version_code.android_version_code(buildid, *args, **kwargs)
mozbuild.android_version_code.android_version_code_v0(buildid, cpu_arch=None, min_sdk=0, max_sdk=0)
mozbuild.android_version_code.android_version_code_v1(buildid, cpu_arch=None, min_sdk=0, max_sdk=0)

Generate a v1 android:versionCode. The important consideration is that version codes be monotonically increasing (per Android package name) for all published builds. The input build IDs are based on timestamps and hence are always monotonically increasing.

The generated v1 version codes look like (in binary):

0111 1000 0010 tttt tttt tttt tttt txpg

The 17 bits labelled ‘t’ represent the number of hours since midnight on September 1, 2015. (2015090100 in YYYYMMMDDHH format.) This yields a little under 15 years worth of hourly build identifiers, since 2**17 / (366 * 24) =~ 14.92.

The bits labelled ‘x’, ‘p’, and ‘g’ are feature flags.

The bit labelled ‘x’ is 1 if the build is for an x86 or x86-64 architecture, and 0 otherwise, which means the build is for an ARM or ARM64 architecture. (Fennec no longer supports ARMv6, so ARM is equivalent to ARMv7.

ARM64 is also known as AArch64; it is logically ARMv8.)

For the same release, x86 and x86_64 builds have higher version codes and take precedence over ARM builds, so that they are preferred over ARM on devices that have ARM emulation.

The bit labelled ‘p’ is 1 if the build is for a 64-bit architecture (x86-64 or ARM64), and 0 otherwise, which means the build is for a 32-bit architecture (x86 or ARM). 64-bit builds have higher version codes so they take precedence over 32-bit builds on devices that support 64-bit.

The bit labelled ‘g’ is 1 if the build targets a recent API level, which is currently always the case, because Firefox no longer ships releases that are split by API levels. However, we may reintroduce a split in the future, in which case the release that targets an older API level will

We throw an explanatory exception when we are within one calendar year of running out of build events. This gives lots of time to update the version scheme. The responsible individual should then bump the range (to allow builds to continue) and use the time remaining to update the version scheme via the reserved high order bits.

N.B.: the reserved 0 bit to the left of the highest order ‘t’ bit can, sometimes, be used to bump the version scheme. In addition, by reducing the granularity of the build identifiers (for example, moving to identifying builds every 2 or 4 hours), the version scheme may be adjusted further still without losing a (valuable) high order bit.


mozbuild.artifact_builds module

mozbuild.artifact_cache module

Fetch and cache artifacts from URLs.

This module manages fetching artifacts from URLS and purging old artifacts using a simple Least Recently Used cache.

This module requires certain modules be importable from the ambient Python environment. Consumers will need to arrange this themselves.

The bulk of the complexity is in managing and persisting several caches. If we found a Python LRU cache that pickled cleanly, we could remove a lot of this code! Sadly, I found no such candidate implementations, so we pickle pylru caches manually.

None of the instances (or the underlying caches) are safe for concurrent use. A future need, perhaps.

class mozbuild.artifact_cache.ArtifactCache(cache_dir, log=None, skip_cache=False)

Bases: object

Fetch artifacts from URLS and purge least recently used artifacts from disk.

fetch(url, force=False)
log(*args, **kwargs)
class mozbuild.artifact_cache.ArtifactPersistLimit(log=None)

Bases: dlmanager.persist_limit.PersistLimit

Handle persistence for a cache of artifacts.

When instantiating a DownloadManager, it starts by filling the PersistLimit instance it’s given with register_dir_content. In practice, this registers all the files already in the cache directory. After a download finishes, the newly downloaded file is registered, and the oldest files registered to the PersistLimit instance are removed depending on the size and file limits it’s configured for. This is all good, but there are a few tweaks we want here: - We have pickle files in the cache directory that we don’t want purged. - Files that were just downloaded in the same session shouldn’t be purged.

(if for some reason we end up downloading more than the default max size,

we don’t want the files to be purged)

To achieve this, this subclass of PersistLimit inhibits the register_file method for pickle files and tracks what files were downloaded in the same session to avoid removing them.

The register_file method may be used to register cache matches too, so that later sessions know they were freshly used.

log(*args, **kwargs)
register_dir_content(directory, pattern='*')

Register every files in a directory that match pattern.


register a single file.


remove oldest registered files.

mozbuild.artifact_commands module

class mozbuild.artifact_commands.ArtifactSubCommand(command, subcommand, description=None, parser=None)

Bases: mach.decorators.SubCommand

class mozbuild.artifact_commands.PackageFrontend(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Fetch and install binary artifacts from Mozilla automation.


Download, cache, and install pre-built binary artifacts to build Firefox.

Use |mach build| as normal to freshen your installed binary libraries: artifact builds automatically download, cache, and install binary artifacts from Mozilla automation, replacing whatever may be in your object directory. Use |mach artifact last| to see what binary artifacts were last used.

Never build libxul again!

artifact_clear_cache(tree=None, job=None, verbose=False)
artifact_install(source=None, skip_cache=False, tree=None, job=None, verbose=False, no_tests=False, symbols=False, host_bins=False, distdir=None, no_process=False, maven_zip=False)
artifact_toolchain(verbose=False, cache_dir=None, skip_cache=False, from_build=(), tooltool_manifest=None, authentication_file=None, no_unpack=False, retry=0, bootstrap=False, artifact_manifest=None, files=())

Download, cache and install pre-built toolchains.

class mozbuild.artifact_commands.SymbolsAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

mozbuild.artifacts module

Fetch build artifacts from a Firefox tree.

This provides an (at-the-moment special purpose) interface to download Android artifacts from Mozilla’s Task Cluster.

This module performs the following steps:

  • find a candidate hg parent revision. At one time we used the local pushlog, which required the mozext hg extension. This isn’t feasible with git, and it is only mildly less efficient to not use the pushlog, so we don’t use it even when querying hg.

  • map the candidate parent to candidate Task Cluster tasks and artifact locations. Pushlog entries might not correspond to tasks (yet), and those tasks might not produce the desired class of artifacts.

  • fetch fresh Task Cluster artifacts and purge old artifacts, using a simple Least Recently Used cache.

  • post-process fresh artifacts, to speed future installation. In particular, extract relevant files from Mac OS X DMG files into a friendly archive format so we don’t have to mount DMG files frequently.

This module requires certain modules be importable from the ambient Python environment. |mach artifact| ensures these modules are available, but other consumers will need to arrange this themselves.

class mozbuild.artifacts.AndroidArtifactJob(log=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: mozbuild.artifacts.ArtifactJob

package_artifact_patterns = {'**/*.so'}
package_re = 'public/build/geckoview_example\\.apk'
process_package_artifact(filename, processed_filename)
process_symbols_archive(filename, processed_filename)
product = 'mobile'
class mozbuild.artifacts.ArtifactJob(log=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: object

candidate_trees = ['mozilla-central', 'integration/autoland', 'releases/mozilla-beta', 'releases/mozilla-release']
log(*args, **kwargs)
process_artifact(filename, processed_filename)
process_host_bin(filename, processed_filename)
process_package_artifact(filename, processed_filename)
process_symbols_archive(filename, processed_filename, skip_compressed=False)
process_tests_tar_artifact(filename, processed_filename)
process_tests_zip_artifact(filename, processed_filename)
test_artifact_patterns = {('bin/BadCertAndPinningServer', ('bin', 'bin')), ('bin/DelegatedCredentialsServer', ('bin', 'bin')), ('bin/GenerateOCSPResponse', ('bin', 'bin')), ('bin/OCSPStaplingServer', ('bin', 'bin')), ('bin/SanctionsTestServer', ('bin', 'bin')), ('bin/certutil', ('bin', 'bin')), ('bin/components/*.xpt', ('bin/components', 'bin/components')), ('bin/geckodriver', ('bin', 'bin')), ('bin/http3server', ('bin', 'bin')), ('bin/pk12util', ('bin', 'bin')), ('bin/plugins/*', ('bin/plugins', 'plugins')), ('bin/plugins/gmp-*/*/*', ('bin/plugins', 'bin')), ('bin/screentopng', ('bin', 'bin')), ('bin/ssltunnel', ('bin', 'bin')), ('bin/xpcshell', ('bin', 'bin'))}
trust_domain = 'gecko'
try_tree = 'try'
class mozbuild.artifacts.Artifacts(tree, substs, defines, job=None, log=None, cache_dir='.', hg=None, git=None, skip_cache=False, topsrcdir=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, no_process=False, mozbuild=None)

Bases: object

Maintain state to efficiently fetch build artifacts from a Firefox tree.

find_pushhead_artifacts(task_cache, job, tree, pushhead)
install_from(source, distdir)

Install artifacts from a source into the given distdir.

install_from_file(filename, distdir)
install_from_revset(revset, distdir)
install_from_task(taskId, distdir)
install_from_url(url, distdir)
log(*args, **kwargs)
run_hg(*args, **kwargs)
class mozbuild.artifacts.CacheManager(cache_dir, cache_name, cache_size, cache_callback=None, log=None, skip_cache=False)

Bases: object

Maintain an LRU cache. Provide simple persistence, including support for loading and saving the state using a “with” block. Allow clearing the cache and printing the cache for debugging.

Provide simple logging.

log(*args, **kwargs)
class mozbuild.artifacts.LinuxArtifactJob(log=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: mozbuild.artifacts.ArtifactJob

property package_artifact_patterns
package_re = 'public/build/target\\.tar\\.bz2'
process_package_artifact(filename, processed_filename)
product = 'firefox'
class mozbuild.artifacts.LinuxThunderbirdArtifactJob(log=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: mozbuild.artifacts.ThunderbirdMixin, mozbuild.artifacts.LinuxArtifactJob

class mozbuild.artifacts.MacArtifactJob(log=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: mozbuild.artifacts.ArtifactJob

package_re = 'public/build/target\\.dmg'
property paths_no_keep_path
process_package_artifact(filename, processed_filename)
product = 'firefox'
class mozbuild.artifacts.MacThunderbirdArtifactJob(log=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: mozbuild.artifacts.ThunderbirdMixin, mozbuild.artifacts.MacArtifactJob

class mozbuild.artifacts.PushheadCache(cache_dir, log=None, skip_cache=False)

Bases: mozbuild.artifacts.CacheManager

Helps map tree/revision pairs to parent pushheads according to the pushlog.

parent_pushhead_id(tree, revision)
pushid_range(tree, start, end)
class mozbuild.artifacts.TaskCache(cache_dir, log=None, skip_cache=False)

Bases: mozbuild.artifacts.CacheManager

Map candidate pushheads to Task Cluster task IDs and artifact URLs.

artifacts(tree, job, artifact_job_class, rev)
class mozbuild.artifacts.ThunderbirdMixin

Bases: object

candidate_trees = ['comm-central']
product = 'thunderbird'
trust_domain = 'comm'
try_tree = 'try-comm-central'
class mozbuild.artifacts.WinArtifactJob(log=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: mozbuild.artifacts.ArtifactJob

property package_artifact_patterns
package_re = 'public/build/target\\.(zip|tar\\.gz)'
process_package_artifact(filename, processed_filename)
product = 'firefox'
test_artifact_patterns = {('bin/BadCertAndPinningServer.exe', ('bin', 'bin')), ('bin/DelegatedCredentialsServer.exe', ('bin', 'bin')), ('bin/GenerateOCSPResponse.exe', ('bin', 'bin')), ('bin/OCSPStaplingServer.exe', ('bin', 'bin')), ('bin/SanctionsTestServer.exe', ('bin', 'bin')), ('bin/certutil.exe', ('bin', 'bin')), ('bin/components/*', ('bin/components', 'bin/components')), ('bin/geckodriver.exe', ('bin', 'bin')), ('bin/http3server.exe', ('bin', 'bin')), ('bin/minidumpwriter.exe', ('bin', 'bin')), ('bin/pk12util.exe', ('bin', 'bin')), ('bin/plugins/*', ('bin/plugins', 'plugins')), ('bin/plugins/gmp-*/*/*', ('bin/plugins', 'bin')), ('bin/screenshot.exe', ('bin', 'bin')), ('bin/ssltunnel.exe', ('bin', 'bin')), ('bin/xpcshell.exe', ('bin', 'bin'))}
class mozbuild.artifacts.WinThunderbirdArtifactJob(log=None, download_tests=True, download_symbols=False, download_host_bins=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: mozbuild.artifacts.ThunderbirdMixin, mozbuild.artifacts.WinArtifactJob


Decorator to wrap a class or instance method with a memoizing callable that saves results in a (possibly shared) cache.

mozbuild.artifacts.startswithwhich(s, prefixes)

mozbuild.base module

exception mozbuild.base.BadEnvironmentException

Bases: Exception

Base class for errors raised when the build environment is not sane.

exception mozbuild.base.BinaryNotFoundException(path)

Bases: Exception

Raised when the binary is not found in the expected location.

exception mozbuild.base.BuildEnvironmentNotFoundException

Bases: mozbuild.base.BadEnvironmentException, AttributeError

Raised when we could not find a build environment.

class mozbuild.base.ExecutionSummary(summary_format, **data)

Bases: dict

Helper for execution summaries.

extend(summary_format, **data)
class mozbuild.base.MachCommandBase(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MozbuildObject

Base class for mach command providers that wish to be MozbuildObjects.

This provides a level of indirection so MozbuildObject can be refactored without having to change everything that inherits from it.

class mozbuild.base.MachCommandConditions

Bases: object

A series of commonly used condition functions which can be applied to mach commands with providers deriving from MachCommandBase.

static has_build(cls)

Must have a build.

static has_build_or_shell(cls)

Must have a build or a shell build.

static is_android(cls)

Must have an Android build.

static is_artifact_build(cls)

Must be an artifact build.

static is_buildapp_in(cls, apps)

Must have a build for one of the given app

static is_firefox(cls)

Must have a Firefox build.

static is_firefox_or_android(cls)

Must have a Firefox or Android build.

static is_firefox_or_thunderbird(cls)

Must have a Firefox or Thunderbird build.

static is_git(cls)

Must have a git source checkout.

static is_hg(cls)

Must have a mercurial source checkout.

static is_jsshell(cls)

Must have a jsshell build.

static is_non_artifact_build(cls)

Must not be an artifact build.

static is_not_android(cls)

Must not have an Android build.

static is_thunderbird(cls)

Must have a Thunderbird build.

class mozbuild.base.MozbuildObject(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>, virtualenv_name=None)

Bases: mach.mixin.process.ProcessExecutionMixin

Base class providing basic functionality useful to many modules.

Modules in this package typically require common functionality such as accessing the current config, getting the location of the source directory, running processes, etc. This classes provides that functionality. Other modules can inherit from this class to obtain this functionality easily.

property bindir
build_out_of_date(output, dep_file)
property config_environment

Returns the ConfigEnvironment for the current build configuration.

This property is only available once configure has executed.

If configure’s output is not available, this will raise.

property defines
property distdir
classmethod from_environment(cwd=None, detect_virtualenv_mozinfo=True, **kwargs)

Create a MozbuildObject by detecting the proper one from the env.

This examines environment state like the current working directory and creates a MozbuildObject from the found source directory, mozconfig, etc.

The role of this function is to identify a topsrcdir, topobjdir, and mozconfig file.

If the current working directory is inside a known objdir, we always use the topsrcdir and mozconfig associated with that objdir.

If the current working directory is inside a known srcdir, we use that topsrcdir and look for mozconfigs using the default mechanism, which looks inside environment variables.

If the current Python interpreter is running from a virtualenv inside an objdir, we use that as our objdir.

If we’re not inside a srcdir or objdir, an exception is raised.

detect_virtualenv_mozinfo determines whether we should look for a mozinfo.json file relative to the virtualenv directory. This was added to facilitate testing. Callers likely shouldn’t change the default.

get_binary_path(what='app', validate_exists=True, where='default')

Obtain the path to a compiled binary for this build configuration.

The what argument is the program or tool being sought after. See the code implementation for supported values.

If validate_exists is True (the default), we will ensure the found path exists before returning, raising an exception if it doesn’t.

If where is ‘staged-package’, we will return the path to the binary in the package staging directory.

If no arguments are specified, we will return the main binary for the configured XUL application.

static get_mozconfig_and_target(topsrcdir, path, env_mozconfig)
property includedir
mozbuild_reader(config_mode='build', vcs_revision=None, vcs_check_clean=True)

Obtain a BuildReader for evaluating files.

Given arguments, returns a mozbuild.frontend.reader.BuildReader that can be used to evaluate files for this repo.

config_mode is either build or empty. If build, self.config_environment is used. This requires a configured build system to work. If empty, an empty config is used. empty is appropriate for file-based traversal mode where Files metadata is read.

If vcs_revision is defined, it specifies a version control revision to use to obtain files content. The default is to use the filesystem. This mode is only supported with Mercurial repositories.

If vcs_revision is not defined and the version control checkout is sparse, this implies vcs_revision='.'.

If vcs_revision is . (denotes the parent of the working directory), we will verify that the working directory is clean unless vcs_check_clean is False. This prevents confusion due to uncommitted file changes not being reflected in the reader.

property mozconfig

Returns information about the current mozconfig file.

This a dict as returned by MozconfigLoader.read_mozconfig()

property mozconfig_and_target

Show a desktop notification with the supplied message

On Linux and Mac, this will show a desktop notification with the message, but on Windows we can only flash the screen.

property platform

Returns current platform and architecture name


A specialized version of the memoize decorator that works for class instance properties.


Force config.status to be re-read and return the new value of self.config_environment.


A specialized version of the memoize decorator that works for class instance properties.

property statedir
property substs
property topobjdir
property virtualenv_manager
exception mozbuild.base.ObjdirMismatchException(objdir1, objdir2)

Bases: mozbuild.base.BadEnvironmentException

Raised when the current dir is an objdir and doesn’t match the mozconfig.

class mozbuild.base.PathArgument(arg, topsrcdir, topobjdir, cwd=None)

Bases: object

Parse a filesystem path argument and transform it in various ways.


Return a path relative to the topsrcdir or topobjdir.

If the argument is a path to a location in one of the base directories (topsrcdir or topobjdir), then strip off the base directory part and just return the path within the base directory.


Emit the parent directories of a path.

mozbuild.base.samepath(path1, path2)

mozbuild.build_commands module

class mozbuild.build_commands.Build(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Interface to build the tree.

build(what=None, jobs=0, directory=None, verbose=False, keep_going=False)

Build the source tree.

With no arguments, this will perform a full build.

Positional arguments define targets to build. These can be make targets or patterns like “<dir>/<target>” to indicate a make target within a directory.

There are a few special targets that can be used to perform a partial build faster than what mach build would perform:

  • binaries - compiles and links all C/C++ sources and produces shared libraries and executables (binaries).

  • faster - builds JavaScript, XUL, CSS, etc files.

“binaries” and “faster” almost fully complement each other. However, there are build actions not captured by either. If things don’t appear to be rebuilding, perform a vanilla mach build to rebuild the world.

build_backend(backend, diff=False, verbose=False, dry_run=False)
configure(options=None, buildstatus_messages=False, line_handler=None)
resource_usage(address=None, port=None, browser=None, url=None)

mozbuild.chunkify module

exception mozbuild.chunkify.ChunkingError

Bases: Exception

mozbuild.chunkify.chunkify(things, this_chunk, chunks)
mozbuild.chunkify.split_evenly(n, chunks)

Split an integer into evenly distributed list

>>> split_evenly(7, 3)
[3, 2, 2]
>>> split_evenly(12, 3)
[4, 4, 4]
>>> split_evenly(35, 10)
[4, 4, 4, 4, 4, 3, 3, 3, 3, 3]
>>> split_evenly(1, 2)
Traceback (most recent call last):
ChunkingError: Number of chunks is greater than number

mozbuild.config_status module

mozbuild.config_status.config_status(topobjdir='.', topsrcdir='.', defines=None, substs=None, source=None, mozconfig=None, args=['doc', '--upload', '--no-open', '--no-serve', '--write-url', '/builds/worker/firefox-source-docs-url.txt'])

Main function, providing config.status functionality.

Contrary to config.status, it doesn’t use CONFIG_FILES or CONFIG_HEADERS variables.

Without the -n option, this program acts as config.status and considers the current directory as the top object directory, even when config.status is in a different directory. It will, however, treat the directory containing config.status as the top object directory with the -n option.

The options to this function are passed when creating the ConfigEnvironment. These lists, as well as the actual wrapper script around this function, are meant to be generated by configure. See build/autoconf/config.status.m4. module

class, objdir, fix)

Bases: object

property cpu
property fs_lastaccess
property memory
property mozillabuild
property platform
prompt_bool(prompt, limit=5)

Prompts the user with prompt and requires a boolean value.

property storage_freespace

mozbuild.dotproperties module

class mozbuild.dotproperties.DotProperties(file=None)

Bases: object

A thin representation of a key=value .properties file.

get(key, default=None)
get_dict(prefix, required_keys=[])

Turns {‘foo.title’:’title’, …} into {‘title’:’title’, …}.

If |required_keys| is present, it must be an iterable of required key names. If a required key is not present, ValueError is thrown.

Returns {} to indicate an empty or missing dict.


Turns {‘list.0’:’foo’, ‘list.1’:’bar’} into [‘foo’, ‘bar’].

Returns [] to indicate an empty or missing list.


Updates properties from a file name or file-like object.

Ignores empty lines and comment lines.

mozbuild.export_telemetry_schema module


mozbuild.faster_daemon module

mozbuild.fork_interpose module

mozbuild.gen_test_backend module


mozbuild.generated_sources module

mozbuild.generated_sources.get_filename_with_digest(name, contents)

Return the filename that will be used to store the generated file in the S3 bucket, consisting of the SHA-512 digest of contents joined with the relative path name.


Yield tuples of (objdir-rel-path, file) for generated source files in this objdir, where file is either an absolute path to the file or a mozpack.File instance.


Return a tuple of (region, bucket) giving the AWS region and S3 bucket to which generated sources should be uploaded.


Generate the SHA-512 digest of data and return it as a hex string.

mozbuild.gn_processor module

class mozbuild.gn_processor.GnConfigGenBackend(environment)

Bases: mozbuild.backend.base.BuildBackend


Called when consume() has completed handling all objects.


Consumes an individual TreeMetadata instance.

This is the main method used by child classes to react to build metadata.

class mozbuild.gn_processor.GnMozbuildWriterBackend(environment)

Bases: mozbuild.backend.base.BuildBackend


Called when consume() has completed handling all objects.


Consumes an individual TreeMetadata instance.

This is the main method used by child classes to react to build metadata.

class mozbuild.gn_processor.MozbuildWriter(fh)

Bases: object

write_mozbuild_dict(key, value)
write_mozbuild_list(key, value)
write_mozbuild_value(key, value)
mozbuild.gn_processor.filter_gn_config(gn_result, config, sandbox_vars, input_vars, gn_target)
mozbuild.gn_processor.find_deps(all_targets, target)
mozbuild.gn_processor.generate_gn_config(config, srcdir, output, non_unified_sources, gn_binary, input_variables, sandbox_variables, gn_target)
mozbuild.gn_processor.process_gn_config(gn_config, srcdir, config, output, non_unified_sources, sandbox_vars, mozilla_flags)
mozbuild.gn_processor.write_mozbuild(config, srcdir, output, non_unified_sources, gn_config_files, mozilla_flags, write_mozbuild_variables)

mozbuild.html_build_viewer module

class mozbuild.html_build_viewer.BuildViewerServer(address='localhost', port=0)

Bases: object

add_resource_json_file(key, path)

Register a resource JSON file with the server.

The file will be made available under the name/key specified.

add_resource_json_url(key, url)

Register a resource JSON file at a URL.

property url
class mozbuild.html_build_viewer.HTTPHandler(request, client_address, server)

Bases: http.server.BaseHTTPRequestHandler

serve_docroot(root, path)

mozbuild.jar module provides a python class to package up chrome content by processing files.

See the documentation for on MDC for further details on the format.

class mozbuild.jar.JarMaker(outputFormat='flat', useJarfileManifest=True, useChromeManifest=False)

Bases: object

JarMaker reads files and process those into jar files or flat directories, along with chrome.manifest files.

class OutputHelper_flat(basepath)

Bases: object

Provide getDestModTime and getOutput for a given flat output directory. The helper method ensureDirFor is used by the symlink subclass.

getOutput(name, mode='wb')
class OutputHelper_jar(jarfile)

Bases: object

Provide getDestModTime and getOutput for a given jarfile.

getOutput(name, mode='wb')

Bases: mozbuild.jar.JarMaker.OutputHelper_flat

Subclass of OutputHelper_flat that provides a helper for creating a symlink including creating the parent directories.

finalizeJar(jardir, jarbase, jarname, chromebasepath, register, doZip=True)
Helper method to write out the chrome registration entries to

jarfile.manifest or chrome.manifest, or both.

The actual file processing is done in updateManifest.


Get a optparse.OptionParser for jarmaker.

This OptionParser has the options for jarmaker as well as the options for the inner PreProcessor.

makeJar(infile, jardir)

makeJar is the main entry point to JarMaker.

It takes the input file, the output directory, the source dirs and the top source dir as argument, and optionally the l10n dirs.

processJarSection(jarinfo, jardir)

Internal method called by makeJar to actually process a section of a file.

updateManifest(manifestPath, chromebasepath, register)

updateManifest replaces the % in the chrome registration entries with the given chrome base path, and updates the given manifest file.

mozbuild.mach_commands module

class mozbuild.mach_commands.Buildsymbols(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Produce a package of debug symbols suitable for use with Breakpad.

class mozbuild.mach_commands.CargoProvider(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Invoke cargo in useful ways.

check(all_crates=None, crates=None, jobs=0, verbose=False)
class mozbuild.mach_commands.Clobber(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

CLOBBER_CHOICES = {'gradle', 'objdir', 'python'}
clobber(what, full=False)

Clean up the source and object directories.

Performing builds and running various commands generate various files.

Sometimes it is necessary to clean up these files in order to make things work again. This command can be used to perform that cleanup.

The objdir target removes most files in the current object directory (where build output is stored). Some files (like Visual Studio project files) are not removed by default. If you would like to remove the object directory in its entirety, run with –full.

The python target will clean up various generated Python files from the source directory and will remove untracked files from well-known directories containing Python packages. Run this to remove .pyc files, compiled C extensions, etc. Note: all files not tracked or ignored by version control in third_party/python will be deleted. Run the status command of your VCS to see if any untracked files you haven’t committed yet will be deleted.

The gradle target will remove the “gradle” subdirectory of the object directory.

By default, the command clobbers the objdir and python targets.

property substs
class mozbuild.mach_commands.CreateMachEnvironment(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Create the mach virtualenvs.

class mozbuild.mach_commands.Doctor(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Provide commands for diagnosing common build environment problems

class mozbuild.mach_commands.GTestCommands(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

android_gtest(test_dir, shuffle, gtest_filter, package, adb_path, device_serial, remote_test_root, libxul_path, enable_webrender, install)
gtest(shuffle, jobs, gtest_filter, tbpl_parser, enable_webrender, package, adb_path, device_serial, remote_test_root, libxul_path, no_install, debug, debugger, debugger_args)
prepend_debugger_args(args, debugger, debugger_args)

Given an array with program arguments, prepend arguments to run it under a debugger.

  • args – The executable and arguments used to run the process normally.

  • debugger – The debugger to use, or empty to use the default debugger.

  • debugger_args – Any additional parameters to pass to the debugger.

class mozbuild.mach_commands.Install(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Install a package.

class mozbuild.mach_commands.L10NCommands(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

package_l10n(verbose=False, locales=[])
class mozbuild.mach_commands.Logs(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Provide commands to read mach logs.

class mozbuild.mach_commands.MachDebug(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

environment(format, output=None, verbose=False)
class mozbuild.mach_commands.Package(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Package the built product for distribution.

class mozbuild.mach_commands.Repackage(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Repackages artifacts into different formats.

This is generally used after packages are signed by the signing scriptworkers in order to bundle things up into shippable formats, such as a .dmg on OSX or an installer exe on Windows.

repackage_dmg(input, output)
repackage_installer(tag, setupexe, package, output, package_name, sfx_stub, use_upx)
repackage_mar(input, mar, output, arch, mar_channel_id)
repackage_msi(wsx, version, locale, arch, setupexe, candle, light, output)
class mozbuild.mach_commands.RunProgram(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Run the compiled program.

class mozbuild.mach_commands.RunSettings

Bases: object

config_settings = [('runprefs.*', 'string', 'Pass a pref into Firefox when using `mach run`, of the form ``.\nPrefs will automatically be cast into the appropriate type. Integers can be\nsingle quoted to force them to be strings.')]
class mozbuild.mach_commands.StoreDebugParamsAndWarnAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

class mozbuild.mach_commands.TelemetrySettings

Bases: object

config_settings = [('build.telemetry', 'boolean', 'Enable submission of build system telemetry.', False)]
class mozbuild.mach_commands.Warnings(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Provide commands for inspecting warnings.

property database
property database_path
join_ensure_dir(dir1, dir2)
list(directory=None, flags=None, report=None)
summary(directory=None, report=None)
class mozbuild.mach_commands.Watch(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Interface to watch and re-build the tree.


Watch and re-build the source tree.


mozbuild.makeutil module

class mozbuild.makeutil.Makefile

Bases: object

Provides an interface for writing simple makefiles

Instances of this class are created, populated with rules, then written.


Add a raw statement in the makefile. Meant to be used for simple variable assignments.


Create a new rule in the makefile for the given targets. Returns the corresponding Rule instance.

dump(fh, removal_guard=True)

Dump all the rules to the given file handle. Optionally (and by default), add guard rules for file removals (empty rules for other rules’ dependencies)

class mozbuild.makeutil.Rule(targets=())

Bases: object

Class handling simple rules in the form: target1 target2 … : dep1 dep2 …

command1 command2 …


Add commands to the rule.


Add dependencies to the rule.


Add additional targets to the rule.


Return an iterator on the rule commands.


Return an iterator on the rule dependencies.


Dump the rule to the given file handle.


Return an iterator on the rule targets.


Read the file handler containing a dep makefile (simple makefile only containing dependencies) and returns an iterator of the corresponding Rules it contains. Ignores removal guard rules.

mozbuild.makeutil.write_dep_makefile(fh, target, deps)

Write a Makefile containing only target’s dependencies to the file handle specified.

mozbuild.mozconfig module

exception mozbuild.mozconfig.MozconfigLoadException(path, message, output=None)

Bases: Exception

Raised when a mozconfig could not be loaded properly.

This typically indicates a malformed or misbehaving mozconfig file.

class mozbuild.mozconfig.MozconfigLoader(topsrcdir)

Bases: object

Handles loading and parsing of mozconfig files.

AUTODETECT = <object object>
RE_MAKE_VARIABLE = re.compile('\n ^\\s* # Leading whitespace\n (?P<var>[a-zA-Z_0-9]+) # Variable name\n \\s* [?:]?= \\s* # Assignment operator surrounded by optional\n , re.VERBOSE)

Read the contents of a mozconfig into a data structure.

This takes the path to a mozconfig to load. If the given path is AUTODETECT, will try to find a mozconfig from the environment using find_mozconfig().

mozconfig files are shell scripts. So, we can’t just parse them. Instead, we run the shell script in a wrapper which allows us to record state from execution. Thus, the output from a mozconfig is a friendly static data structure.

mozbuild.mozinfo module

mozbuild.mozinfo.build_dict(config, env=environ({'GECKO_BASE_REPOSITORY': '', 'MOZ_AUTOMATION': '1', 'USER': 'worker', 'HOSTNAME': 'taskcluster-worker', 'TASKCLUSTER_PORT': 'tcp://', 'SHLVL': '1', 'TASKCLUSTER_PROXY_URL': 'http://taskcluster', 'HOME': '/builds/worker', 'TASKCLUSTER_NAME': '/stoic_mccarthy/taskcluster', 'MOZ_SCM_LEVEL': '3', 'TASKCLUSTER_PORT_80_TCP_ADDR': '', 'TASKCLUSTER_PORT_80_TCP_PORT': '80', 'GECKO_HEAD_REV': '8d8561344299516728989604a0c7a14d2bff91e7', 'TASKCLUSTER_PORT_80_TCP_PROTO': 'tcp', 'TASK_ID': 'dp-Nzcn_TzmW3yIsH_LnSg', 'LOGNAME': 'worker', '_': './mach', 'TERM': 'xterm', 'PATH': '/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/docs/html/_venv/bin:/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'GECKO_PATH': '/builds/worker/checkouts/gecko', 'TASKCLUSTER_WORKER_GROUP': 'us-west-1', 'TASKCLUSTER_PORT_80_TCP': 'tcp://', 'TASKCLUSTER_PUBLIC_IP': '', 'LANG': 'en_US.UTF-8', 'TASKCLUSTER_WORKER_TYPE': 't-linux-xlarge-source', 'SHELL': '/bin/bash', 'TASKCLUSTER_ROOT_URL': '', 'TASKCLUSTER_INSTANCE_TYPE': 'm5d.xlarge', 'TASKCLUSTER_WORKER_LOCATION': '{"availabilityZone":"us-west-1c","cloud":"aws","region":"us-west-1"}', 'GECKO_HEAD_REPOSITORY': '', 'LC_ALL': 'en_US.UTF-8', 'PWD': '/builds/worker/checkouts/gecko', 'SCCACHE_DISABLE': '1', 'RUN_ID': '0', 'HG_STORE_PATH': '/builds/worker/checkouts/hg-store', 'MACH_MAIN_PID': '16', 'VIRTUAL_ENV': '/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/docs/html/_venv', 'DOCUTILSCONFIG': '/builds/worker/checkouts/gecko/docs/docutils.conf'}))

Build a dict containing data about the build configuration from the environment.

mozbuild.mozinfo.write_mozinfo(file, config, env=environ({'GECKO_BASE_REPOSITORY': '', 'MOZ_AUTOMATION': '1', 'USER': 'worker', 'HOSTNAME': 'taskcluster-worker', 'TASKCLUSTER_PORT': 'tcp://', 'SHLVL': '1', 'TASKCLUSTER_PROXY_URL': 'http://taskcluster', 'HOME': '/builds/worker', 'TASKCLUSTER_NAME': '/stoic_mccarthy/taskcluster', 'MOZ_SCM_LEVEL': '3', 'TASKCLUSTER_PORT_80_TCP_ADDR': '', 'TASKCLUSTER_PORT_80_TCP_PORT': '80', 'GECKO_HEAD_REV': '8d8561344299516728989604a0c7a14d2bff91e7', 'TASKCLUSTER_PORT_80_TCP_PROTO': 'tcp', 'TASK_ID': 'dp-Nzcn_TzmW3yIsH_LnSg', 'LOGNAME': 'worker', '_': './mach', 'TERM': 'xterm', 'PATH': '/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/docs/html/_venv/bin:/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/docs/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'GECKO_PATH': '/builds/worker/checkouts/gecko', 'TASKCLUSTER_WORKER_GROUP': 'us-west-1', 'TASKCLUSTER_PORT_80_TCP': 'tcp://', 'TASKCLUSTER_PUBLIC_IP': '', 'LANG': 'en_US.UTF-8', 'TASKCLUSTER_WORKER_TYPE': 't-linux-xlarge-source', 'SHELL': '/bin/bash', 'TASKCLUSTER_ROOT_URL': '', 'TASKCLUSTER_INSTANCE_TYPE': 'm5d.xlarge', 'TASKCLUSTER_WORKER_LOCATION': '{"availabilityZone":"us-west-1c","cloud":"aws","region":"us-west-1"}', 'GECKO_HEAD_REPOSITORY': '', 'LC_ALL': 'en_US.UTF-8', 'PWD': '/builds/worker/checkouts/gecko', 'SCCACHE_DISABLE': '1', 'RUN_ID': '0', 'HG_STORE_PATH': '/builds/worker/checkouts/hg-store', 'MACH_MAIN_PID': '16', 'VIRTUAL_ENV': '/builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/docs/html/_venv', 'DOCUTILSCONFIG': '/builds/worker/checkouts/gecko/docs/docutils.conf'}))

Write JSON data about the configuration specified in config and an environment variable dict to |file|, which may be a filename or file-like object. See build_dict for information about what environment variables are used, and what keys are produced.

mozbuild.nodeutil module

mozbuild.nodeutil.check_executable_version(exe, wrap_call_with_node=False)

Determine the version of a Node executable by invoking it.

May raise subprocess.CalledProcessError or ValueError on failure.

mozbuild.nodeutil.find_executable(names, min_version, use_node_for_version_check=False)
mozbuild.nodeutil.find_node_executable(nodejs_exe=None, min_version=StrictVersion('10.22.1'))

Find a Node executable from the mozbuild directory.

Returns a tuple containing the the path to an executable binary and a version tuple. Both tuple entries will be None if a Node executable could not be resolved.


Determines the possible paths for node executables.

Returns a list of paths, which includes the build state directory.


Find a Node executable from the mozbuild directory.

Returns a tuple containing the the path to an executable binary and a version tuple. Both tuple entries will be None if a Node executable could not be resolved.

mozbuild.preprocessor module

This is a very primitive line based preprocessor, for times when using a C preprocessor isn’t an option.

It currently supports the following grammar for expressions, whitespace is ignored:

expression :

and_cond ( ‘||’ expression ) ? ;


test ( ‘&&’ and_cond ) ? ;


unary ( ( ‘==’ | ‘!=’ ) unary ) ? ;

unary :

‘!’? value ;

value :

[0-9]+ # integer | ‘defined(‘ w+ ‘)’ | w+ # string identifier or value;

class mozbuild.preprocessor.Context

Bases: dict

This class holds variable values by subclassing dict, and while it truthfully reports True and False on

name in context

it returns the variable name itself on


to reflect the ambiguity between string literals and preprocessor variables.

class mozbuild.preprocessor.Expression(expression_string)

Bases: object

exception ParseError(expression)

Bases: Exception

Error raised when parsing fails. It has two members, offset and content, which give the offset of the error and the offending content.


Evaluate the expression with the given context

class mozbuild.preprocessor.Preprocessor(defines=None, marker='#')

Bases: object

Class for preprocessing text files.

exception Error(cpp, MSG, context)

Bases: RuntimeError


Adds the specified defines to the preprocessor. defines may be a dictionary object or an iterable of key/value pairs (as tuples or other iterables of length two)


Create a clone of the current processor, including line ending settings, marker, variable definitions, output stream.


Reads the input stream, and computes the dependencies for that input.

do_else(args, ifState=2)
do_if(args, replace=False)
do_ifdef(args, replace=False)
do_ifndef(args, replace=False)
do_include(args, filters=True)

Preprocess a given file. args can either be a file name, or a file-like object. Files should be opened, and will be closed after processing.

filter_substitution(aLine, fatal=True)
handleCommandLine(args, defaultToStdin=False)

Parse a commandline into this parser. Uses OptionParser internally, no args mean sys.argv[1:].


Handle a single line of input (internal).

processFile(input, output, depfile=None)

Preprocesses the contents of the input stream and writes the result to the output stream. If depfile is set, the dependencies of output file are written to depfile in Makefile format.


Set the marker to be used for processing directives. Used for handling CSS files, with pp.setMarker(‘%’), for example. The given marker may be None, in which case no markers are processed.


Sets whether missing directive warnings are silenced, according to value. The default behavior of the preprocessor is to emit such warnings.


Internal method for handling output.

mozbuild.preprocessor.preprocess(includes=[<_io.TextIOWrapper name='<stdin>' mode='r' encoding='UTF-8'>], defines={}, output=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>, marker='#')

mozbuild.pythonutil module


Determine the version of a Python executable by invoking it.

May raise subprocess.CalledProcessError or ValueError on failure.

mozbuild.schedules module

Constants for SCHEDULES configuration in files and for skip-unless-schedules optimizations in task-graph generation.

mozbuild.shellutil module

exception mozbuild.shellutil.MetaCharacterException(char)

Bases: Exception


Given one or more strings, returns a quoted string that can be used literally on a shell command line.

>>> quote('a', 'b')
"a b"
>>> quote('a b', 'c')
"'a b' c"

Split the given command line string.

mozbuild.sphinx module

class mozbuild.sphinx.MozbuildSymbols(name, arguments, options, content, lineno, content_offset, block_text, state, state_machine)

Bases: docutils.parsers.rst.Directive

Directive to insert mozbuild sandbox symbol information.

required_arguments = 1
mozbuild.sphinx.function_reference(f, attr, args, doc)
mozbuild.sphinx.special_reference(v, func, typ, doc)
mozbuild.sphinx.variable_reference(v, st_type, in_type, doc)

mozbuild.telemetry module


Read the CPU brand string out of /proc/cpuinfo on Linux.


Get the CPU brand string via sysctl on macos.


Read the CPU brand string from the registry on Windows.

mozbuild.telemetry.filter_args(command, argv, instance)

Given the full list of command-line arguments, remove anything up to and including command, and attempt to filter absolute pathnames out of any arguments after that.

mozbuild.telemetry.gather_telemetry(command, success, start_time, end_time, mach_context, instance, command_attrs)

Gather telemetry about the build and the user’s system and pass it to the telemetry handler to be stored for later submission.

Any absolute paths on the command line will be made relative to a relevant base path or replaced with a placeholder to avoid including paths from developer’s machines.


Extracts clobber and cpu usage info from command attributes.


Translate selected items from substs into build_opts keys in the schema.


Get a client id, which is a UUID, from a file in the state directory. If the file doesn’t exist, generate a UUID and save it to a file.


Get the CPU brand string as returned by CPUID.


Return whether psutil exists and its associated stats.

@returns (bool, int, int, int) whether psutil exists, the logical CPU count,

physical CPU count, and total number of bytes of memory.


Gather info to fill the system keys in the schema.


Verifies the statedir is structured correctly. Returns the outgoing, submitted and log paths.

Requires presence of the following directories; will raise if absent: - statedir/telemetry - statedir/telemetry/outgoing

Creates the following directories and files if absent (first submission): - statedir/telemetry/submitted

mozbuild.testing module

class mozbuild.testing.SupportFilesConverter

Bases: object

Processes a “support-files” entry from a test object, either from a parsed object from a test manifests or its representation in and returns the installs to perform for this test object.

Processing the same support files multiple times will not have any further effect, and the structure of the parsed objects from manifests will have a lot of repeated entries, so this class takes care of memoizing.

convert_support_files(test, install_root, manifest_dir, out_dir)
class mozbuild.testing.TestInstallInfo

Bases: object

mozbuild.testing.install_test_files(topsrcdir, topobjdir, tests_root)

Installs the requested test files to the objdir. This is invoked by test runners to avoid installing tens of thousands of test files when only a few tests need to be run.

mozbuild.testing.read_manifestparser_manifest(context, manifest_path)
mozbuild.testing.read_reftest_manifest(context, manifest_path)
mozbuild.testing.read_wpt_manifest(context, paths)

mozbuild.util module

class mozbuild.util.DefinesAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: argparse.Action

An ArgumentParser action to handle -Dvar[=value] type of arguments.

class mozbuild.util.EmptyValue

Bases: str

A dummy type that behaves like an empty string and sequence.

This type exists in order to support mozbuild.frontend.reader.EmptyConfig. It should likely not be used elsewhere.

class mozbuild.util.EnumString(value)

Bases: str

A string type that only can have a limited set of values, similarly to an Enum, and can only be compared against that set of values.

The class is meant to be subclassed, where the subclass defines POSSIBLE_VALUES. The subclass method is a helper to create such subclasses.

static subclass(*possible_values)
exception mozbuild.util.EnumStringComparisonError

Bases: Exception

class mozbuild.util.FileAvoidWrite(filename, capture_diff=False, dry_run=False, readmode='rU')

Bases: _io.BytesIO

File-like object that buffers output and only writes if content changed.

We create an instance from an existing filename. New content is written to it. When we close the file object, if the content in the in-memory buffer differs from what is on disk, then we write out the new content. Otherwise, the original file is untouched.

Instances can optionally capture diffs of file changes. This feature is not enabled by default because it a) doesn’t make sense for binary files b) could add unwanted overhead to calls.

Additionally, there is dry run mode where the file is not actually written out, but reports whether the file was existing and would have been updated still occur, as well as diff capture if requested.


Stop accepting writes, compare file contents, and rewrite if needed.

Returns a tuple of bools indicating what action was performed:

(file existed, file updated)

If capture_diff was specified at construction time and the underlying file was changed, .diff will be populated with the diff of the result.


Write bytes to file.

Return the number of bytes written.


Returns a class which holds optional flags for an item in a list.

The flags are defined in the dict given as argument, where keys are the flag names, and values the type used for the value of that flag.

The resulting class is used by the various <TypeName>WithFlagsFactory functions below.

class mozbuild.util.HierarchicalStringList

Bases: object

A hierarchy of lists of strings.

Each instance of this object contains a list of strings, which can be set or appended to. A sub-level of the hierarchy is also an instance of this class, can be added by appending to an attribute instead.

For example, the variable EXPORTS is an instance of this class. We can do:

EXPORTS += [‘foo.h’] EXPORTS.mozilla.dom += [‘bar.h’]

In this case, we have 3 instances (EXPORTS, EXPORTS.mozilla, and EXPORTS.mozilla.dom), and the first and last each have one element in their list.

class StringListAdaptor(hsl)



Walk over all HierarchicalStringLists in the hierarchy.

This is a generator of (path, sequence).

The path is ‘’ for the root level and ‘/’-delimited strings for any descendants. The sequence is a read-only sequence of the strings contained at that level.

class mozbuild.util.ImmutableStrictOrderingOnAppendList(iterable=None, **kwargs)

Bases: mozbuild.util.StrictOrderingOnAppendList

Like StrictOrderingOnAppendList, but not allowing mutations of the value.

append(object) → None – append object to end
extend(iterable) → None – extend list by appending elements from the iterable
class mozbuild.util.KeyedDefaultDict(default_factory, *args, **kwargs)

Bases: dict

Like a defaultdict, but the default_factory function takes the key as argument

class mozbuild.util.List(iterable=None, **kwargs)

Bases: list

A list specialized for environments.

We overload the assignment and append operations to require that the appended thing is a list. This avoids bad surprises coming from appending a string to a list, which would just add each letter of the string.

extend(iterable) → None – extend list by appending elements from the iterable
class mozbuild.util.LockFile(lockfile)

Bases: object

LockFile is used by the lock_file method to hold the lock.

This object should not be used directly, but only through the lock_file method below.

exception mozbuild.util.MozbuildDeletionError

Bases: Exception

class mozbuild.util.OrderedDefaultDict(default_factory, *args, **kwargs)

Bases: collections.OrderedDict

A combination of OrderedDict and defaultdict.

class mozbuild.util.ReadOnlyDefaultDict(default_factory, *args, **kwargs)

Bases: mozbuild.util.ReadOnlyDict

A read-only dictionary that supports default values on retrieval.

class mozbuild.util.ReadOnlyDict(*args, **kwargs)

Bases: dict

A read-only dictionary.

update([E, ]**F) → None. Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

class mozbuild.util.ReadOnlyKeyedDefaultDict(default_factory, *args, **kwargs)

Bases: mozbuild.util.KeyedDefaultDict, mozbuild.util.ReadOnlyDict

Like KeyedDefaultDict, but read-only.

class mozbuild.util.ReadOnlyNamespace(**kwargs)

Bases: object

A class for objects with immutable attributes set at initialization.

class mozbuild.util.StrictOrderingOnAppendList(iterable=None, **kwargs)

Bases: mozbuild.util.List

A list specialized for environments.

We overload the assignment and append operations to require that incoming elements be ordered. This enforces cleaner style in files.

static ensure_sorted(l)
extend(iterable) → None – extend list by appending elements from the iterable
class mozbuild.util.StrictOrderingOnAppendListWithAction(iterable=(), action=None)

Bases: mozbuild.util.StrictOrderingOnAppendList

An ordered list that accepts a callable to be applied to each item.

A callable (action) passed to the constructor is run on each item of input. The result of running the callable on each item will be stored in place of the original input, but the original item must be used to enforce sortedness.

extend(iterable) → None – extend list by appending elements from the iterable
class mozbuild.util.StrictOrderingOnAppendListWithFlags(iterable=None, **kwargs)

Bases: mozbuild.util.StrictOrderingOnAppendList

A list with flags specialized for environments.

Each subclass has a set of typed flags; this class lets us use isinstance for natural testing.


Returns a StrictOrderingOnAppendList-like object, with optional flags on each item.

The flags are defined in the dict given as argument, where keys are the flag names, and values the type used for the value of that flag.


FooList = StrictOrderingOnAppendListWithFlagsFactory({

‘foo’: bool, ‘bar’: unicode

}) foo = FooList([‘a’, ‘b’, ‘c’]) foo[‘a’].foo = True foo[‘b’].bar = ‘bar’

mozbuild.util.TypedList(type, base_class=<class 'mozbuild.util.List'>)

A list with type coercion.

The given type is what list elements are being coerced to. It may do strict validation, throwing ValueError exceptions.

A base_class type can be given for more specific uses than a List. For example, a Typed StrictOrderingOnAppendList can be created with:

TypedList(unicode, StrictOrderingOnAppendList)

mozbuild.util.TypedNamedTuple(name, fields)

Factory for named tuple types with strong typing.

Arguments are an iterable of 2-tuples. The first member is the the field name. The second member is a type the field will be validated to be.

Construction of instances varies from collections.namedtuple.

First, if a single tuple argument is given to the constructor, this is treated as the equivalent of passing each tuple value as a separate argument into __init__. e.g.:

t = (1, 2)
TypedTuple(t) == TypedTuple(1, 2)

This behavior is meant for files, so vanilla tuples are automatically cast to typed tuple instances.

Second, fields in the tuple are validated to be instances of the specified type. This is done via an isinstance() check. To allow multiple types, pass a tuple as the allowed types field.

exception mozbuild.util.UnsortedError(srtd, original)

Bases: Exception


Ensures the directory parent to the given file exists.

mozbuild.util.ensure_bytes(value, encoding='utf-8')
mozbuild.util.ensure_subprocess_env(env, encoding='utf-8')

Ensure the environment is in the correct format for the subprocess module.

This will convert all keys and values to bytes on Python 2, and text on Python 3.

  • env (dict) – Environment to ensure.

  • encoding (str) – Encoding to use when converting to/from bytes/text (default: utf-8).

mozbuild.util.ensure_unicode(value, encoding='utf-8')
mozbuild.util.exec_(object, globals=None, locals=None)

Wrapper around the exec statement to avoid bogus errors like:

SyntaxError: unqualified exec is not allowed in function … it is a nested function.


SyntaxError: unqualified exec is not allowed in function … it contains a nested function with free variable

which happen with older versions of python 2.7.

mozbuild.util.expand_variables(s, variables)

Given a string with $(var) variable references, replace those references with the corresponding entries from the given variables dict.

If a variable value is not a string, it is iterated and its items are joined with a whitespace.

mozbuild.util.group_unified_files(files, unified_prefix, unified_suffix, files_per_unified_file)

Return an iterator of (unified_filename, source_filenames) tuples.

We compile most C and C++ files in “unified mode”; instead of compiling a.cpp, b.cpp, and c.cpp separately, we compile a single file that looks approximately like:

#include "a.cpp"
#include "b.cpp"
#include "c.cpp"

This function handles the details of generating names for the unified files, and determining which original source files go in which unified file.

mozbuild.util.hash_file(path, hasher=None)

Hashes a file specified by the path given and returns the hex digest.


Returns a list of hexdump-like lines corresponding to the given input buffer.

mozbuild.util.lock_file(lockfile, max_wait=600)

Create and hold a lockfile of the given name, with the given timeout.

To release the lock, delete the returned object.

class mozbuild.util.memoize(func)

Bases: dict

A decorator to memoize the results of function calls depending on its arguments. Both functions and instance methods are handled, although in the instance method case, the results are cache in the instance itself.

method_call(instance, *args)
class mozbuild.util.memoized_property(func)

Bases: object

A specialized version of the memoize decorator that works for class instance properties.

mozbuild.util.mkdir(path, not_indexed=False)

Ensure a directory exists.

If not_indexed is True, an attribute is set that disables content indexing on the directory.


Given an iterable, returns an iterable pairing its items.

For example,



[(1,2), (3,4), (5,6)]


Given an iterable, returns an iterable of overlapped pairs of its items. Based on the Python itertools documentation.

For example,



[(1,2), (2,3), (3,4), (4,5), (5,6)]


This is a hack to work around the fact that Windows multiprocessing needs to import the original main module, and assumes that it corresponds to a file ending in .py.

We do this by a sort of two-level function interposing. The first level interposes forking.get_command_line() with our version defined in my_get_command_line(). Our version of get_command_line will replace the command string with the contents of the fork_interpose() function to be used in the subprocess.

The subprocess then gets an interposed imp.find_module(), which we hack up to find the main module name multiprocessing will assume, since we know what this will be based on the main module in the parent. If we’re not looking for our main module, then the original find_module will suffice.

See also: And:

mozbuild.util.resolve_target_to_make(topobjdir, target)

Resolve target (a target, directory, or file) to a make target.

topobjdir is the object directory; all make targets will be rooted at or below the top-level Makefile in this directory.

Returns a pair (reldir, target) where reldir is a directory relative to topobjdir containing a Makefile and target is a make target (possibly None).

A directory resolves to the nearest directory at or above containing a Makefile, and target None.

A regular (non-Makefile) file resolves to the nearest directory at or above the file containing a Makefile, and an appropriate target.

A Makefile resolves to the nearest parent strictly above the Makefile containing a different Makefile, and an appropriate target.

mozbuild.util.simple_diff(filename, old_lines, new_lines)

Returns the diff between old_lines and new_lines, in unified diff form, as a list of lines.

old_lines and new_lines are lists of non-newline terminated lines to compare. old_lines can be None, indicating a file creation. new_lines can be None, indicating a file deletion.

class mozbuild.util.undefined_default

Bases: object

Represents an undefined argument value that isn’t None.

mozbuild.util.write_indented_repr(f, o, indent=4)

Write an indented representation (similar to repr()) of the object to the given file f.

One notable difference with repr is that the returned representation assumes from __future__ import unicode_literals.

mozbuild.virtualenv module

class mozbuild.virtualenv.VirtualenvHelper(virtualenv_path)

Bases: object

Contains basic logic for getting information about virtualenvs.

property bin_path
property python_path
class mozbuild.virtualenv.VirtualenvManager(topsrcdir, virtualenv_path, log_handle, manifest_path, populate_local_paths=True)

Bases: mozbuild.virtualenv.VirtualenvHelper

Contains logic for managing virtualenvs for building the tree.


Activate the virtualenv in this Python context.

If you run a random Python script and wish to “activate” the virtualenv, you can simply instantiate an instance of this class and call .ensure() and .activate() to make the virtualenv active.

property activate_path

Build a virtualenv per tree conventions.

This returns the path of the created virtualenv.

call_setup(directory, arguments)

Calls in a directory.


Create a new, empty virtualenv.

Receives the path to virtualenv’s script (which will be called out to), the path to create the virtualenv in, and a handle to write output to.


Ensure the virtualenv is present and up to date.

If the virtualenv is up to date, this does nothing. Otherwise, it creates and populates the virtualenv as necessary.

This should be the main API used from this class as it is the highest-level.


Returns the version of the python executable that was in use when this virtualenv was created.

install_pip_package(package, vendored=False)

Install a package via pip.

The supplied package is specified using a pip requirement specifier. e.g. ‘foo’ or ‘foo==1.0’.

If the package is already installed, this is a no-op.

If vendored is True, no package index will be used and no dependencies will be installed.

install_pip_requirements(path, require_hashes=True, quiet=False, vendored=False)

Install a pip requirements.txt file.

The supplied path is a text file containing pip requirement specifiers.

If require_hashes is True, each specifier must contain the expected hash of the downloaded package. See:


Populate the virtualenv.

The manifest file consists of colon-delimited fields. The first field specifies the action. The remaining fields are arguments to that action. The following actions are supported: – Invoke for a package. Expects the arguments:
  1. relative path directory containing

  2. argument(s) to e.g. “develop”. Each program argument is delimited by a colon. Arguments with colons are not yet supported.

filename.pth – Adds the path given as argument to filename.pth under

the virtualenv site packages directory.

optional – This denotes the action as optional. The requested action

is attempted. If it fails, we issue a warning and go on. The initial “optional” field is stripped then the remaining line is processed like normal. e.g. “”

packages.txt – Denotes that the specified path is a child manifest. It

will be read and processed as if its contents were concatenated into the manifest being read.

windows – This denotes that the action should only be taken when run

on Windows.

!windows – This denotes that the action should only be taken when run

on non-Windows systems.

python3 – This denotes that the action should only be taken when run

on Python 3.

python2 – This denotes that the action should only be taken when run

on python 2.

set-variable – Set the given environment variable; e.g.

set-variable FOO=1.

Note that the Python interpreter running this function should be the one from the virtualenv. If it is the system Python or if the environment is not configured properly, packages could be installed into the wrong place. This is how virtualenv’s work.


Run a Python executable and return its sys.hexversion value.


Returns whether the virtualenv is present and up to date.


python – Full path string to the Python executable that this virtualenv should be running. If the Python executable passed in to this argument is not the same version as the Python the virtualenv was built with then this method will return False.

property version_info
property virtualenv_script_path

Path to virtualenv’s own populator script.


Records the the version of the python executable that was in use when this virtualenv was created. We record this explicitly because on OS X our python path may end up being a different or modified executable.

mozbuild.virtualenv.ensure_binary(s, encoding='utf-8')
mozbuild.virtualenv.ensure_subprocess_env(env, encoding='utf-8')

Ensure the environment is in the correct format for the subprocess module.

This method uses the method with same name from mozbuild.utils as must be a standalone module.

This will convert all keys and values to bytes on Python 2, and text on Python 3.

  • env (dict) – Environment to ensure.

  • encoding (str) – Encoding to use when converting to/from bytes/text (default: utf-8).

mozbuild.virtualenv.ensure_text(s, encoding='utf-8')

Ensure the current version of Python is sufficient.

Module contents