mozbuild package

Subpackages

Submodules

mozbuild.android_version_code module

mozbuild.android_version_code.android_version_code(buildid, *args, **kwargs)
mozbuild.android_version_code.android_version_code_v0(buildid, cpu_arch=None, min_sdk=0, max_sdk=0)
mozbuild.android_version_code.android_version_code_v1(buildid, cpu_arch=None, min_sdk=0, max_sdk=0)

Generate a v1 android:versionCode. The important consideration is that version codes be monotonically increasing (per Android package name) for all published builds. The input build IDs are based on timestamps and hence are always monotonically increasing.

The generated v1 version codes look like (in binary):

0111 1000 0010 tttt tttt tttt tttt txpg

The 17 bits labelled ‘t’ represent the number of hours since midnight on September 1, 2015. (2015090100 in YYYYMMMDDHH format.) This yields a little under 15 years worth of hourly build identifiers, since 2**17 / (366 * 24) =~ 14.92.

The bits labelled ‘x’, ‘p’, and ‘g’ are feature flags.

The bit labelled ‘x’ is 1 if the build is for an x86 or x86-64 architecture, and 0 otherwise, which means the build is for an ARM or ARM64 architecture. (Fennec no longer supports ARMv6, so ARM is equivalent to ARMv7.

ARM64 is also known as AArch64; it is logically ARMv8.)

For the same release, x86 and x86_64 builds have higher version codes and take precedence over ARM builds, so that they are preferred over ARM on devices that have ARM emulation.

The bit labelled ‘p’ is 1 if the build is for a 64-bit architecture (x86-64 or ARM64), and 0 otherwise, which means the build is for a 32-bit architecture (x86 or ARM). 64-bit builds have higher version codes so they take precedence over 32-bit builds on devices that support 64-bit.

The bit labelled ‘g’ is 1 if the build targets a recent API level, which is currently always the case, because Firefox no longer ships releases that are split by API levels. However, we may reintroduce a split in the future, in which case the release that targets an older API level will

We throw an explanatory exception when we are within one calendar year of running out of build events. This gives lots of time to update the version scheme. The responsible individual should then bump the range (to allow builds to continue) and use the time remaining to update the version scheme via the reserved high order bits.

N.B.: the reserved 0 bit to the left of the highest order ‘t’ bit can, sometimes, be used to bump the version scheme. In addition, by reducing the granularity of the build identifiers (for example, moving to identifying builds every 2 or 4 hours), the version scheme may be adjusted further still without losing a (valuable) high order bit.

mozbuild.android_version_code.main(argv)

mozbuild.artifact_builds module

mozbuild.artifact_cache module

Fetch and cache artifacts from URLs.

This module manages fetching artifacts from URLS and purging old artifacts using a simple Least Recently Used cache.

This module requires certain modules be importable from the ambient Python environment. Consumers will need to arrange this themselves.

The bulk of the complexity is in managing and persisting several caches. If we found a Python LRU cache that pickled cleanly, we could remove a lot of this code! Sadly, I found no such candidate implementations, so we pickle pylru caches manually.

None of the instances (or the underlying caches) are safe for concurrent use. A future need, perhaps.

class mozbuild.artifact_cache.ArtifactCache(cache_dir, log=None, skip_cache=False)

Bases: object

Fetch artifacts from URLS and purge least recently used artifacts from disk.

clear_cache()
fetch(url, force=False)
log(*args, **kwargs)
class mozbuild.artifact_cache.ArtifactPersistLimit(log=None)

Bases: PersistLimit

Handle persistence for a cache of artifacts.

When instantiating a DownloadManager, it starts by filling the PersistLimit instance it’s given with register_dir_content. In practice, this registers all the files already in the cache directory. After a download finishes, the newly downloaded file is registered, and the oldest files registered to the PersistLimit instance are removed depending on the size and file limits it’s configured for.

This is all good, but there are a few tweaks we want here:

  • We have pickle files in the cache directory that we don’t want purged.

  • Files that were just downloaded in the same session shouldn’t be purged. (if for some reason we end up downloading more than the default max size, we don’t want the files to be purged)

To achieve this, this subclass of PersistLimit inhibits the register_file method for pickle files and tracks what files were downloaded in the same session to avoid removing them.

The register_file method may be used to register cache matches too, so that later sessions know they were freshly used.

log(*args, **kwargs)
register_dir_content(directory, pattern='*')

Register every files in a directory that match pattern.

register_file(path)

register a single file.

remove_all()
remove_old_files()

remove oldest registered files.

mozbuild.artifact_commands module

class mozbuild.artifact_commands.ArtifactSubCommand(command, subcommand, description=None, parser=None, metrics_path: str | None = None, virtualenv_name: str | None = None)

Bases: SubCommand

class mozbuild.artifact_commands.SymbolsAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: Action

mozbuild.artifact_commands.artifact(command_context)

Download, cache, and install pre-built binary artifacts to build Firefox.

Use mach build as normal to freshen your installed binary libraries: artifact builds automatically download, cache, and install binary artifacts from Mozilla automation, replacing whatever may be in your object directory. Use mach artifact last to see what binary artifacts were last used.

Never build libxul again!

mozbuild.artifact_commands.artifact_clear_cache(command_context, tree=None, job=None, verbose=False)
mozbuild.artifact_commands.artifact_install(command_context, source=None, skip_cache=False, tree=None, job=None, verbose=False, no_tests=False, symbols=False, distdir=None, no_process=False, maven_zip=False)
mozbuild.artifact_commands.artifact_toolchain(command_context, verbose=False, cache_dir=None, skip_cache=False, from_build=(), from_task=(), tooltool_manifest=None, no_unpack=False, retry=0, bootstrap=False, artifact_manifest=None)

Download, cache and install pre-built toolchains.

mozbuild.artifacts module

Fetch build artifacts from a Firefox tree.

This provides an (at-the-moment special purpose) interface to download Android artifacts from Mozilla’s Task Cluster.

This module performs the following steps:

  • find a candidate hg parent revision. At one time we used the local pushlog, which required the mozext hg extension. This isn’t feasible with git, and it is only mildly less efficient to not use the pushlog, so we don’t use it even when querying hg.

  • map the candidate parent to candidate Task Cluster tasks and artifact locations. Pushlog entries might not correspond to tasks (yet), and those tasks might not produce the desired class of artifacts.

  • fetch fresh Task Cluster artifacts and purge old artifacts, using a simple Least Recently Used cache.

  • post-process fresh artifacts, to speed future installation. In particular, extract relevant files from Mac OS X DMG files into a friendly archive format so we don’t have to mount DMG files frequently.

This module requires certain modules be importable from the ambient Python environment. mach artifact ensures these modules are available, but other consumers will need to arrange this themselves.

class mozbuild.artifacts.AndroidArtifactJob(log=None, download_tests=True, download_symbols=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: ArtifactJob

package_artifact_patterns = {'**/*.so'}
package_re = 'public/build/geckoview_example\\.apk$'
process_package_artifact(filename, processed_filename)
process_symbols_archive(filename, processed_filename)
product = 'mobile'
class mozbuild.artifacts.ArtifactJob(log=None, download_tests=True, download_symbols=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: object

beta_candidate_trees = ['releases/mozilla-beta']
property candidate_trees
default_candidate_trees = ['releases/mozilla-release']
esr_candidate_trees = ['releases/mozilla-esr115', 'releases/mozilla-esr128']
find_candidate_artifacts(artifacts)
get_writer(**kwargs)
iter_artifact_archive(filename)
log(*args, **kwargs)
nightly_candidate_trees = ['mozilla-central', 'integration/autoland']
process_artifact(filename, processed_filename)
process_extra_archive(filename, processed_filename)
process_package_artifact(filename, processed_filename)
process_symbols_archive(filename, processed_filename, skip_compressed=False)
process_tests_tar_artifact(filename, processed_filename)
process_tests_zip_artifact(filename, processed_filename)
select_candidate_trees()
test_artifact_patterns = {('bin/BadCertAndPinningServer', ('bin', 'bin')), ('bin/DelegatedCredentialsServer', ('bin', 'bin')), ('bin/EncryptedClientHelloServer', ('bin', 'bin')), ('bin/FaultyServer', ('bin', 'bin')), ('bin/GenerateOCSPResponse', ('bin', 'bin')), ('bin/OCSPStaplingServer', ('bin', 'bin')), ('bin/SanctionsTestServer', ('bin', 'bin')), ('bin/certutil', ('bin', 'bin')), ('bin/geckodriver', ('bin', 'bin')), ('bin/http3server', ('bin', 'bin')), ('bin/pk12util', ('bin', 'bin')), ('bin/plugin-container', ('bin', 'bin')), ('bin/plugins/*', ('bin/plugins', 'plugins')), ('bin/plugins/gmp-*/*/*', ('bin/plugins', 'bin')), ('bin/screentopng', ('bin', 'bin')), ('bin/ssltunnel', ('bin', 'bin')), ('bin/xpcshell', ('bin', 'bin'))}
trust_domain = 'gecko'
try_tree = 'try'
class mozbuild.artifacts.Artifacts(tree, substs, defines, job=None, log=None, cache_dir='.', hg=None, git=None, skip_cache=False, topsrcdir=None, download_tests=True, download_symbols=False, download_maven_zip=False, no_process=False, mozbuild=None)

Bases: object

Maintain state to efficiently fetch build artifacts from a Firefox tree.

clear_cache()
find_pushhead_artifacts(task_cache, job, tree, pushhead)
install_from(source, distdir)

Install artifacts from a source into the given distdir.

install_from_file(filename, distdir)
install_from_recent(distdir)
install_from_revset(revset, distdir)
install_from_task(taskId, distdir)
install_from_url(url, distdir)
log(*args, **kwargs)
run_hg(*args, **kwargs)
class mozbuild.artifacts.CacheManager(cache_dir, cache_name, cache_size, cache_callback=None, log=None, skip_cache=False)

Bases: object

Maintain an LRU cache. Provide simple persistence, including support for loading and saving the state using a “with” block. Allow clearing the cache and printing the cache for debugging.

Provide simple logging.

clear_cache()
dump_cache()
load_cache()
log(*args, **kwargs)
class mozbuild.artifacts.LinuxArtifactJob(log=None, download_tests=True, download_symbols=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: ArtifactJob

property package_artifact_patterns
package_re = 'public/build/target\\.tar\\.(bz2|xz)$'
process_package_artifact(filename, processed_filename)
product = 'firefox'
class mozbuild.artifacts.LinuxThunderbirdArtifactJob(log=None, download_tests=True, download_symbols=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: ThunderbirdMixin, LinuxArtifactJob

class mozbuild.artifacts.MacArtifactJob(log=None, download_tests=True, download_symbols=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: ArtifactJob

get_writer(**kwargs)
package_re = 'public/build/target\\.dmg$'
property paths_no_keep_path
process_package_artifact(filename, processed_filename)
product = 'firefox'
class mozbuild.artifacts.MacThunderbirdArtifactJob(log=None, download_tests=True, download_symbols=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: ThunderbirdMixin, MacArtifactJob

class mozbuild.artifacts.PushheadCache(cache_dir, log=None, skip_cache=False)

Bases: CacheManager

Helps map tree/revision pairs to parent pushheads according to the pushlog.

parent_pushhead_id(tree, revision)
pushid_range(tree, start, end)
class mozbuild.artifacts.ResignJarWriter(job, **kwargs)

Bases: JarWriter

add(name, data, mode=None)

Add a new member to the jar archive, with the given name and the given data. The compress option indicates how the given data should be compressed (one of JAR_STORED or JAR_DEFLATE), or compressed according to the default defined when creating the JarWriter (None). True and False are allowed values for backwards compatibility, mapping, respectively, to JAR_DEFLATE and JAR_STORED. When the data should be compressed, it is only really compressed if the compressed size is smaller than the uncompressed size. The mode option gives the unix permissions that should be stored for the jar entry, which defaults to 0o100644 (regular file, u+rw, g+r, o+r) if not specified. If a duplicated member is found skip_duplicates will prevent raising an exception if set to True. The given data may be a buffer, a file-like instance, a Deflater or a JarFileReader instance. The latter two allow to avoid uncompressing data to recompress it.

class mozbuild.artifacts.TaskCache(cache_dir, log=None, skip_cache=False)

Bases: CacheManager

Map candidate pushheads to Task Cluster task IDs and artifact URLs.

artifacts(tree, job, artifact_job_class, rev)
class mozbuild.artifacts.ThunderbirdMixin

Bases: object

beta_candidate_trees = ['releases/comm-beta']
default_candidate_trees = ['releases/comm-release']
esr_candidate_trees = ['releases/comm-esr115', 'releases/comm-esr128']
nightly_candidate_trees = ['comm-central']
product = 'thunderbird'
trust_domain = 'comm'
try_tree = 'try-comm-central'
class mozbuild.artifacts.WinArtifactJob(log=None, download_tests=True, download_symbols=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: ArtifactJob

property package_artifact_patterns
package_re = 'public/build/target\\.(zip|tar\\.gz)$'
process_package_artifact(filename, processed_filename)
product = 'firefox'
test_artifact_patterns = {('bin/BadCertAndPinningServer.exe', ('bin', 'bin')), ('bin/DelegatedCredentialsServer.exe', ('bin', 'bin')), ('bin/EncryptedClientHelloServer.exe', ('bin', 'bin')), ('bin/FaultyServer.exe', ('bin', 'bin')), ('bin/GenerateOCSPResponse.exe', ('bin', 'bin')), ('bin/OCSPStaplingServer.exe', ('bin', 'bin')), ('bin/SanctionsTestServer.exe', ('bin', 'bin')), ('bin/certutil.exe', ('bin', 'bin')), ('bin/components/*', ('bin/components', 'bin/components')), ('bin/content_analysis_sdk_agent.exe', ('bin', 'bin')), ('bin/geckodriver.exe', ('bin', 'bin')), ('bin/http3server.exe', ('bin', 'bin')), ('bin/minidumpwriter.exe', ('bin', 'bin')), ('bin/pk12util.exe', ('bin', 'bin')), ('bin/plugins/*', ('bin/plugins', 'plugins')), ('bin/plugins/gmp-*/*/*', ('bin/plugins', 'bin')), ('bin/screenshot.exe', ('bin', 'bin')), ('bin/ssltunnel.exe', ('bin', 'bin')), ('bin/xpcshell.exe', ('bin', 'bin'))}
class mozbuild.artifacts.WinThunderbirdArtifactJob(log=None, download_tests=True, download_symbols=False, download_maven_zip=False, substs=None, mozbuild=None)

Bases: ThunderbirdMixin, WinArtifactJob

mozbuild.artifacts.cachedmethod(cachefunc)

Decorator to wrap a class or instance method with a memoizing callable that saves results in a (possibly shared) cache.

mozbuild.artifacts.startswithwhich(s, prefixes)

mozbuild.base module

exception mozbuild.base.BadEnvironmentException

Bases: Exception

Base class for errors raised when the build environment is not sane.

exception mozbuild.base.BinaryNotFoundException(path)

Bases: Exception

Raised when the binary is not found in the expected location.

help()
exception mozbuild.base.BuildEnvironmentNotFoundException

Bases: BadEnvironmentException, AttributeError

Raised when we could not find a build environment.

class mozbuild.base.ExecutionSummary(summary_format, **data)

Bases: dict

Helper for execution summaries.

extend(summary_format, **data)
class mozbuild.base.MachCommandBase(context, virtualenv_name=None, metrics=None, no_auto_log=False)

Bases: MozbuildObject

Base class for mach command providers that wish to be MozbuildObjects.

This provides a level of indirection so MozbuildObject can be refactored without having to change everything that inherits from it.

class mozbuild.base.MachCommandConditions

Bases: object

A series of commonly used condition functions which can be applied to mach commands with providers deriving from MachCommandBase.

static has_build(cls)

Must have a build.

static has_build_or_shell(cls)

Must have a build or a shell build.

static is_android(cls)

Must have an Android build.

static is_artifact_build(cls)

Must be an artifact build.

static is_buildapp_in(cls, apps)

Must have a build for one of the given app

static is_firefox(cls)

Must have a Firefox build.

static is_firefox_or_android(cls)

Must have a Firefox or Android build.

static is_firefox_or_thunderbird(cls)

Must have a Firefox or Thunderbird build.

static is_git(cls)

Must have a git source checkout.

static is_hg(cls)

Must have a mercurial source checkout.

static is_jsshell(cls)

Must have a jsshell build.

static is_non_artifact_build(cls)

Must not be an artifact build.

static is_not_android(cls)

Must not have an Android build.

static is_thunderbird(cls)

Must have a Thunderbird build.

class mozbuild.base.MozbuildObject(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>, virtualenv_name=None)

Bases: ProcessExecutionMixin

Base class providing basic functionality useful to many modules.

Modules in this package typically require common functionality such as accessing the current config, getting the location of the source directory, running processes, etc. This classes provides that functionality. Other modules can inherit from this class to obtain this functionality easily.

activate_virtualenv()
backend_out_of_date(backend_file)
property base_mozconfig_info
property bindir
build_out_of_date(output, dep_file)
property config_environment

Returns the ConfigEnvironment for the current build configuration.

This property is only available once configure has executed.

If configure’s output is not available, this will raise.

property defines
property distdir
classmethod from_environment(cwd=None, detect_virtualenv_mozinfo=True, **kwargs)

Create a MozbuildObject by detecting the proper one from the env.

This examines environment state like the current working directory and creates a MozbuildObject from the found source directory, mozconfig, etc.

The role of this function is to identify a topsrcdir, topobjdir, and mozconfig file.

If the current working directory is inside a known objdir, we always use the topsrcdir and mozconfig associated with that objdir.

If the current working directory is inside a known srcdir, we use that topsrcdir and look for mozconfigs using the default mechanism, which looks inside environment variables.

If the current Python interpreter is running from a virtualenv inside an objdir, we use that as our objdir.

If we’re not inside a srcdir or objdir, an exception is raised.

detect_virtualenv_mozinfo determines whether we should look for a mozinfo.json file relative to the virtualenv directory. This was added to facilitate testing. Callers likely shouldn’t change the default.

static get_base_mozconfig_info(topsrcdir, path, env_mozconfig)
get_binary_path(what='app', validate_exists=True, where='default')

Obtain the path to a compiled binary for this build configuration.

The what argument is the program or tool being sought after. See the code implementation for supported values.

If validate_exists is True (the default), we will ensure the found path exists before returning, raising an exception if it doesn’t.

If where is ‘staged-package’, we will return the path to the binary in the package staging directory.

If no arguments are specified, we will return the main binary for the configured XUL application.

property includedir
is_clobber_needed()
mozbuild_reader(config_mode='build', vcs_revision=None, vcs_check_clean=True)

Obtain a BuildReader for evaluating moz.build files.

Given arguments, returns a mozbuild.frontend.reader.BuildReader that can be used to evaluate moz.build files for this repo.

config_mode is either build or empty. If build, self.config_environment is used. This requires a configured build system to work. If empty, an empty config is used. empty is appropriate for file-based traversal mode where Files metadata is read.

If vcs_revision is defined, it specifies a version control revision to use to obtain files content. The default is to use the filesystem. This mode is only supported with Mercurial repositories.

If vcs_revision is not defined and the version control checkout is sparse, this implies vcs_revision='.'.

If vcs_revision is . (denotes the parent of the working directory), we will verify that the working directory is clean unless vcs_check_clean is False. This prevents confusion due to uncommitted file changes not being reflected in the reader.

property mozconfig

Returns information about the current mozconfig file.

This a dict as returned by MozconfigLoader.read_mozconfig()

notify(msg)

Show a desktop notification with the supplied message

On Linux and Mac, this will show a desktop notification with the message, but on Windows we can only flash the screen.

property platform

Returns current platform and architecture name

reload_config_environment()

Force config.status to be re-read and return the new value of self.config_environment.

repository

A specialized version of the memoize decorator that works for class instance properties.

resolve_config_guess()
resolve_mozconfig_topobjdir(default=None)
property statedir
property substs
property topobjdir
property virtualenv_manager
exception mozbuild.base.ObjdirMismatchException(objdir1, objdir2)

Bases: BadEnvironmentException

Raised when the current dir is an objdir and doesn’t match the mozconfig.

class mozbuild.base.PathArgument(arg, topsrcdir, topobjdir, cwd=None)

Bases: object

Parse a filesystem path argument and transform it in various ways.

objdir_path()
relpath()

Return a path relative to the topsrcdir or topobjdir.

If the argument is a path to a location in one of the base directories (topsrcdir or topobjdir), then strip off the base directory part and just return the path within the base directory.

srcdir_path()

mozbuild.bootstrap module

mozbuild.bootstrap.bootstrap_all_toolchains_for(configure_args=[])
mozbuild.bootstrap.bootstrap_toolchain(toolchain_job)

mozbuild.build_commands module

mozbuild.build_commands.build(command_context, what=None, jobs=0, job_size=0, directory=None, verbose=False, keep_going=False, priority='idle')

Build the source tree.

With no arguments, this will perform a full build.

Positional arguments define targets to build. These can be make targets or patterns like “<dir>/<target>” to indicate a make target within a directory.

There are a few special targets that can be used to perform a partial build faster than what mach build would perform:

  • binaries - compiles and links all C/C++ sources and produces shared libraries and executables (binaries).

  • faster - builds JavaScript, XUL, CSS, etc files.

“binaries” and “faster” almost fully complement each other. However, there are build actions not captured by either. If things don’t appear to be rebuilding, perform a vanilla mach build to rebuild the world.

mozbuild.build_commands.build_backend(command_context, backend, diff=False, verbose=False, dry_run=False)
mozbuild.build_commands.configure(command_context, options=None, buildstatus_messages=False, line_handler=None)
mozbuild.build_commands.resource_usage(command_context, address=None, port=None, browser=None, url=None)

mozbuild.buildversion module

mozbuild.buildversion.mozilla_build_version()

mozbuild.chunkify module

exception mozbuild.chunkify.ChunkingError

Bases: Exception

mozbuild.chunkify.chunkify(things, this_chunk, chunks)
mozbuild.chunkify.split_evenly(n, chunks)

Split an integer into evenly distributed list

>>> split_evenly(7, 3)
[3, 2, 2]

>>> split_evenly(12, 3)
[4, 4, 4]

>>> split_evenly(35, 10)
[4, 4, 4, 4, 4, 3, 3, 3, 3, 3]

>>> split_evenly(1, 2)
Traceback (most recent call last):
    ...
ChunkingError: Number of chunks is greater than number

mozbuild.config_status module

class mozbuild.config_status.BackendPool(definitions, *, processes=None)

Bases: object

per_process_definitions = None
run(backends)
mozbuild.config_status.config_status(topobjdir='.', topsrcdir='.', defines=None, substs=None, source=None, mozconfig=None, args=['doc', '--upload', '--no-open', '--no-serve', '--write-url', '/builds/worker/firefox-source-docs-url.txt', '--dump-trees=/builds/worker/artifacts/trees.json'])

Main function, providing config.status functionality.

Contrary to config.status, it doesn’t use CONFIG_FILES or CONFIG_HEADERS variables.

Without the -n option, this program acts as config.status and considers the current directory as the top object directory, even when config.status is in a different directory. It will, however, treat the directory containing config.status as the top object directory with the -n option.

The options to this function are passed when creating the ConfigEnvironment. These lists, as well as the actual wrapper script around this function, are meant to be generated by configure. See build/autoconf/config.status.m4.

mozbuild.dirutils module

mozbuild.dirutils.ensureParentDir(path)

Ensures the directory parent to the given file exists.

mozbuild.dirutils.mkdir(path, not_indexed=False)

Ensure a directory exists.

If not_indexed is True, an attribute is set that disables content indexing on the directory.

mozbuild.doctor module

mozbuild.dotproperties module

class mozbuild.dotproperties.DotProperties(file=None)

Bases: object

A thin representation of a key=value .properties file.

get(key, default=None)
get_dict(prefix, required_keys=[])

Turns {‘foo.title’:’title’, …} into {‘title’:’title’, …}.

If |required_keys| is present, it must be an iterable of required key names. If a required key is not present, ValueError is thrown.

Returns {} to indicate an empty or missing dict.

get_list(prefix)

Turns {‘list.0’:’foo’, ‘list.1’:’bar’} into [‘foo’, ‘bar’].

Returns [] to indicate an empty or missing list.

update(file)

Updates properties from a file name or file-like object.

Ignores empty lines and comment lines.

mozbuild.faster_daemon module

Use pywatchman to watch source directories and perform partial mach build faster builds.

class mozbuild.faster_daemon.Daemon(config_environment)

Bases: object

changed_files()
property defines
file_copier

A specialized version of the memoize decorator that works for class instance properties.

incremental_copy(copier, force=False, verbose=True)
input_changes(verbose=True)

Return an iterator of FasterBuildChange instances as inputs to the faster build system change.

output_changes(verbose=True)

Return an iterator of FasterBuildChange instances as outputs from the faster build system are updated.

subscribe_to_dir(name, dir_to_watch)
subscribe_to_topsrcdir()
watch(verbose=True)
class mozbuild.faster_daemon.FasterBuildChange

Bases: object

exception mozbuild.faster_daemon.FasterBuildException(message, cause)

Bases: Exception

mozbuild.faster_daemon.print_copy_result(elapsed, destdir, result, verbose=True)
mozbuild.faster_daemon.print_line(prefix, m, now=None)

mozbuild.gen_test_backend module

mozbuild.gen_test_backend.gen_test_backend()

mozbuild.generated_sources module

mozbuild.generated_sources.get_filename_with_digest(name, contents)

Return the filename that will be used to store the generated file in the S3 bucket, consisting of the SHA-512 digest of contents joined with the relative path name.

mozbuild.generated_sources.get_generated_sources()

Yield tuples of (objdir-rel-path, file) for generated source files in this objdir, where file is either an absolute path to the file or a mozpack.File instance.

mozbuild.generated_sources.get_s3_region_and_bucket()

Return a tuple of (region, bucket) giving the AWS region and S3 bucket to which generated sources should be uploaded.

mozbuild.generated_sources.sha512_digest(data)

Generate the SHA-512 digest of data and return it as a hex string.

mozbuild.gn_processor module

class mozbuild.gn_processor.MozbuildWriter(fh)

Bases: object

finalize()
mb_serialize(v)
terminate_condition()
write(content)
write_attrs(context_attrs)
write_condition(values)
write_ln(line)
write_mozbuild_dict(key, value)
write_mozbuild_list(key, value)
write_mozbuild_value(key, value)
mozbuild.gn_processor.filter_gn_config(path, gn_result, sandbox_vars, input_vars, gn_target)
mozbuild.gn_processor.find_common_attrs(config_attributes)
mozbuild.gn_processor.find_deps(all_targets, target)
mozbuild.gn_processor.generate_gn_config(build_root_dir, target_dir, gn_binary, input_variables, sandbox_variables, gn_target, preprocessor, moz_build_flag)
mozbuild.gn_processor.load_preprocessor(script_name)
mozbuild.gn_processor.main()
mozbuild.gn_processor.process_gn_config(gn_config, topsrcdir, srcdir, non_unified_sources, sandbox_vars, mozilla_flags)
mozbuild.gn_processor.write_mozbuild(topsrcdir, srcdir, non_unified_sources, gn_configs, mozilla_flags, write_mozbuild_variables)

mozbuild.html_build_viewer module

class mozbuild.html_build_viewer.BuildViewerServer(address='localhost', port=0)

Bases: object

add_resource_json_file(key, path)

Register a resource JSON file with the server.

The file will be made available under the name/key specified.

add_resource_json_url(key, url)

Register a resource JSON file at a URL.

run()
property url
class mozbuild.html_build_viewer.HTTPHandler(request, client_address, server)

Bases: BaseHTTPRequestHandler

do_GET()
do_POST()
serve_docroot(root, path)

mozbuild.jar module

jarmaker.py provides a python class to package up chrome content by processing jar.mn files.

See the documentation for jar.mn on MDC for further details on the format.

class mozbuild.jar.JarMaker(outputFormat='flat', useJarfileManifest=True, useChromeManifest=False)

Bases: object

JarMaker reads jar.mn files and process those into jar files or flat directories, along with chrome.manifest files.

class OutputHelper_flat(basepath)

Bases: object

Provide getDestModTime and getOutput for a given flat output directory. The helper method ensureDirFor is used by the symlink subclass.

ensureDirFor(name)
getDestModTime(aPath)
getOutput(name, mode='wb')
class OutputHelper_jar(jarfile)

Bases: object

Provide getDestModTime and getOutput for a given jarfile.

getDestModTime(aPath)
getOutput(name, mode='wb')

Bases: OutputHelper_flat

Subclass of OutputHelper_flat that provides a helper for creating a symlink including creating the parent directories.

finalizeJar(jardir, jarbase, jarname, chromebasepath, register, doZip=True)
Helper method to write out the chrome registration entries to

jarfile.manifest or chrome.manifest, or both.

The actual file processing is done in updateManifest.

generateLocaleDirs(relativesrcdir)
getCommandLineParser()

Get a optparse.OptionParser for jarmaker.

This OptionParser has the options for jarmaker as well as the options for the inner PreProcessor.

makeJar(infile, jardir)

makeJar is the main entry point to JarMaker.

It takes the input file, the output directory, the source dirs and the top source dir as argument, and optionally the l10n dirs.

processJarSection(jarinfo, jardir)

Internal method called by makeJar to actually process a section of a jar.mn file.

updateManifest(manifestPath, chromebasepath, register)

updateManifest replaces the % in the chrome registration entries with the given chrome base path, and updates the given manifest file.

mozbuild.lock module

class mozbuild.lock.LockFile(lockfile)

Bases: object

LockFile is used by the lock_file method to hold the lock.

This object should not be used directly, but only through the lock_file method below.

mozbuild.lock.lock_file(lockfile, max_wait=600)

Create and hold a lockfile of the given name, with the given timeout.

To release the lock, delete the returned object.

mozbuild.mach_commands module

class mozbuild.mach_commands.StoreDebugParamsAndWarnAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: Action

mozbuild.mach_commands.android_gtest(command_context, test_dir, shuffle, gtest_filter, package, adb_path, device_serial, remote_test_root, libxul_path, install)
mozbuild.mach_commands.buildsymbols(command_context)

Produce a package of debug symbols suitable for use with Breakpad.

mozbuild.mach_commands.cargo(command_context, cargo_command, all_crates=None, package=None, jobs=0, verbose=False, message_format_json=False, continue_on_error=False, subcommand_args=[])
mozbuild.mach_commands.cargo_vet(command_context, arguments, stdout=None, env=environ({'SHELL': '/bin/bash', 'MOZ_FETCHES': '[{"artifact": "public/build/node-modules.tar.zst", "extract": true, "task": "KqWt2BxmSjG_MRP94VDhtw"}, {"artifact": "public/build/node.tar.zst", "extract": true, "task": "WvkxU60FS6SsP_xTS5Z8-Q"}, {"artifact": "public/build/python.tar.zst", "extract": true, "task": "T9CF4ZEYSr6VhCN1NV_W0A"}]', 'MOZPYTHONHOME': '/builds/worker/fetches/python', 'GECKO_HEAD_REV': 'e8452cb27c317626557cf1583a9c98ad4926bd60', 'HOSTNAME': 'taskcluster-worker', 'MOZ_AUTOMATION': '1', 'TASKCLUSTER_ROOT_URL': 'https://firefox-ci-tc.services.mozilla.com', 'PWD': '/builds/worker/checkouts/gecko', 'LOGNAME': 'worker', 'GECKO_PATH': '/builds/worker/checkouts/gecko', 'TASKCLUSTER_WORKER_POOL': 'gecko-3/b-linux-medium-gcp', 'TASKCLUSTER_PORT_80_TCP_PORT': '80', 'HOME': '/builds/worker', 'TASKCLUSTER_WORKER_GROUP': 'us-central1-b', 'LANG': 'C', 'TASKCLUSTER_NAME': '/compassionate_babbage/taskcluster', 'MOZ_PYTHON_HOME': '/builds/worker/fetches/python', 'TASKCLUSTER_INSTANCE_TYPE': 'projects/970387039909/machineTypes/c2-standard-8', 'GECKO_BASE_REPOSITORY': 'https://hg.mozilla.org/mozilla-unified', 'TASKCLUSTER_PORT': 'tcp://172.17.0.2:80', 'TASKCLUSTER_WORKER_LOCATION': '{"cloud":"google","region":"us-central1","zone":"us-central1-b"}', 'TERM': 'xterm', 'TASKCLUSTER_PUBLIC_IP': '34.56.16.90', 'HG_STORE_PATH': '/builds/worker/checkouts/hg-store', 'USER': 'worker', 'TASKCLUSTER_PORT_80_TCP_PROTO': 'tcp', 'MOZ_SCM_LEVEL': '3', 'SHLVL': '0', 'TASKCLUSTER_PROXY_URL': 'http://taskcluster', 'GECKO_HEAD_REPOSITORY': 'https://hg.mozilla.org/mozilla-central', 'SCCACHE_DISABLE': '1', 'LC_ALL': 'C', 'MOZ_SOURCE_DOCS_USE_GOOGLE': '1', 'MOZ_FETCHES_DIR': '/builds/worker/fetches', 'PATH': '/builds/worker/checkouts/gecko/node_modules/.bin:/builds/worker/fetches/node/bin:/builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/docs/bin:/builds/worker/fetches/python/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'RUN_ID': '0', 'TASK_ID': 'TPzOIxwqR36r1j2b-Cpovw', 'TASKCLUSTER_WORKER_TYPE': 'b-linux-medium-gcp', 'TASKCLUSTER_PORT_80_TCP_ADDR': '172.17.0.2', 'DEBIAN_FRONTEND': 'noninteractive', 'TASKCLUSTER_PORT_80_TCP': 'tcp://172.17.0.2:80', '_': './mach', 'MACH_MAIN_PID': '26', 'VIRTUAL_ENV': '/builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/docs', 'DOCUTILSCONFIG': '/builds/worker/checkouts/gecko/docs/docutils.conf'}))
mozbuild.mach_commands.clobber(command_context, what, full=False)

Clean up the source and object directories.

Performing builds and running various commands generate various files.

Sometimes it is necessary to clean up these files in order to make things work again. This command can be used to perform that cleanup.

The objdir target removes most files in the current object directory (where build output is stored). Some files (like Visual Studio project files) are not removed by default. If you would like to remove the object directory in its entirety, run with –full.

The python target will clean up Python’s generated files (virtualenvs, “.pyc”, “__pycache__”, etc).

The gradle target will remove the “gradle” subdirectory of the object directory.

By default, the command clobbers the objdir and python targets.

mozbuild.mach_commands.database_path(command_context)
mozbuild.mach_commands.doctor(command_context, fix=False, verbose=False)

Diagnose common build environment problems

mozbuild.mach_commands.environment(command_context, format, output=None, verbose=False)
mozbuild.mach_commands.get_warnings_database(command_context)
mozbuild.mach_commands.gtest(command_context, shuffle, jobs, gtest_filter, list_tests, tbpl_parser, enable_webrender, package, adb_path, device_serial, remote_test_root, libxul_path, no_install, debug, debugger, debugger_args)
mozbuild.mach_commands.handle_log_file(command_context, log_file)
mozbuild.mach_commands.install(command_context, **kwargs)

Install a package.

mozbuild.mach_commands.join_ensure_dir(dir1, dir2)
mozbuild.mach_commands.list_warnings(command_context, directory=None, flags=None, report=None)
mozbuild.mach_commands.package(command_context, verbose=False)

Package the built product for distribution.

mozbuild.mach_commands.package_l10n(command_context, verbose=False, locales=[])
mozbuild.mach_commands.repackage(command_context)

Repackages artifacts into different formats.

This is generally used after packages are signed by the signing scriptworkers in order to bundle things up into shippable formats, such as a .dmg on OSX or an installer exe on Windows.

mozbuild.mach_commands.repackage_deb(command_context, input, output, arch, version, build_number, templates, release_product, release_type)
mozbuild.mach_commands.repackage_deb_l10n(command_context, input_xpi_file, input_tar_file, output, version, build_number, templates, release_product)
mozbuild.mach_commands.repackage_desktop_file(command_context, output, flavor, release_product, release_type)
mozbuild.mach_commands.repackage_dmg(command_context, input, output, attribution_sentinel, compression)
mozbuild.mach_commands.repackage_installer(command_context, tag, setupexe, package, output, package_name, sfx_stub, use_upx)
mozbuild.mach_commands.repackage_mar(command_context, input, mar, output, arch, mar_channel_id)
mozbuild.mach_commands.repackage_msi(command_context, wsx, version, locale, arch, setupexe, candle, light, output)
mozbuild.mach_commands.repackage_msix(command_context, input, version=None, channel=None, distribution_dirs=[], arch=None, identity_name=None, vendor=None, publisher=None, publisher_display_name=None, verbose=False, output=None, makeappx=None, sign=False, unsigned=False)
mozbuild.mach_commands.repackage_pkg(command_context, input, output)
mozbuild.mach_commands.repackage_sign_msix(command_context, input, force=False, verbose=False)
mozbuild.mach_commands.repackage_snap(command_context, snapcraft=None, snap_name=None, upstream_repo=None, upstream_branch=None, output=None, input_pkg=None, tmp_dir=None, clean=False, install=False, dry_run=False)
mozbuild.mach_commands.repackage_snap_install(command_context, snap_file, snap_name, sudo=None)
mozbuild.mach_commands.run(command_context, **kwargs)

Run the compiled program.

mozbuild.mach_commands.setup_install_parser()
mozbuild.mach_commands.setup_run_parser()
mozbuild.mach_commands.show_log(command_context, log_file=None)

Show mach logs If we’re in a terminal context, the log is piped to ‘less’ for more convenient viewing. (https://man7.org/linux/man-pages/man1/less.1.html)

mozbuild.mach_commands.summary(command_context, directory=None, report=None)
mozbuild.mach_commands.watch(command_context, verbose=False)

Watch and re-build (parts of) the source tree.

mozbuild.makeutil module

class mozbuild.makeutil.Makefile

Bases: object

Provides an interface for writing simple makefiles

Instances of this class are created, populated with rules, then written.

add_statement(statement)

Add a raw statement in the makefile. Meant to be used for simple variable assignments.

create_rule(targets=())

Create a new rule in the makefile for the given targets. Returns the corresponding Rule instance.

dump(fh, removal_guard=True)

Dump all the rules to the given file handle. Optionally (and by default), add guard rules for file removals (empty rules for other rules’ dependencies)

class mozbuild.makeutil.Rule(targets=())

Bases: object

Class handling simple rules in the form: target1 target2 … : dep1 dep2 … command1 command2 …

add_commands(commands)

Add commands to the rule.

add_dependencies(deps)

Add dependencies to the rule.

add_targets(targets)

Add additional targets to the rule.

commands()

Return an iterator on the rule commands.

dependencies()

Return an iterator on the rule dependencies.

dump(fh)

Dump the rule to the given file handle.

targets()

Return an iterator on the rule targets.

mozbuild.makeutil.read_dep_makefile(fh)

Read the file handler containing a dep makefile (simple makefile only containing dependencies) and returns an iterator of the corresponding Rules it contains. Ignores removal guard rules.

mozbuild.makeutil.write_dep_makefile(fh, target, deps)

Write a Makefile containing only target’s dependencies to the file handle specified.

mozbuild.mozconfig module

exception mozbuild.mozconfig.MozconfigLoadException(path, message, output=None)

Bases: Exception

Raised when a mozconfig could not be loaded properly.

This typically indicates a malformed or misbehaving mozconfig file.

class mozbuild.mozconfig.MozconfigLoader(topsrcdir)

Bases: object

Handles loading and parsing of mozconfig files.

AUTODETECT = <object object>
ENVIRONMENT_VARIABLES = {'CC', 'CFLAGS', 'CXX', 'CXXFLAGS', 'LDFLAGS', 'MOZ_OBJDIR'}
IGNORE_SHELL_VARIABLES = {'BASH_ARGC', 'BASH_ARGV', 'BASH_ARGV0', '_'}
RE_MAKE_VARIABLE = re.compile('\n        ^\\s*                    # Leading whitespace\n        (?P<var>[a-zA-Z_0-9]+)  # Variable name\n        \\s* [?:]?= \\s*          # Assignment operator surrounded by optional\n             , re.VERBOSE)
read_mozconfig(path=None)

Read the contents of a mozconfig into a data structure.

This takes the path to a mozconfig to load. If the given path is AUTODETECT, will try to find a mozconfig from the environment using find_mozconfig().

mozconfig files are shell scripts. So, we can’t just parse them. Instead, we run the shell script in a wrapper which allows us to record state from execution. Thus, the output from a mozconfig is a friendly static data structure.

mozbuild.mozinfo module

mozbuild.mozinfo.build_dict(config, env=environ({'SHELL': '/bin/bash', 'MOZ_FETCHES': '[{"artifact": "public/build/node-modules.tar.zst", "extract": true, "task": "KqWt2BxmSjG_MRP94VDhtw"}, {"artifact": "public/build/node.tar.zst", "extract": true, "task": "WvkxU60FS6SsP_xTS5Z8-Q"}, {"artifact": "public/build/python.tar.zst", "extract": true, "task": "T9CF4ZEYSr6VhCN1NV_W0A"}]', 'MOZPYTHONHOME': '/builds/worker/fetches/python', 'GECKO_HEAD_REV': 'e8452cb27c317626557cf1583a9c98ad4926bd60', 'HOSTNAME': 'taskcluster-worker', 'MOZ_AUTOMATION': '1', 'TASKCLUSTER_ROOT_URL': 'https://firefox-ci-tc.services.mozilla.com', 'PWD': '/builds/worker/checkouts/gecko', 'LOGNAME': 'worker', 'GECKO_PATH': '/builds/worker/checkouts/gecko', 'TASKCLUSTER_WORKER_POOL': 'gecko-3/b-linux-medium-gcp', 'TASKCLUSTER_PORT_80_TCP_PORT': '80', 'HOME': '/builds/worker', 'TASKCLUSTER_WORKER_GROUP': 'us-central1-b', 'LANG': 'C', 'TASKCLUSTER_NAME': '/compassionate_babbage/taskcluster', 'MOZ_PYTHON_HOME': '/builds/worker/fetches/python', 'TASKCLUSTER_INSTANCE_TYPE': 'projects/970387039909/machineTypes/c2-standard-8', 'GECKO_BASE_REPOSITORY': 'https://hg.mozilla.org/mozilla-unified', 'TASKCLUSTER_PORT': 'tcp://172.17.0.2:80', 'TASKCLUSTER_WORKER_LOCATION': '{"cloud":"google","region":"us-central1","zone":"us-central1-b"}', 'TERM': 'xterm', 'TASKCLUSTER_PUBLIC_IP': '34.56.16.90', 'HG_STORE_PATH': '/builds/worker/checkouts/hg-store', 'USER': 'worker', 'TASKCLUSTER_PORT_80_TCP_PROTO': 'tcp', 'MOZ_SCM_LEVEL': '3', 'SHLVL': '0', 'TASKCLUSTER_PROXY_URL': 'http://taskcluster', 'GECKO_HEAD_REPOSITORY': 'https://hg.mozilla.org/mozilla-central', 'SCCACHE_DISABLE': '1', 'LC_ALL': 'C', 'MOZ_SOURCE_DOCS_USE_GOOGLE': '1', 'MOZ_FETCHES_DIR': '/builds/worker/fetches', 'PATH': '/builds/worker/checkouts/gecko/node_modules/.bin:/builds/worker/fetches/node/bin:/builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/docs/bin:/builds/worker/fetches/python/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'RUN_ID': '0', 'TASK_ID': 'TPzOIxwqR36r1j2b-Cpovw', 'TASKCLUSTER_WORKER_TYPE': 'b-linux-medium-gcp', 'TASKCLUSTER_PORT_80_TCP_ADDR': '172.17.0.2', 'DEBIAN_FRONTEND': 'noninteractive', 'TASKCLUSTER_PORT_80_TCP': 'tcp://172.17.0.2:80', '_': './mach', 'MACH_MAIN_PID': '26', 'VIRTUAL_ENV': '/builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/docs', 'DOCUTILSCONFIG': '/builds/worker/checkouts/gecko/docs/docutils.conf'}))

Build a dict containing data about the build configuration from the environment.

mozbuild.mozinfo.write_mozinfo(file, config, env=environ({'SHELL': '/bin/bash', 'MOZ_FETCHES': '[{"artifact": "public/build/node-modules.tar.zst", "extract": true, "task": "KqWt2BxmSjG_MRP94VDhtw"}, {"artifact": "public/build/node.tar.zst", "extract": true, "task": "WvkxU60FS6SsP_xTS5Z8-Q"}, {"artifact": "public/build/python.tar.zst", "extract": true, "task": "T9CF4ZEYSr6VhCN1NV_W0A"}]', 'MOZPYTHONHOME': '/builds/worker/fetches/python', 'GECKO_HEAD_REV': 'e8452cb27c317626557cf1583a9c98ad4926bd60', 'HOSTNAME': 'taskcluster-worker', 'MOZ_AUTOMATION': '1', 'TASKCLUSTER_ROOT_URL': 'https://firefox-ci-tc.services.mozilla.com', 'PWD': '/builds/worker/checkouts/gecko', 'LOGNAME': 'worker', 'GECKO_PATH': '/builds/worker/checkouts/gecko', 'TASKCLUSTER_WORKER_POOL': 'gecko-3/b-linux-medium-gcp', 'TASKCLUSTER_PORT_80_TCP_PORT': '80', 'HOME': '/builds/worker', 'TASKCLUSTER_WORKER_GROUP': 'us-central1-b', 'LANG': 'C', 'TASKCLUSTER_NAME': '/compassionate_babbage/taskcluster', 'MOZ_PYTHON_HOME': '/builds/worker/fetches/python', 'TASKCLUSTER_INSTANCE_TYPE': 'projects/970387039909/machineTypes/c2-standard-8', 'GECKO_BASE_REPOSITORY': 'https://hg.mozilla.org/mozilla-unified', 'TASKCLUSTER_PORT': 'tcp://172.17.0.2:80', 'TASKCLUSTER_WORKER_LOCATION': '{"cloud":"google","region":"us-central1","zone":"us-central1-b"}', 'TERM': 'xterm', 'TASKCLUSTER_PUBLIC_IP': '34.56.16.90', 'HG_STORE_PATH': '/builds/worker/checkouts/hg-store', 'USER': 'worker', 'TASKCLUSTER_PORT_80_TCP_PROTO': 'tcp', 'MOZ_SCM_LEVEL': '3', 'SHLVL': '0', 'TASKCLUSTER_PROXY_URL': 'http://taskcluster', 'GECKO_HEAD_REPOSITORY': 'https://hg.mozilla.org/mozilla-central', 'SCCACHE_DISABLE': '1', 'LC_ALL': 'C', 'MOZ_SOURCE_DOCS_USE_GOOGLE': '1', 'MOZ_FETCHES_DIR': '/builds/worker/fetches', 'PATH': '/builds/worker/checkouts/gecko/node_modules/.bin:/builds/worker/fetches/node/bin:/builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/docs/bin:/builds/worker/fetches/python/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'RUN_ID': '0', 'TASK_ID': 'TPzOIxwqR36r1j2b-Cpovw', 'TASKCLUSTER_WORKER_TYPE': 'b-linux-medium-gcp', 'TASKCLUSTER_PORT_80_TCP_ADDR': '172.17.0.2', 'DEBIAN_FRONTEND': 'noninteractive', 'TASKCLUSTER_PORT_80_TCP': 'tcp://172.17.0.2:80', '_': './mach', 'MACH_MAIN_PID': '26', 'VIRTUAL_ENV': '/builds/worker/.mozbuild/srcdirs/gecko-8a5b87fe5d69/_virtualenvs/docs', 'DOCUTILSCONFIG': '/builds/worker/checkouts/gecko/docs/docutils.conf'}))

Write JSON data about the configuration specified in config and an environment variable dict to |file|, which may be a filename or file-like object. See build_dict for information about what environment variables are used, and what keys are produced.

mozbuild.nodeutil module

mozbuild.nodeutil.check_executable_version(exe, wrap_call_with_node=False)

Determine the version of a Node executable by invoking it.

May raise subprocess.CalledProcessError or ValueError on failure.

mozbuild.nodeutil.find_executable(name, min_version, use_node_for_version_check=False)
mozbuild.nodeutil.find_node_executable(nodejs_exe=None, min_version=<Version('12.22.12')>)

Find a Node executable from the mozbuild directory.

Returns a tuple containing the the path to an executable binary and a version tuple. Both tuple entries will be None if a Node executable could not be resolved.

mozbuild.nodeutil.find_node_paths()

Determines the possible paths for node executables.

Returns a list of paths, which includes the build state directory.

mozbuild.nodeutil.find_npm_executable(min_version=<Version('6.14.16')>)

Find a Node executable from the mozbuild directory.

Returns a tuple containing the the path to an executable binary and a version tuple. Both tuple entries will be None if a Node executable could not be resolved.

mozbuild.preprocessor module

This is a very primitive line based preprocessor, for times when using a C preprocessor isn’t an option.

It currently supports the following grammar for expressions, whitespace is ignored:

expression :

and_cond ( ‘||’ expression ) ? ;

and_cond:

test ( ‘&&’ and_cond ) ? ;

test:

unary ( ( ‘==’ | ‘!=’ ) unary ) ? ;

unary :

‘!’? value ;

value :

[0-9]+ # integer | ‘defined(’ w+ ‘)’ | w+ # string identifier or value;

class mozbuild.preprocessor.Context

Bases: dict

This class holds variable values by subclassing dict, and while it truthfully reports True and False on

name in context

it returns the variable name itself on

context[“name”]

to reflect the ambiguity between string literals and preprocessor variables.

class mozbuild.preprocessor.Expression(expression_string)

Bases: object

exception ParseError(expression)

Bases: Exception

Error raised when parsing fails. It has two members, offset and content, which give the offset of the error and the offending content.

evaluate(context)

Evaluate the expression with the given context

class mozbuild.preprocessor.Preprocessor(defines=None, marker='#')

Bases: object

Class for preprocessing text files.

exception Error(cpp, MSG, context)

Bases: RuntimeError

addDefines(defines)

Adds the specified defines to the preprocessor. defines may be a dictionary object or an iterable of key/value pairs (as tuples or other iterables of length two)

applyFilters(aLine)
clone()

Create a clone of the current processor, including line ending settings, marker, variable definitions, output stream.

computeDependencies(input)

Reads the input stream, and computes the dependencies for that input.

do_define(args)
do_elif(args)
do_elifdef(args)
do_elifndef(args)
do_else(args, ifState=2)
do_endif(args)
do_error(args)
do_expand(args)
do_filter(args)
do_if(args, replace=False)
do_ifdef(args, replace=False)
do_ifndef(args, replace=False)
do_include(args, filters=True)

Preprocess a given file. args can either be a file name, or a file-like object. Files should be opened, and will be closed after processing.

do_includesubst(args)
do_literal(args)
do_undef(args)
do_unfilter(args)
ensure_not_else()
failUnused(file)
filter_attemptSubstitution(aLine)
filter_dumbComments(aLine)
filter_emptyLines(aLine)
filter_substitution(aLine, fatal=True)
getCommandLineParser(unescapeDefines=False)
handleCommandLine(args, defaultToStdin=False)

Parse a commandline into this parser. Uses OptionParser internally, no args mean sys.argv[1:].

handleLine(aLine)

Handle a single line of input (internal).

noteLineInfo()
processFile(input, output, depfile=None)

Preprocesses the contents of the input stream and writes the result to the output stream. If depfile is set, the dependencies of output file are written to depfile in Makefile format.

setMarker(aMarker)

Set the marker to be used for processing directives. Used for handling CSS files, with pp.setMarker(‘%’), for example. The given marker may be None, in which case no markers are processed.

setSilenceDirectiveWarnings(value)

Sets whether missing directive warnings are silenced, according to value. The default behavior of the preprocessor is to emit such warnings.

write(aLine)

Internal method for handling output.

mozbuild.preprocessor.preprocess(includes=[<_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>], defines={}, output=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, marker='#')

mozbuild.pythonutil module

mozbuild.pythonutil.iter_modules_in_path(*paths)

mozbuild.schedules module

Constants for SCHEDULES configuration in moz.build files and for skip-unless-schedules optimizations in task-graph generation.

mozbuild.shellutil module

exception mozbuild.shellutil.MetaCharacterException(char)

Bases: Exception

mozbuild.shellutil.quote(*strings)

Given one or more strings, returns a quoted string that can be used literally on a shell command line.

>>> quote('a', 'b')
"a b"
>>> quote('a b', 'c')
"'a b' c"
mozbuild.shellutil.split(cline)

Split the given command line string.

mozbuild.sphinx module

class mozbuild.sphinx.MozbuildSymbols(name, arguments, options, content, lineno, content_offset, block_text, state, state_machine)

Bases: Directive

Directive to insert mozbuild sandbox symbol information.

required_arguments = 1

Number of required directive arguments.

run()
class mozbuild.sphinx.Searchfox

Bases: ReferenceRole

Role which links a relative path from the source to it’s searchfox URL.

Can be used like:

Will generate a link to https://searchfox.org/mozilla-central/source/browser/base/content/browser-places.js

The example above will use the path as the text, to use custom text:

See this file for more details.

To specify a different source tree:

run()
mozbuild.sphinx.export_mots(config_path)

Load mots configuration and export it to file.

mozbuild.sphinx.find_mots_config_path(app)

Find and return mots config path if it exists.

mozbuild.sphinx.format_module(m)
mozbuild.sphinx.function_reference(f, attr, args, doc)
mozbuild.sphinx.setup(app)
mozbuild.sphinx.special_reference(v, func, typ, doc)
mozbuild.sphinx.variable_reference(v, st_type, in_type, doc)

mozbuild.telemetry module

This file contains functions used for telemetry.

mozbuild.telemetry.cpu_brand_linux()

Read the CPU brand string out of /proc/cpuinfo on Linux.

mozbuild.telemetry.cpu_brand_mac()

Get the CPU brand string via sysctl on macos.

mozbuild.telemetry.cpu_brand_windows()

Read the CPU brand string from the registry on Windows.

mozbuild.telemetry.filter_args(command, argv, topsrcdir, topobjdir, cwd=None)

Given the full list of command-line arguments, remove anything up to and including command, and attempt to filter absolute pathnames out of any arguments after that.

mozbuild.telemetry.get_cpu_brand()

Get the CPU brand string as returned by CPUID.

mozbuild.telemetry.get_distro_and_version()
mozbuild.telemetry.get_psutil_stats()

Return whether psutil exists and its associated stats.

@returns (bool, int, int, int) whether psutil exists, the logical CPU count,

physical CPU count, and total number of bytes of memory.

mozbuild.telemetry.get_shell_info()

Returns if the current shell was opened by vscode and if it’s a SSH connection

mozbuild.telemetry.get_vscode_running()

Return if the vscode is currently running.

mozbuild.testing module

class mozbuild.testing.SupportFilesConverter

Bases: object

Processes a “support-files” entry from a test object, either from a parsed object from a test manifests or its representation in moz.build and returns the installs to perform for this test object.

Processing the same support files multiple times will not have any further effect, and the structure of the parsed objects from manifests will have a lot of repeated entries, so this class takes care of memoizing.

convert_support_files(test, install_root, manifest_dir, out_dir)
class mozbuild.testing.TestInstallInfo

Bases: object

mozbuild.testing.all_test_flavors()
mozbuild.testing.install_test_files(topsrcdir, topobjdir, tests_root)

Installs the requested test files to the objdir. This is invoked by test runners to avoid installing tens of thousands of test files when only a few tests need to be run.

mozbuild.testing.read_manifestparser_manifest(context, manifest_path)
mozbuild.testing.read_reftest_manifest(context, manifest_path)
mozbuild.testing.read_wpt_manifest(context, paths)

mozbuild.toolchains module

mozbuild.toolchains.toolchain_task_definitions()

mozbuild.util module

class mozbuild.util.DefinesAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)

Bases: Action

An ArgumentParser action to handle -Dvar[=value] type of arguments.

class mozbuild.util.EmptyValue

Bases: str

A dummy type that behaves like an empty string and sequence.

This type exists in order to support mozbuild.frontend.reader.EmptyConfig. It should likely not be used elsewhere.

class mozbuild.util.EnumString(value)

Bases: str

A string type that only can have a limited set of values, similarly to an Enum, and can only be compared against that set of values.

The class is meant to be subclassed, where the subclass defines POSSIBLE_VALUES. The subclass method is a helper to create such subclasses.

POSSIBLE_VALUES = ()
exception mozbuild.util.EnumStringComparisonError

Bases: Exception

class mozbuild.util.FileAvoidWrite(filename, capture_diff=False, dry_run=False, readmode='r')

Bases: BytesIO

File-like object that buffers output and only writes if content changed.

We create an instance from an existing filename. New content is written to it. When we close the file object, if the content in the in-memory buffer differs from what is on disk, then we write out the new content. Otherwise, the original file is untouched.

Instances can optionally capture diffs of file changes. This feature is not enabled by default because it a) doesn’t make sense for binary files b) could add unwanted overhead to calls.

Additionally, there is dry run mode where the file is not actually written out, but reports whether the file was existing and would have been updated still occur, as well as diff capture if requested.

avoid_writing_to_file()
close()

Stop accepting writes, compare file contents, and rewrite if needed.

Returns a tuple of bools indicating what action was performed:

(file existed, file updated)

If capture_diff was specified at construction time and the underlying file was changed, .diff will be populated with the diff of the result.

write(buf)

Write bytes to file.

Return the number of bytes written.

mozbuild.util.FlagsFactory(flags)

Returns a class which holds optional flags for an item in a list.

The flags are defined in the dict given as argument, where keys are the flag names, and values the type used for the value of that flag.

The resulting class is used by the various <TypeName>WithFlagsFactory functions below.

class mozbuild.util.HierarchicalStringList

Bases: object

A hierarchy of lists of strings.

Each instance of this object contains a list of strings, which can be set or appended to. A sub-level of the hierarchy is also an instance of this class, can be added by appending to an attribute instead.

For example, the moz.build variable EXPORTS is an instance of this class. We can do:

EXPORTS += [‘foo.h’] EXPORTS.mozilla.dom += [‘bar.h’]

In this case, we have 3 instances (EXPORTS, EXPORTS.mozilla, and EXPORTS.mozilla.dom), and the first and last each have one element in their list.

class StringListAdaptor(hsl)

Bases: Sequence

walk()

Walk over all HierarchicalStringLists in the hierarchy.

This is a generator of (path, sequence).

The path is ‘’ for the root level and ‘/’-delimited strings for any descendants. The sequence is a read-only sequence of the strings contained at that level.

class mozbuild.util.ImmutableStrictOrderingOnAppendList(iterable=None, **kwargs)

Bases: StrictOrderingOnAppendList

Like StrictOrderingOnAppendList, but not allowing mutations of the value.

append(elt)

Append object to the end of the list.

extend(iterable)

Extend list by appending elements from the iterable.

class mozbuild.util.KeyedDefaultDict(default_factory, *args, **kwargs)

Bases: dict

Like a defaultdict, but the default_factory function takes the key as argument

class mozbuild.util.List(iterable=None, **kwargs)

Bases: list

A list specialized for moz.build environments.

We overload the assignment and append operations to require that the appended thing is a list. This avoids bad surprises coming from appending a string to a list, which would just add each letter of the string.

extend(l)

Extend list by appending elements from the iterable.

exception mozbuild.util.MozbuildDeletionError

Bases: Exception

class mozbuild.util.ReadOnlyDefaultDict(default_factory, *args, **kwargs)

Bases: ReadOnlyDict

A read-only dictionary that supports default values on retrieval.

class mozbuild.util.ReadOnlyDict(*args, **kwargs)

Bases: dict

A read-only dictionary.

update([E, ]**F) → None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

class mozbuild.util.ReadOnlyKeyedDefaultDict(default_factory, *args, **kwargs)

Bases: KeyedDefaultDict, ReadOnlyDict

Like KeyedDefaultDict, but read-only.

class mozbuild.util.ReadOnlyNamespace(**kwargs)

Bases: object

A class for objects with immutable attributes set at initialization.

class mozbuild.util.StrictOrderingOnAppendList(iterable=None, **kwargs)

Bases: List

A list specialized for moz.build environments.

We overload the assignment and append operations to require that incoming elements be ordered. This enforces cleaner style in moz.build files.

static ensure_sorted(l)
extend(l)

Extend list by appending elements from the iterable.

class mozbuild.util.StrictOrderingOnAppendListWithAction(iterable=(), action=None)

Bases: StrictOrderingOnAppendList

An ordered list that accepts a callable to be applied to each item.

A callable (action) passed to the constructor is run on each item of input. The result of running the callable on each item will be stored in place of the original input, but the original item must be used to enforce sortedness.

extend(l)

Extend list by appending elements from the iterable.

class mozbuild.util.StrictOrderingOnAppendListWithFlags(iterable=None, **kwargs)

Bases: StrictOrderingOnAppendList

A list with flags specialized for moz.build environments.

Each subclass has a set of typed flags; this class lets us use isinstance for natural testing.

mozbuild.util.StrictOrderingOnAppendListWithFlagsFactory(flags)

Returns a StrictOrderingOnAppendList-like object, with optional flags on each item.

The flags are defined in the dict given as argument, where keys are the flag names, and values the type used for the value of that flag.

Example:

FooList = StrictOrderingOnAppendListWithFlagsFactory({
    'foo': bool, 'bar': unicode
})
foo = FooList(['a', 'b', 'c'])
foo['a'].foo = True
foo['b'].bar = 'bar'
mozbuild.util.TypedList(type, base_class=<class 'mozbuild.util.List'>)

A list with type coercion.

The given type is what list elements are being coerced to. It may do strict validation, throwing ValueError exceptions.

A base_class type can be given for more specific uses than a List. For example, a Typed StrictOrderingOnAppendList can be created with:

TypedList(unicode, StrictOrderingOnAppendList)

mozbuild.util.TypedNamedTuple(name, fields)

Factory for named tuple types with strong typing.

Arguments are an iterable of 2-tuples. The first member is the the field name. The second member is a type the field will be validated to be.

Construction of instances varies from collections.namedtuple.

First, if a single tuple argument is given to the constructor, this is treated as the equivalent of passing each tuple value as a separate argument into __init__. e.g.:

t = (1, 2)
TypedTuple(t) == TypedTuple(1, 2)

This behavior is meant for moz.build files, so vanilla tuples are automatically cast to typed tuple instances.

Second, fields in the tuple are validated to be instances of the specified type. This is done via an isinstance() check. To allow multiple types, pass a tuple as the allowed types field.

exception mozbuild.util.UnsortedError(srtd, original)

Bases: Exception

mozbuild.util.cpu_count()

Returns the number of CPUs available to us. This may be different than os.cpu_count() because of affinity.

See the Python documentation for os.cpu_count().

mozbuild.util.ensure_bytes(value, encoding='utf-8')
mozbuild.util.ensure_unicode(value, encoding='utf-8')
mozbuild.util.expand_variables(s, variables)

Given a string with $(var) variable references, replace those references with the corresponding entries from the given variables dict.

If a variable value is not a string, it is iterated and its items are joined with a whitespace.

mozbuild.util.group_unified_files(files, unified_prefix, unified_suffix, files_per_unified_file)

Return an iterator of (unified_filename, source_filenames) tuples.

We compile most C and C++ files in “unified mode”; instead of compiling a.cpp, b.cpp, and c.cpp separately, we compile a single file that looks approximately like:

#include "a.cpp"
#include "b.cpp"
#include "c.cpp"

This function handles the details of generating names for the unified files, and determining which original source files go in which unified file.

mozbuild.util.hash_file(path, hasher=None)

Hashes a file specified by the path given and returns the hex digest.

mozbuild.util.hexdump(buf)

Returns a list of hexdump-like lines corresponding to the given input buffer.

mozbuild.util.macos_performance_cores()

Returns the number of performance cores on Mac OS

See the Python documentation for os.cpu_count().

class mozbuild.util.memoize(func)

Bases: dict

A decorator to memoize the results of function calls depending on its arguments. Both functions and instance methods are handled, although in the instance method case, the results are cache in the instance itself.

method_call(instance, *args)
class mozbuild.util.memoized_property(func)

Bases: object

A specialized version of the memoize decorator that works for class instance properties.

mozbuild.util.pair(iterable)

Given an iterable, returns an iterable pairing its items.

For example,

list(pair([1,2,3,4,5,6]))

returns

[(1,2), (3,4), (5,6)]

mozbuild.util.pairwise(iterable)

Given an iterable, returns an iterable of overlapped pairs of its items. Based on the Python itertools documentation.

For example,

list(pairwise([1,2,3,4,5,6]))

returns

[(1,2), (2,3), (3,4), (4,5), (5,6)]

mozbuild.util.resolve_target_to_make(topobjdir, target)

Resolve target (a target, directory, or file) to a make target.

topobjdir is the object directory; all make targets will be rooted at or below the top-level Makefile in this directory.

Returns a pair (reldir, target) where reldir is a directory relative to topobjdir containing a Makefile and target is a make target (possibly None).

A directory resolves to the nearest directory at or above containing a Makefile, and target None.

A regular (non-Makefile) file resolves to the nearest directory at or above the file containing a Makefile, and an appropriate target.

A Makefile resolves to the nearest parent strictly above the Makefile containing a different Makefile, and an appropriate target.

mozbuild.util.simple_diff(filename, old_lines, new_lines)

Returns the diff between old_lines and new_lines, in unified diff form, as a list of lines.

old_lines and new_lines are lists of non-newline terminated lines to compare. old_lines can be None, indicating a file creation. new_lines can be None, indicating a file deletion.

class mozbuild.util.undefined_default

Bases: object

Represents an undefined argument value that isn’t None.

Module contents