mozbuild.action package

Submodules

mozbuild.action.buildlist module

A generic script to add entries to a file if the entry does not already exist.

Usage: buildlist.py <filename> <entry> [<entry> …]

mozbuild.action.buildlist.addEntriesToListFile(listFile, entries)

Given a file listFile containing one entry per line, add each entry in entries to the file, unless it is already present.

mozbuild.action.buildlist.main(args)

mozbuild.action.check_binary module

mozbuild.action.download_wpt_manifest module

mozbuild.action.download_wpt_manifest.main()

mozbuild.action.dump_env module

mozbuild.action.dump_env.environ()

mozbuild.action.dumpsymbols module

mozbuild.action.dumpsymbols.dump_symbols(target, tracking_file, count_ctors=False)
mozbuild.action.dumpsymbols.main(argv)

mozbuild.action.exe_7z_archive module

mozbuild.action.exe_7z_archive.archive_exe(pkg_dir, tagfile, sfx_package, package, use_upx)
mozbuild.action.exe_7z_archive.main(args)

mozbuild.action.fat_aar module

Fetch and unpack architecture-specific Maven zips, verify cross-architecture compatibility, and ready inputs to an Android multi-architecture fat AAR build.

mozbuild.action.fat_aar.fat_aar(distdir, aars_paths, no_process=False, no_compatibility_check=False)
mozbuild.action.fat_aar.main(argv)

mozbuild.action.file_generate module

mozbuild.action.file_generate.main(argv)

mozbuild.action.file_generate_wrapper module

mozbuild.action.file_generate_wrapper.action(fh, script, target_dir, *args)

mozbuild.action.generate_symbols_file module

mozbuild.action.generate_symbols_file.generate_symbols_file(output, *args)

mozbuild.action.html_fragment_preprocesor module

mozbuild.action.html_fragment_preprocesor.fill_html_fragments_map(fragment_map, path, template, doctype=None)
mozbuild.action.html_fragment_preprocesor.generate(output, *inputs)

Builds an html fragments loader JS file from the input xml file(s)

The xml files are expected to be in the format of: <template>…xhtml markup…</template>

or <template><template name=”fragment_name”>…xhtml markup…</template>…</template> Where there are multiple templates. All markup is expected to be properly namespaced.

In the JS file, calling getHTMLFragment(key) will return the HTML string from the xml file that matches the key.

The key format is filename_without_extension/template_name for files with multiple templates, or just filename_without_extension for files with one template. filename_without_extension is the xml filename without the .xml extension and template_name is the name attribute of template node containing the xml fragment.

Arguments: output – File handle to JS file being generated inputs – list of xml filenames to include in loader

Returns: The set of dependencies which should trigger this command to be re-run. This is ultimately returned to the build system for use by the backend to ensure that incremental rebuilds happen when any dependency changes.

mozbuild.action.html_fragment_preprocesor.get_fragment_key(path, template_name=None)
mozbuild.action.html_fragment_preprocesor.get_html_fragments_from_file(fragment_map, path)

mozbuild.action.install module

mozbuild.action.install.main(args)

mozbuild.action.jar_maker module

mozbuild.action.jar_maker.main(args)

mozbuild.action.l10n_merge module

mozbuild.action.l10n_merge.main(argv)

mozbuild.action.langpack_manifest module

mozbuild.action.langpack_manifest.convert_entry_flags_to_platform_codes(flags)
mozbuild.action.langpack_manifest.create_webmanifest(locstr, version, min_app_ver, max_app_ver, app_name, l10n_basedir, langpack_eid, ftl, chrome_entries)
mozbuild.action.langpack_manifest.get_author(ftl)
mozbuild.action.langpack_manifest.get_build_date()

Return the current date or SOURCE_DATE_EPOCH, if set.

mozbuild.action.langpack_manifest.get_dt_from_hg(path)
mozbuild.action.langpack_manifest.get_timestamp_for_locale(path)
mozbuild.action.langpack_manifest.get_title_and_description(app, locale)
mozbuild.action.langpack_manifest.get_version_maybe_buildid(app_version)
mozbuild.action.langpack_manifest.main(args)
mozbuild.action.langpack_manifest.parse_chrome_manifest(path, base_path, chrome_entries)
mozbuild.action.langpack_manifest.parse_flat_ftl(path)
mozbuild.action.langpack_manifest.write_file(path, content)

mozbuild.action.make_dmg module

mozbuild.action.make_dmg.main(args)

mozbuild.action.make_unzip module

mozbuild.action.make_unzip.main(args)
mozbuild.action.make_unzip.make_unzip(package)

mozbuild.action.node module

mozbuild.action.node.execute_node_cmd(node_cmd_list)

Execute the given node command list.

Arguments: node_cmd_list – a list of the command and arguments to be executed

Returns: The set of dependencies which should trigger this command to be re-run. This is ultimately returned to the build system for use by the backend to ensure that incremental rebuilds happen when any dependency changes.

The node script is expected to output lines for all of the dependencies to stdout, each prefixed by the string “dep:”. These lines will make up the returned set of dependencies. Any line not so-prefixed will simply be printed to stderr instead.

mozbuild.action.node.generate(output, node_script, *files)

Call the given node_script to transform the given modules.

Arguments: output – a dummy file, used by the build system. Can be ignored. node_script – the script to be executed. Must be in the SCRIPT_ALLOWLIST files – files to be transformed, will be passed to the script as arguments

Returns: The set of dependencies which should trigger this command to be re-run. This is ultimately returned to the build system for use by the backend to ensure that incremental rebuilds happen when any dependency changes.

mozbuild.action.node.is_script_in_allowlist(script_path)

mozbuild.action.package_generated_sources module

mozbuild.action.package_generated_sources.main(argv)

mozbuild.action.preprocessor module

mozbuild.action.preprocessor.generate(output, *args)
mozbuild.action.preprocessor.main(args)

mozbuild.action.process_define_files module

mozbuild.action.process_define_files.main(argv)
mozbuild.action.process_define_files.process_define_file(output, input)

Creates the given config header. A config header is generated by taking the corresponding source file and replacing some #define/#undef occurences:

  • “#undef NAME” is turned into “#define NAME VALUE”

  • “#define NAME” is unchanged

  • “#define NAME ORIGINAL_VALUE” is turned into “#define NAME VALUE”

  • “#undef UNKNOWN_NAME” is turned into “/* #undef UNKNOWN_NAME */”

  • Whitespaces are preserved.

As a special rule, “#undef ALLDEFINES” is turned into “#define NAME VALUE” for all the defined variables.

mozbuild.action.process_install_manifest module

mozbuild.action.process_install_manifest.main(argv)
mozbuild.action.process_install_manifest.process_manifest(destdir, paths, track, no_symlinks=False, defines={})

mozbuild.action.symbols_archive module

mozbuild.action.symbols_archive.main(argv)
mozbuild.action.symbols_archive.make_archive(archive_name, base, exclude, include)

mozbuild.action.test_archive module

mozbuild.action.tooltool module

exception mozbuild.action.tooltool.BadFilenameException(filename)

Bases: ExceptionWithFilename

exception mozbuild.action.tooltool.BadHeaderValue

Bases: Exception

exception mozbuild.action.tooltool.DigestMismatchException(filename)

Bases: ExceptionWithFilename

exception mozbuild.action.tooltool.ExceptionWithFilename(filename)

Bases: Exception

class mozbuild.action.tooltool.FileRecord(filename, size, digest, algorithm, unpack=False, version=None, visibility=None)

Bases: object

describe()
present()
validate()
validate_digest()
validate_size()
class mozbuild.action.tooltool.FileRecordJSONDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)

Bases: JSONDecoder

I help the json module materialize a FileRecord from a JSON file. I understand FileRecords and lists of FileRecords. I ignore things that I don’t expect for now

decode(s)

Return the Python representation of s (a str instance containing a JSON document).

process_file_records(obj)
class mozbuild.action.tooltool.FileRecordJSONEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: JSONEncoder

default(f)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)
encode_file_record(obj)
exception mozbuild.action.tooltool.FileRecordJSONEncoderException

Bases: Exception

exception mozbuild.action.tooltool.InvalidCredentials

Bases: Exception

exception mozbuild.action.tooltool.InvalidManifest

Bases: Exception

class mozbuild.action.tooltool.Manifest(file_records=None)

Bases: object

copy()
dump(output_file, fmt='json')
dumps(fmt='json')
load(data_file, fmt='json')
loads(data_string, fmt='json')
present()
valid_formats = ('json',)
validate()
validate_digests()
validate_sizes()
exception mozbuild.action.tooltool.MissingFileException(filename)

Bases: ExceptionWithFilename

mozbuild.action.tooltool.add_files(manifest_file, algorithm, filenames, version, visibility, unpack)
mozbuild.action.tooltool.calculate_mac(mac_type, access_token, algorithm, timestamp, nonce, method, name, host, port, content_hash)
mozbuild.action.tooltool.calculate_payload_hash(algorithm, payload, content_type)
mozbuild.action.tooltool.change_visibility(base_urls, digest, visibility, auth_file)
mozbuild.action.tooltool.clean_path(dirname)

Remove a subtree if is exists. Helper for unpack_file().

mozbuild.action.tooltool.create_file_record(filename, algorithm)
mozbuild.action.tooltool.delete_instances(base_urls, digest, auth_file)
mozbuild.action.tooltool.digest_file(f, a)

I take a file like object ‘f’ and return a hex-string containing of the result of the algorithm ‘a’ applied to ‘f’.

mozbuild.action.tooltool.execute(cmd)

Execute CMD, logging its stdout at the info level

mozbuild.action.tooltool.fetch_file(base_urls, file_record, grabchunk=4096, auth_file=None, region=None)
mozbuild.action.tooltool.fetch_files(manifest_file, base_urls, filenames=[], cache_folder=None, auth_file=None, region=None)
mozbuild.action.tooltool.freespace(p)

Returns the number of bytes free under directory p

mozbuild.action.tooltool.get_hexdigest(val)
mozbuild.action.tooltool.list_manifest(manifest_file)

I know how print all the files in a location

mozbuild.action.tooltool.main(argv, _skip_logging=False)
mozbuild.action.tooltool.make_taskcluster_header(credentials, req)
mozbuild.action.tooltool.normalize_header_attr(val)
mozbuild.action.tooltool.normalize_string(mac_type, timestamp, nonce, method, name, host, port, content_hash)
mozbuild.action.tooltool.open_manifest(manifest_file)

I know how to take a filename and load it into a Manifest object

mozbuild.action.tooltool.parse_content_type(content_type)
mozbuild.action.tooltool.parse_url(url)
mozbuild.action.tooltool.prepare_header_val(val)
mozbuild.action.tooltool.process_command(options, args)

I know how to take a list of program arguments and start doing the right thing with them

mozbuild.action.tooltool.purge(folder, gigs)

If gigs is non 0, it deletes files in folder until gigs GB are free, starting from older files. If gigs is 0, a full purge will be performed. No recursive deletion of files in subfolder is performed.

mozbuild.action.tooltool.random() x in the interval [0, 1).
mozbuild.action.tooltool.random_string(length)
mozbuild.action.tooltool.request(url, auth_file=None)
mozbuild.action.tooltool.request_has_data(req)
mozbuild.action.tooltool.retriable(*retry_args, **retry_kwargs)

This function originates from redo 2.0.3 https://github.com/mozilla-releng/redo A decorator factory for retry(). Wrap your function in @retriable(…) to give it retry powers!

mozbuild.action.tooltool.retrier(attempts=5, sleeptime=10, max_sleeptime=300, sleepscale=1.5, jitter=1)

This function originates from redo 2.0.3 https://github.com/mozilla-releng/redo A generator function that sleeps between retries, handles exponential backoff and jitter. The action you are retrying is meant to run after retrier yields.

mozbuild.action.tooltool.retry(action, attempts=5, sleeptime=60, max_sleeptime=300, sleepscale=1.5, jitter=1, retry_exceptions=(<class 'Exception'>, ), cleanup=None, args=(), kwargs={}, log_args=True)

This function originates from redo 2.0.3 https://github.com/mozilla-releng/redo Calls an action function until it succeeds, or we give up.

mozbuild.action.tooltool.safe_extract(tar, path='.', *, numeric_owner=False)
mozbuild.action.tooltool.send_operation_on_file(data, base_urls, digest, auth_file)
mozbuild.action.tooltool.touch(f)

Used to modify mtime in cached files; mtime is used by the purge command

mozbuild.action.tooltool.unpack_file(filename)

Untar filename, assuming it is uncompressed or compressed with bzip2, xz, gzip, zst, or unzip a zip file. The file is assumed to contain a single directory with a name matching the base of the given filename. Xz support is handled by shelling out to ‘tar’.

mozbuild.action.tooltool.upload(manifest, message, base_urls, auth_file, region)
mozbuild.action.tooltool.utc_now(offset_in_seconds=0.0)
mozbuild.action.tooltool.validate_manifest(manifest_file)

I validate that all files in a manifest are present and valid but don’t fetch or delete them if they aren’t

mozbuild.action.tooltool.validate_tar_member(member, path)
mozbuild.action.tooltool.validate_taskcluster_credentials(credentials)

mozbuild.action.unify_symbols module

class mozbuild.action.unify_symbols.UnifiedSymbolsFinder(finder1, finder2, sorted=[], **kargs)

Bases: UnifiedFinder

unify_file(path, file1, file2)

Given two BaseFiles and the path they were found at, return a unified version of the files. If the files match, the first BaseFile may be returned. If the files don’t match or one of them is None, the method returns None. Subclasses may decide to unify by using one of the files in that case.

mozbuild.action.unify_symbols.main()

mozbuild.action.unify_tests module

class mozbuild.action.unify_tests.UnifiedTestFinder(finder1, finder2, sorted=[], **kargs)

Bases: UnifiedFinder

unify_file(path, file1, file2)

Given two BaseFiles and the path they were found at, return a unified version of the files. If the files match, the first BaseFile may be returned. If the files don’t match or one of them is None, the method returns None. Subclasses may decide to unify by using one of the files in that case.

mozbuild.action.unify_tests.main()

mozbuild.action.unpack_dmg module

mozbuild.action.unpack_dmg.main(args)

mozbuild.action.webidl module

mozbuild.action.webidl.main(argv)

Perform WebIDL code generation required by the build system.

mozbuild.action.wrap_rustc module

mozbuild.action.wrap_rustc.parse_outputs(crate_output, dep_outputs, pass_l_flag)
mozbuild.action.wrap_rustc.wrap_rustc(args)

mozbuild.action.xpccheck module

A generic script to verify all test files are in the corresponding .toml file.

Usage: xpccheck.py <directory> [<directory> …]

mozbuild.action.xpccheck.getIniTests(testdir)
mozbuild.action.xpccheck.main(argv)
mozbuild.action.xpccheck.verifyDirectory(initests, directory)
mozbuild.action.xpccheck.verifyIniFile(initests, directory)

mozbuild.action.xpidl-process module

mozbuild.action.zip module

mozbuild.action.zip.main(args)

Module contents