mozbuild.action package

Submodules

mozbuild.action.buildlist module

A generic script to add entries to a file if the entry does not already exist.

Usage: buildlist.py <filename> <entry> [<entry> …]

mozbuild.action.buildlist.addEntriesToListFile(listFile, entries): Given a file listFile containing one entry per line, add each entry in entries to the file, unless it is already present.

mozbuild.action.buildlist.main(args)

mozbuild.action.check_binary module

mozbuild.action.download_wpt_manifest module

mozbuild.action.download_wpt_manifest.main()

mozbuild.action.dump_env module

mozbuild.action.dumpsymbols module

mozbuild.action.dumpsymbols.dump_symbols(target, tracking_file, count_ctors=False)

mozbuild.action.dumpsymbols.main(argv)

mozbuild.action.exe_7z_archive module

mozbuild.action.exe_7z_archive.archive_exe(pkg_dir, tagfile, sfx_package, package, use_upx)

mozbuild.action.exe_7z_archive.main(args)

mozbuild.action.fat_aar module

Fetch and unpack architecture-specific Maven zips, verify cross-architecture compatibility, and ready inputs to an Android multi-architecture fat AAR build.

mozbuild.action.fat_aar.fat_aar(distdir, aars_paths, no_process=False, no_compatibility_check=False)

mozbuild.action.fat_aar.main(argv)

mozbuild.action.file_generate module

mozbuild.action.file_generate.main(argv)

mozbuild.action.file_generate_wrapper module

mozbuild.action.file_generate_wrapper.action(fh, script, target_dir, *args)

mozbuild.action.generate_symbols_file module

mozbuild.action.generate_symbols_file.generate_symbols_file(output, *args)

mozbuild.action.html_fragment_preprocesor module

mozbuild.action.html_fragment_preprocesor.fill_html_fragments_map(fragment_map, path, template, doctype=None)

mozbuild.action.html_fragment_preprocesor.generate(output, *inputs)

Builds an html fragments loader JS file from the input xml file(s)

The xml files are expected to be in the format of: <template>…xhtml markup…</template>

or <template><template name=”fragment_name”>…xhtml markup…</template>…</template> Where there are multiple templates. All markup is expected to be properly namespaced.

In the JS file, calling getHTMLFragment(key) will return the HTML string from the xml file that matches the key.

The key format is filename_without_extension/template_name for files with multiple templates, or just filename_without_extension for files with one template. filename_without_extension is the xml filename without the .xml extension and template_name is the name attribute of template node containing the xml fragment.

Arguments: output – File handle to JS file being generated inputs – list of xml filenames to include in loader

Returns: The set of dependencies which should trigger this command to be re-run. This is ultimately returned to the build system for use by the backend to ensure that incremental rebuilds happen when any dependency changes.

mozbuild.action.html_fragment_preprocesor.get_fragment_key(path, template_name=None)

mozbuild.action.html_fragment_preprocesor.get_html_fragments_from_file(fragment_map, path)

mozbuild.action.install module

mozbuild.action.install.main(args)

mozbuild.action.jar_maker module

mozbuild.action.jar_maker.main(args)

mozbuild.action.langpack_manifest module

mozbuild.action.langpack_manifest.convert_entry_flags_to_platform_codes(flags)

mozbuild.action.langpack_manifest.create_webmanifest(locstr, version, min_app_ver, max_app_ver, app_name, l10n_basedir, langpack_eid, ftl, chrome_entries)

mozbuild.action.langpack_manifest.get_author(ftl)

mozbuild.action.langpack_manifest.get_build_date(): Return the current date or SOURCE_DATE_EPOCH, if set.

mozbuild.action.langpack_manifest.get_dt_from_hg(path)

mozbuild.action.langpack_manifest.get_timestamp_for_locale(path)

mozbuild.action.langpack_manifest.get_title_and_description(app, locale)

mozbuild.action.langpack_manifest.get_version_maybe_buildid(app_version)

mozbuild.action.langpack_manifest.main(args)

mozbuild.action.langpack_manifest.parse_chrome_manifest(path, base_path, chrome_entries)

mozbuild.action.langpack_manifest.parse_flat_ftl(path)

mozbuild.action.langpack_manifest.write_file(path, content)

mozbuild.action.make_dmg module

mozbuild.action.make_dmg.main(args)

mozbuild.action.make_unzip module

mozbuild.action.make_unzip.main(args)

mozbuild.action.make_unzip.make_unzip(package)

mozbuild.action.node module

mozbuild.action.node.execute_node_cmd(node_cmd_list)

Execute the given node command list.

Arguments: node_cmd_list – a list of the command and arguments to be executed

Returns: The set of dependencies which should trigger this command to be re-run. This is ultimately returned to the build system for use by the backend to ensure that incremental rebuilds happen when any dependency changes.

The node script is expected to output lines for all of the dependencies to stdout, each prefixed by the string “dep:”. These lines will make up the returned set of dependencies. Any line not so-prefixed will simply be printed to stderr instead.

mozbuild.action.node.generate(output, node_script, *files)

Call the given node_script to transform the given modules.

Arguments: output – a dummy file, used by the build system. Can be ignored. node_script – the script to be executed. Must be in the SCRIPT_ALLOWLIST files – files to be transformed, will be passed to the script as arguments

Returns: The set of dependencies which should trigger this command to be re-run. This is ultimately returned to the build system for use by the backend to ensure that incremental rebuilds happen when any dependency changes.

mozbuild.action.node.is_script_in_allowlist(script_path)

mozbuild.action.package_generated_sources module

mozbuild.action.package_generated_sources.main(argv)

mozbuild.action.preprocessor module

mozbuild.action.preprocessor.generate(output, *args)

mozbuild.action.preprocessor.main(args)

mozbuild.action.process_define_files module

mozbuild.action.process_define_files.main(argv)

mozbuild.action.process_define_files.process_define_file(output, input)

Creates the given config header. A config header is generated by taking the corresponding source file and replacing some #define/#undef occurences:

“#undef NAME” is turned into “#define NAME VALUE”

“#define NAME” is unchanged

“#define NAME ORIGINAL_VALUE” is turned into “#define NAME VALUE”

“#undef UNKNOWN_NAME” is turned into “/* #undef UNKNOWN_NAME */”

Whitespaces are preserved.

As a special rule, “#undef ALLDEFINES” is turned into “#define NAME VALUE” for all the defined variables.

mozbuild.action.process_install_manifest module

mozbuild.action.process_install_manifest.main(argv)

mozbuild.action.process_install_manifest.process_manifest(destdir, paths, track, no_symlinks=False, defines={})

mozbuild.action.symbols_archive module

mozbuild.action.symbols_archive.main(argv)

mozbuild.action.symbols_archive.make_archive(archive_name, base, exclude, include)

mozbuild.action.test_archive module

mozbuild.action.tooltool module

exception mozbuild.action.tooltool.BadFilenameException(filename): Bases: ExceptionWithFilename

exception mozbuild.action.tooltool.BadHeaderValue: Bases: Exception

exception mozbuild.action.tooltool.DigestMismatchException(filename): Bases: ExceptionWithFilename

exception mozbuild.action.tooltool.ExceptionWithFilename(filename): Bases: Exception

class mozbuild.action.tooltool.FileRecord(filename, size, digest, algorithm, unpack=False, version=None, visibility=None)

Bases: object

describe()

present()

validate()

validate_digest()

validate_size()

class mozbuild.action.tooltool.FileRecordJSONDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)

Bases: JSONDecoder

I help the json module materialize a FileRecord from a JSON file. I understand FileRecords and lists of FileRecords. I ignore things that I don’t expect for now

decode(s): Return the Python representation of s (a str instance containing a JSON document).

process_file_records(obj)

class mozbuild.action.tooltool.FileRecordJSONEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)

Bases: JSONEncoder

default(f)

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return JSONEncoder.default(self, o)

encode_file_record(obj)

exception mozbuild.action.tooltool.FileRecordJSONEncoderException: Bases: Exception

exception mozbuild.action.tooltool.InvalidCredentials: Bases: Exception

exception mozbuild.action.tooltool.InvalidManifest: Bases: Exception

class mozbuild.action.tooltool.Manifest(file_records=None)

Bases: object

copy()

dump(output_file, fmt='json')

dumps(fmt='json')

load(data_file, fmt='json')

loads(data_string, fmt='json')

present()

valid_formats = ('json',)

validate()

validate_digests()

validate_sizes()

exception mozbuild.action.tooltool.MissingFileException(filename): Bases: ExceptionWithFilename

class mozbuild.action.tooltool.TarFile(name=None, mode='r', fileobj=None, format=None, tarinfo=None, dereference=None, ignore_zeros=None, encoding=None, errors='surrogateescape', pax_headers=None, debug=None, errorlevel=None, copybufsize=None)

Bases: TarFile

extract(*args, **kwargs)

Extract a member from the archive to the current working directory, using its full name. Its file information is extracted as accurately as possible. member’ may be a filename or a TarInfo object. You can specify a different directory using `path’. File attributes (owner, mtime, mode) are set unless `set_attrs’ is False. If `numeric_owner is True, only the numbers for user/group names are used and not the names.

The filter function will be called before extraction. It can return a changed TarInfo or None to skip the member. String names of common filters are accepted.

extractall(*args, **kwargs)

Extract all members from the archive to the current working directory and set owner, modification time and permissions on directories afterwards. path’ specifies a different directory to extract to. `members’ is optional and must be a subset of the list returned by getmembers(). If `numeric_owner is True, only the numbers for user/group names are used and not the names.

The filter function will be called on each member just before extraction. It can return a changed TarInfo or None to skip the member. String names of common filters are accepted.

mozbuild.action.tooltool.add_files(manifest_file, algorithm, filenames, version, visibility, unpack)

mozbuild.action.tooltool.calculate_mac(mac_type, access_token, algorithm, timestamp, nonce, method, name, host, port, content_hash)

mozbuild.action.tooltool.calculate_payload_hash(algorithm, payload, content_type)

mozbuild.action.tooltool.change_visibility(base_urls, digest, visibility, auth_file)

mozbuild.action.tooltool.clean_path(dirname): Remove a subtree if is exists. Helper for unpack_file().

mozbuild.action.tooltool.create_file_record(filename, algorithm)

mozbuild.action.tooltool.delete_instances(base_urls, digest, auth_file)

mozbuild.action.tooltool.digest_file(f, a): I take a file like object ‘f’ and return a hex-string containing of the result of the algorithm ‘a’ applied to ‘f’.

mozbuild.action.tooltool.execute(cmd): Execute CMD, logging its stdout at the info level

mozbuild.action.tooltool.fetch_file(base_urls, file_record, grabchunk=4096, auth_file=None, region=None)

mozbuild.action.tooltool.fetch_files(manifest_file, base_urls, filenames=[], cache_folder=None, auth_file=None, region=None)

mozbuild.action.tooltool.freespace(p): Returns the number of bytes free under directory p

mozbuild.action.tooltool.get_hexdigest(val)

mozbuild.action.tooltool.list_manifest(manifest_file): I know how print all the files in a location

mozbuild.action.tooltool.main(argv, _skip_logging=False)

mozbuild.action.tooltool.make_taskcluster_header(credentials, req)

mozbuild.action.tooltool.normalize_header_attr(val)

mozbuild.action.tooltool.normalize_string(mac_type, timestamp, nonce, method, name, host, port, content_hash)

mozbuild.action.tooltool.open_manifest(manifest_file): I know how to take a filename and load it into a Manifest object

mozbuild.action.tooltool.parse_content_type(content_type)

mozbuild.action.tooltool.parse_url(url)

mozbuild.action.tooltool.prepare_header_val(val)

mozbuild.action.tooltool.process_command(options, args): I know how to take a list of program arguments and start doing the right thing with them

mozbuild.action.tooltool.purge(folder, gigs): If gigs is non 0, it deletes files in folder until gigs GB are free, starting from older files. If gigs is 0, a full purge will be performed. No recursive deletion of files in subfolder is performed.

mozbuild.action.tooltool.random() → x in the interval [0, 1).

mozbuild.action.tooltool.random_string(length)

mozbuild.action.tooltool.request(url, auth_file=None)

mozbuild.action.tooltool.request_has_data(req)

mozbuild.action.tooltool.retriable(*retry_args, **retry_kwargs): This function originates from redo 2.0.3 https://github.com/mozilla-releng/redo A decorator factory for retry(). Wrap your function in @retriable(…) to give it retry powers!

mozbuild.action.tooltool.retrier(attempts=5, sleeptime=10, max_sleeptime=300, sleepscale=1.5, jitter=1): This function originates from redo 2.0.3 https://github.com/mozilla-releng/redo A generator function that sleeps between retries, handles exponential backoff and jitter. The action you are retrying is meant to run after retrier yields.

mozbuild.action.tooltool.retry(action, attempts=5, sleeptime=60, max_sleeptime=300, sleepscale=1.5, jitter=1, retry_exceptions=(<class 'Exception'>, ), cleanup=None, args=(), kwargs={}, log_args=True): This function originates from redo 2.0.3 https://github.com/mozilla-releng/redo Calls an action function until it succeeds, or we give up.

mozbuild.action.tooltool.safe_extract(tar, path='.', *, numeric_owner=False)

mozbuild.action.tooltool.send_operation_on_file(data, base_urls, digest, auth_file)

mozbuild.action.tooltool.touch(f): Used to modify mtime in cached files; mtime is used by the purge command

mozbuild.action.tooltool.unpack_file(filename): Untar filename, assuming it is uncompressed or compressed with bzip2, xz, gzip, zst, or unzip a zip file. The file is assumed to contain a single directory with a name matching the base of the given filename. Xz support is handled by shelling out to ‘tar’.

mozbuild.action.tooltool.upload(manifest, message, base_urls, auth_file, region)

mozbuild.action.tooltool.utc_now(offset_in_seconds=0.0)

mozbuild.action.tooltool.validate_manifest(manifest_file): I validate that all files in a manifest are present and valid but don’t fetch or delete them if they aren’t

mozbuild.action.tooltool.validate_tar_member(member, path)

mozbuild.action.tooltool.validate_taskcluster_credentials(credentials)

mozbuild.action.unify_symbols module

class mozbuild.action.unify_symbols.UnifiedSymbolsFinder(finder1, finder2, sorted=[], **kargs)

Bases: UnifiedFinder

unify_file(path, file1, file2): Given two BaseFiles and the path they were found at, return a unified version of the files. If the files match, the first BaseFile may be returned. If the files don’t match or one of them is None, the method returns None. Subclasses may decide to unify by using one of the files in that case.

mozbuild.action.unify_symbols.main()

mozbuild.action.unify_tests module

class mozbuild.action.unify_tests.UnifiedTestFinder(finder1, finder2, sorted=[], **kargs)

Bases: UnifiedFinder

unify_file(path, file1, file2): Given two BaseFiles and the path they were found at, return a unified version of the files. If the files match, the first BaseFile may be returned. If the files don’t match or one of them is None, the method returns None. Subclasses may decide to unify by using one of the files in that case.

mozbuild.action.unify_tests.main()

mozbuild.action.unpack_dmg module

mozbuild.action.unpack_dmg.main(args)

mozbuild.action.webidl module

mozbuild.action.webidl.main(argv): Perform WebIDL code generation required by the build system.

mozbuild.action.wrap_rustc module

mozbuild.action.wrap_rustc.parse_outputs(crate_output, dep_outputs, pass_l_flag)

mozbuild.action.wrap_rustc.wrap_rustc(args)

mozbuild.action.xpccheck module

A generic script to verify all test files are in the corresponding .toml file.

Usage: xpccheck.py <directory> [<directory> …]

mozbuild.action.xpccheck.getIniTests(testdir)

mozbuild.action.xpccheck.main(argv)

mozbuild.action.xpccheck.verifyDirectory(initests, directory)

mozbuild.action.xpccheck.verifyIniFile(initests, directory)

mozbuild.action.xpidl-process module

mozbuild.action.zip module

mozbuild.action.zip.main(args)

mozbuild.action package

Submodules

mozbuild.action.buildlist module

mozbuild.action.check_binary module

mozbuild.action.download_wpt_manifest module

mozbuild.action.dump_env module

mozbuild.action.dumpsymbols module

mozbuild.action.exe_7z_archive module

mozbuild.action.fat_aar module

mozbuild.action.file_generate module

mozbuild.action.file_generate_wrapper module

mozbuild.action.generate_symbols_file module

mozbuild.action.html_fragment_preprocesor module

mozbuild.action.install module

mozbuild.action.jar_maker module

mozbuild.action.langpack_manifest module

mozbuild.action.make_dmg module

mozbuild.action.make_unzip module

mozbuild.action.node module

mozbuild.action.package_generated_sources module

mozbuild.action.preprocessor module

mozbuild.action.process_define_files module

mozbuild.action.process_install_manifest module

mozbuild.action.symbols_archive module

mozbuild.action.test_archive module

mozbuild.action.tooltool module

mozbuild.action.unify_symbols module

mozbuild.action.unify_tests module

mozbuild.action.unpack_dmg module

mozbuild.action.webidl module

mozbuild.action.wrap_rustc module

mozbuild.action.xpccheck module

mozbuild.action.xpidl-process module

mozbuild.action.zip module

Module contents