mozbuild.vendor package

Submodules

mozbuild.vendor.host_github module

class mozbuild.vendor.host_github.GitHubHost(manifest)

Bases: object

upstream_commit(revision)

Query the github api for a git commit id and timestamp.

upstream_snapshot(revision)

mozbuild.vendor.host_gitlab module

class mozbuild.vendor.host_gitlab.GitLabHost(manifest)

Bases: object

upstream_commit(revision)

Query the gitlab api for a git commit id and timestamp.

upstream_snapshot(revision)

mozbuild.vendor.host_googlesource module

class mozbuild.vendor.host_googlesource.GoogleSourceHost(manifest)

Bases: object

upstream_commit(revision)

Query for a git commit and timestamp.

upstream_snapshot(revision)

mozbuild.vendor.mach_commands module

class mozbuild.vendor.mach_commands.Vendor(context, virtualenv_name=None, metrics=None)

Bases: mozbuild.base.MachCommandBase

Vendor third-party dependencies into the source repository.

check_modified_files()

Ensure that there aren’t any uncommitted changes to files in the working copy, since we’re going to change some state on the user.

vendor(library, revision, ignore_modified=False, check_for_update=False, verify=False)

Vendor third-party dependencies into the source repository.

Vendoring rust and python can be done with ./mach vendor [rust/python]. Vendoring other libraries can be done with ./mach vendor [arguments] path/to/file.yaml

vendor_python(**kwargs)
vendor_rust(**kwargs)

mozbuild.vendor.moz_yaml module

class mozbuild.vendor.moz_yaml.License

Bases: object

Voluptuous validator which verifies the license(s) are valid as per our allow list.

exception mozbuild.vendor.moz_yaml.MozYamlVerifyError(filename, error)

Bases: Exception

mozbuild.vendor.moz_yaml.RE_FIELD(string=None, pos=0, endpos=9223372036854775807, *, pattern=None)

Scan through string looking for a match, and return a corresponding match object instance.

Return None if no position in the string matches.

mozbuild.vendor.moz_yaml.RE_SECTION(string=None, pos=0, endpos=9223372036854775807, *, pattern=None)

Scan through string looking for a match, and return a corresponding match object instance.

Return None if no position in the string matches.

class mozbuild.vendor.moz_yaml.UpdateActions

Bases: object

Voluptuous validator which verifies the update actions(s) are valid.

mozbuild.vendor.moz_yaml.VALID_SOURCE_HOSTS = ['gitlab', 'googlesource', 'github']

— # Third-Party Library Template # All fields are mandatory unless otherwise noted

# Version of this schema schema: 1

bugzilla:

# Bugzilla product and component for this directory and subdirectories product: product name component: component name

# Document the source of externally hosted code origin:

# Short name of the package/library name: name of the package

description: short (one line) description

# Full URL for the package’s homepage/etc # Usually different from repository url url: package’s homepage url

# Human-readable identifier for this version/release # Generally “version NNN”, “tag SSS”, “bookmark SSS” release: identifier

# Revision to pull in # Must be a long or short commit SHA (long preferred) revision: sha

# The package’s license, where possible using the mnemonic from # https://spdx.org/licenses/ # Multiple licenses can be specified (as a YAML list) # A “LICENSE” file must exist containing the full license text license: MPL-2.0

# If the package’s license is specified in a particular file, # this is the name of the file. # optional license-file: COPYING

# Configuration for the automated vendoring system. # optional vendoring:

# Repository URL to vendor from # eg. https://github.com/kinetiknz/nestegg.git # Any repository host can be specified here, however initially we’ll only # support automated vendoring from selected sources initially. url: source url (generally repository clone url)

# Type of hosting for the upstream repository # Valid values are ‘gitlab’, ‘github’ source-hosting: gitlab

# Base directory of the location where the source files will live in-tree. # If omitted, will default to the location the moz.yaml file is in. vendor-directory: third_party/directory

# List of patch files to apply after vendoring. Applied in the order # specified, and alphabetically if globbing is used. Patches must apply # cleanly before changes are pushed # All patch files are implicitly added to the keep file list. # optional patches:

  • file

  • path/to/file

  • path/*.patch

# List of files that are not deleted while vendoring # Implicitly contains “moz.yaml”, any files referenced as patches # optional keep:

  • file

  • path/to/file

  • another/path

  • *.mozilla

# Files/paths that will not be vendored from source repository # Implicitly contains “.git”, and “.gitignore” # optional exclude:

  • file

  • path/to/file

  • another/path

  • docs

  • src/*.test

# Files/paths that will always be vendored, even if they would # otherwise be excluded by “exclude”. # optional include:

  • file

  • path/to/file

  • another/path

  • docs/LICENSE.*

# If neither “exclude” or “include” are set, all files will be vendored # Files/paths in “include” will always be vendored, even if excluded # eg. excluding “docs/” then including “docs/LICENSE” will vendor just the # LICENSE file from the docs directory

# All three file/path parameters (“keep”, “exclude”, and “include”) support # filenames, directory names, and globs/wildcards.

# Actions to take after updating. Applied in order. # The action subfield is required. It must be one of: # - copy-file # - replace-in-file # - delete-path # - run-script # Unless otherwise noted, all subfields of action are required. # # If the action is copy-file: # from is the source file # to is the destination # # If the action is replace-in-file: # pattern is what in the file to search for. It is an exact strng match. # with is the string to replace it with. Accepts the special keyword # ‘{revision}’ for the commit we are updating to. # File is the file to replace it in. # # If the action is delete-path # path is the file or directory to recursively delete # # If the action is run-script: # script is the script to run # cwd is the directory the script should run with as its cwd # # Unless specified otherwise, all files/directories are relative to the # vendor-directory. If the vendor-directory is different from the # directory of the yaml file, the keyword ‘{yaml_dir}’ may be used # to make the path relative to that directory. # ‘run-script’ supports the addictional keyword {cwd} which, if used, # must only be used at the beginning of the path. # # optional update-actions:

  • action: copy-file from: include/vcs_version.h.in to: ‘{yaml_dir}/vcs_version.h’

  • action: replace-in-file pattern: ‘@VCS_TAG@’ with: ‘{revision}’ file: ‘{yaml_dir}/vcs_version.h’

  • action: delete-path path: ‘{yaml_dir}/config’

  • action: run-script script: ‘{cwd}/generate_sources.sh’ cwd: ‘{yaml_dir}’

mozbuild.vendor.moz_yaml.load_moz_yaml(filename, verify=True, require_license_file=True)

Loads and verifies the specified manifest.

mozbuild.vendor.moz_yaml.update_moz_yaml(filename, release, revision, verify=True, write=True)

Update origin:release and vendoring:revision without stripping comments or reordering fields.

mozbuild.vendor.vendor_manifest module

class mozbuild.vendor.vendor_manifest.VendorManifest(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>, virtualenv_name=None)

Bases: mozbuild.base.MozbuildObject

clean_upstream()

Remove files we don’t want to import.

fetch_and_unpack(revision)

Fetch and unpack upstream source

get_source_host()
update_files(revision, yaml_file)
update_moz_build(yaml_file)
update_yaml(yaml_file, revision, timestamp)
vendor(yaml_file, manifest, revision, check_for_update)

mozbuild.vendor.vendor_python module

class mozbuild.vendor.vendor_python.VendorPython(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>, virtualenv_name=None)

Bases: mozbuild.base.MozbuildObject

vendor(packages=None, with_windows_wheel=False, keep_extra_files=False)

mozbuild.vendor.vendor_rust module

class mozbuild.vendor.vendor_rust.VendorRust(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>, virtualenv_name=None)

Bases: mozbuild.base.MozbuildObject

BUILDTIME_LICENSE_WHITELIST = {'BSD-3-Clause': ['bindgen', 'fuchsia-zircon', 'fuchsia-zircon-sys', 'fuchsia-cprng', 'glsl', 'instant']}
RUNTIME_LICENSE_FILE_PACKAGE_WHITELIST = {'deque': '6485b8ed310d3f0340bf1ad1f47645069ce4069dcc6bb46c7d5c6faf41de1fdb', 'fuchsia-cprng': '03b114f53e6587a398931762ee11e2395bfdba252a329940e2c8c9e81813845b'}
RUNTIME_LICENSE_PACKAGE_WHITELIST = {'BSD-2-Clause': ['arrayref', 'cloudabi', 'Inflector', 'mach', 'qlog'], 'BSD-3-Clause': []}
RUNTIME_LICENSE_WHITELIST = ['Apache-2.0', 'Apache-2.0 WITH LLVM-exception', 'CC0-1.0', 'ISC', 'MIT', 'MPL-2.0', 'Unlicense', 'Zlib']
check_cargo_version(cargo)

Ensure that cargo is new enough. cargo 1.42 fixed some issue with the vendor command. cargo 1.47 similarly did so for windows, but as of this writing is the current nightly, so we restrict this check only to the platform it’s actually required on

check_modified_files()

Ensure that there aren’t any uncommitted changes to files in the working copy, since we’re going to change some state on the user. Allow changes to Cargo.{toml,lock} since that’s likely to be a common use case.

check_openssl()

Set environment flags for building with openssl.

MacOS doesn’t include openssl, but the openssl-sys crate used by mach-vendor expects one of the system. It’s common to have one installed in /usr/local/opt/openssl by homebrew, but custom link flags are necessary to build against it.

get_cargo_path()
static runtime_license(package, license_string)

Cargo docs say: — https://doc.rust-lang.org/cargo/reference/manifest.html

This is an SPDX 2.1 license expression for this package. Currently crates.io will validate the license provided against a whitelist of known license and exception identifiers from the SPDX license list 2.4. Parentheses are not currently supported.

Multiple licenses can be separated with a /, although that usage is deprecated. Instead, use a license expression with AND and OR operators to get more explicit semantics. — But I have no idea how you can meaningfully AND licenses, so we will abort if that is detected. We’ll handle / and OR as equivalent and approve is any is in our approved list.

vendor(ignore_modified=False, build_peers_said_large_imports_were_ok=False)

Module contents