gecko_taskgraph.transforms.test package

Submodules

gecko_taskgraph.transforms.test.chunk module

gecko_taskgraph.transforms.test.chunk.DYNAMIC_CHUNK_DURATION = 1200

The approximate time each test chunk should take to run.

gecko_taskgraph.transforms.test.chunk.DYNAMIC_CHUNK_MULTIPLIER = {'^(?!android).*-xpcshell.*': 0.2}

A multiplication factor to tweak the total duration per platform / suite.

gecko_taskgraph.transforms.test.chunk.resolve_dynamic_chunks(config, tasks)

Determine how many chunks are needed to handle the given set of manifests.

gecko_taskgraph.transforms.test.chunk.set_test_manifests(config, tasks)

Determine the set of test manifests that should run in this task.

gecko_taskgraph.transforms.test.chunk.set_test_verify_chunks(config, tasks)

Set the number of chunks we use for test-verify.

gecko_taskgraph.transforms.test.chunk.split_chunks(config, tasks)

Based on the ‘chunks’ key, split tests up into chunks by duplicating them and assigning ‘this-chunk’ appropriately and updating the treeherder symbol.

gecko_taskgraph.transforms.test.confirm_failure module

gecko_taskgraph.transforms.test.confirm_failure.test_confirm_failure_tasks(config, tasks)

Copy test-* tasks to have -cf copy.

gecko_taskgraph.transforms.test.other module

gecko_taskgraph.transforms.test.other.allow_software_gl_layers(config, tasks)

Handle the “allow-software-gl-layers” property for platforms where it applies.

gecko_taskgraph.transforms.test.other.apply_raptor_tier_optimization(config, tasks)

gecko_taskgraph.transforms.test.other.disable_try_only_platforms(config, tasks)

Turns off platforms that should only run on try.

gecko_taskgraph.transforms.test.other.disable_wpt_timeouts_on_autoland(config, tasks)

do not run web-platform-tests that are expected TIMEOUT on autoland

gecko_taskgraph.transforms.test.other.enable_code_coverage(config, tasks)

Enable code coverage for the ccov build-platforms

gecko_taskgraph.transforms.test.other.enable_parallel_marking_in_tsan_tests(config, tasks)

Enable parallel marking in TSAN tests

gecko_taskgraph.transforms.test.other.enable_webrender(config, tasks)

Handle the “webrender” property by passing a flag to mozharness if it is enabled.

gecko_taskgraph.transforms.test.other.ensure_spi_disabled_on_all_but_spi(config, tasks)

gecko_taskgraph.transforms.test.other.get_mobile_project(task)

Returns the mobile project of the specified task or None.

gecko_taskgraph.transforms.test.other.handle_artifact_prefix(config, tasks)

Handle translating artifact_prefix appropriately

gecko_taskgraph.transforms.test.other.handle_keyed_by(config, tasks)

Resolve fields that can be keyed by platform, etc.

gecko_taskgraph.transforms.test.other.handle_run_on_projects(config, tasks)

Handle translating built-projects appropriately

gecko_taskgraph.transforms.test.other.handle_suite_category(config, tasks)

gecko_taskgraph.transforms.test.other.handle_tier(config, tasks)

Set the tier based on policy for all test descriptions that do not specify a tier otherwise.

gecko_taskgraph.transforms.test.other.limit_platforms(config, tasks)

gecko_taskgraph.transforms.test.other.set_download_symbols(config, tasks)

In general, we download symbols immediately for debug builds, but only on demand for everything else. ASAN builds shouldn’t download symbols since they don’t product symbol zips see bug 1283879

gecko_taskgraph.transforms.test.other.set_profile(config, tasks)

Set profiling mode for tests.

gecko_taskgraph.transforms.test.other.set_retry_exit_status(config, tasks)

Set the retry exit status to TBPL_RETRY, the value returned by mozharness scripts to indicate a transient failure that should be retried.

gecko_taskgraph.transforms.test.other.set_schedules_components(config, tasks)

gecko_taskgraph.transforms.test.other.set_schedules_for_webrender_android(config, tasks)

android-hw has limited resources, we need webrender on phones

gecko_taskgraph.transforms.test.other.set_tag(config, tasks)

Set test for a specific tag.

gecko_taskgraph.transforms.test.other.set_target(config, tasks)

gecko_taskgraph.transforms.test.other.set_test_setting(config, tasks)

A test setting is the set of configuration that uniquely distinguishes a test task from other tasks that run the same suite (ignoring chunks).

There are three different types of information that make up a setting:

1. Platform - Information describing the underlying platform tests run on, e.g, OS, CPU architecture, etc.

2. Build - Information describing the build being tested, e.g build type, ccov, asan/tsan, etc.

3. Runtime - Information describing which runtime parameters are enabled, e.g, prefs, environment variables, etc.

This transform adds a test-setting object to the extra portion of all test tasks, of the form:

{
    "platform": { ... },
    "build": { ... },
    "runtime": { ... }
}

This information could be derived from the label, but consuming this object is less brittle.

gecko_taskgraph.transforms.test.other.set_test_type(config, tasks)

gecko_taskgraph.transforms.test.other.set_treeherder_machine_platform(config, tasks)

Set the appropriate task.extra.treeherder.machine.platform

gecko_taskgraph.transforms.test.other.setup_browsertime(config, tasks)

Configure browsertime dependencies for Raptor pageload tests that have –browsertime extra option.

gecko_taskgraph.transforms.test.other.setup_browsertime_flag(config, tasks)

Optionally add –browsertime flag to Raptor pageload tests.

gecko_taskgraph.transforms.test.other.setup_raptor_external_browser_platforms(config, tasks)

gecko_taskgraph.transforms.test.other.setup_talos(config, tasks)

Add options that are specific to talos jobs (identified by suite=talos)

gecko_taskgraph.transforms.test.other.test_setting_description_schema = <Schema({'_hash': <class 'str'>, 'platform': {'arch': Any('32', '64', 'aarch64', 'arm7', 'x86', 'x86_64', msg=None), 'os': {'name': Any('android', 'linux', 'macosx', 'windows', msg=None), 'version': <class 'str'>, 'build': <class 'str'>}, 'device': <class 'str'>, 'display': 'wayland', 'machine': 'hw-ref'}, 'build': {'type': Any('opt', 'debug', 'debug-isolated-process', msg=None), Any('asan', 'ccov', 'clang-trunk', 'devedition', 'domstreams', 'lite', 'mingwclang', 'nightlyasrelease', 'shippable', 'tsan', msg=None): <class 'bool'>}, 'runtime': {Any('1proc', msg=None): <class 'bool'>}}, extra=PREVENT_EXTRA, required=False) object>

Schema test settings must conform to. Validated by test_test_setting()

gecko_taskgraph.transforms.test.raptor module

gecko_taskgraph.transforms.test.raptor.add_extra_options(config, tests)

gecko_taskgraph.transforms.test.raptor.add_scopes_and_proxy(config, tasks)

gecko_taskgraph.transforms.test.raptor.handle_keyed_by(config, tests)

gecko_taskgraph.transforms.test.raptor.handle_keyed_by_prereqs(config, tests)

Only resolve keys for prerequisite fields here since the these keyed-by options might have keyed-by fields as well.

gecko_taskgraph.transforms.test.raptor.modify_extra_options(config, tests)

gecko_taskgraph.transforms.test.raptor.modify_mozharness_configs(config, tests)

gecko_taskgraph.transforms.test.raptor.set_defaults(config, tests)

gecko_taskgraph.transforms.test.raptor.split_apps(config, tests)

gecko_taskgraph.transforms.test.raptor.split_page_load_by_url(config, tests)

gecko_taskgraph.transforms.test.raptor.split_raptor_subtests(config, tests)

gecko_taskgraph.transforms.test.variant module

gecko_taskgraph.transforms.test.variant.split_variants(config, tasks)

Splits test definitions into multiple tasks based on the variants key.

If variants are defined, the original task will be yielded along with a copy of the original task for each variant defined in the list. The copies will have the ‘unittest_variant’ attribute set.

gecko_taskgraph.transforms.test.variant.transforms = TransformSequence(_transforms=[<function split_variants>])

List of available test variants defined.

gecko_taskgraph.transforms.test.variant.variant_description_schema = <Schema({<class 'str'>: {'description': <class 'str'>, 'suffix': <class 'str'>, 'mozinfo': <class 'str'>, 'component': <class 'str'>, 'expiration': <class 'str'>, 'when': {Any('$eval', '$if', msg=None): <class 'str'>}, 'replace': {<class 'str'>: <class 'object'>}, 'merge': {<class 'str'>: <class 'object'>}}}, extra=PREVENT_EXTRA, required=False) object>

variant description schema

gecko_taskgraph.transforms.test.worker module

gecko_taskgraph.transforms.test.worker.set_wayland_env(config, tasks)

gecko_taskgraph.transforms.test.worker.set_worker_type(config, tasks)

Set the worker type based on the test platform.

Module contents

These transforms construct a task description to run the given test, based on a test description. The implementation here is shared among all test kinds, but contains specific support for how we run tests in Gecko (via mozharness, invoked in particular ways).

This is a good place to translate a test-description option such as single-core: true to the implementation of that option in a task description (worker options, mozharness commandline, environment variables, etc.)

The test description should be fully formed by the time it reaches these transforms, and these transforms should not embody any specific knowledge about what should run where. this is the wrong place for special-casing platforms, for example - use all_tests.py instead.

gecko_taskgraph.transforms.test.get_firefox_version()

gecko_taskgraph.transforms.test.handle_keyed_by_mozharness(config, tasks)

Resolve a mozharness field if it is keyed by something

gecko_taskgraph.transforms.test.make_job_description(config, tasks)

Convert test descriptions to job descriptions (input to gecko_taskgraph.transforms.job)

gecko_taskgraph.transforms.test.normpath(path)

gecko_taskgraph.transforms.test.resolve_keys(config, tasks)

gecko_taskgraph.transforms.test.run_remaining_transforms(config, tasks)

Runs other transform files next to this module.

gecko_taskgraph.transforms.test.run_variant_transforms(config, tasks)

Variant transforms are run as soon as possible to allow other transforms to key by variant.

gecko_taskgraph.transforms.test.set_defaults(config, tasks)