gecko_taskgraph.transforms package¶
Subpackages¶
- gecko_taskgraph.transforms.job package
- Submodules
- gecko_taskgraph.transforms.job.common module
- gecko_taskgraph.transforms.job.distro_package module
- gecko_taskgraph.transforms.job.hazard module
- gecko_taskgraph.transforms.job.mach module
- gecko_taskgraph.transforms.job.mozharness module
- gecko_taskgraph.transforms.job.mozharness_test module
- gecko_taskgraph.transforms.job.python_test module
- gecko_taskgraph.transforms.job.run_task module
- gecko_taskgraph.transforms.job.spidermonkey module
- gecko_taskgraph.transforms.job.toolchain module
- Module contents
- gecko_taskgraph.transforms.test package
Submodules¶
gecko_taskgraph.transforms.balrog_submit module¶
gecko_taskgraph.transforms.balrog_toplevel module¶
Transform the beetmover task into an actual task description.
- gecko_taskgraph.transforms.balrog_toplevel.generate_update_line(config, jobs)¶
Resolve fields that can be keyed by platform, etc.
gecko_taskgraph.transforms.base module¶
- class gecko_taskgraph.transforms.base.TransformConfig(kind, path: str, config: dict, params: taskgraph.parameters.Parameters, kind_dependencies_tasks, graph_config: taskgraph.config.GraphConfig, write_artifacts: bool)¶
Bases:
object
A container for configuration affecting transforms. The config argument to transforms is an instance of this class.
- class gecko_taskgraph.transforms.base.TransformSequence(transforms=NOTHING)¶
Bases:
object
Container for a sequence of transforms. Each transform is represented as a callable taking (config, items) and returning a generator which will yield transformed items. The resulting sequence has the same interface.
This is convenient to use in a file full of transforms, as it provides a decorator, @transforms.add, that will add the decorated function to the sequence.
- add(func)¶
- add_validate(schema)¶
- class gecko_taskgraph.transforms.base.ValidateSchema(schema: gecko_taskgraph.util.schema.Schema)¶
Bases:
object
gecko_taskgraph.transforms.beetmover module¶
gecko_taskgraph.transforms.beetmover_checksums module¶
gecko_taskgraph.transforms.beetmover_emefree_checksums module¶
gecko_taskgraph.transforms.beetmover_geckoview module¶
gecko_taskgraph.transforms.beetmover_langpack_checksums module¶
gecko_taskgraph.transforms.beetmover_push_to_release module¶
gecko_taskgraph.transforms.beetmover_repackage module¶
gecko_taskgraph.transforms.beetmover_repackage_l10n module¶
Transform the signing task into an actual task description.
- gecko_taskgraph.transforms.beetmover_repackage_l10n.make_beetmover_description(config, jobs)¶
gecko_taskgraph.transforms.beetmover_repackage_partner module¶
gecko_taskgraph.transforms.beetmover_snap module¶
Transform the snap beetmover kind into an actual task description.
- gecko_taskgraph.transforms.beetmover_snap.leave_snap_repackage_dependencies_only(config, jobs)¶
- gecko_taskgraph.transforms.beetmover_snap.set_custom_treeherder_job_name(config, jobs)¶
gecko_taskgraph.transforms.beetmover_source module¶
Transform the beetmover-source task to also append build as dependency
- gecko_taskgraph.transforms.beetmover_source.remove_build_dependency_in_beetmover_source(config, jobs)¶
gecko_taskgraph.transforms.beetmover_source_checksums module¶
gecko_taskgraph.transforms.bouncer_aliases module¶
gecko_taskgraph.transforms.bouncer_check module¶
- gecko_taskgraph.transforms.bouncer_check.add_command(config, jobs)¶
- gecko_taskgraph.transforms.bouncer_check.add_previous_versions(config, jobs)¶
- gecko_taskgraph.transforms.bouncer_check.command_to_string(config, jobs)¶
Convert command to string to make it work properly with run-task
- gecko_taskgraph.transforms.bouncer_check.handle_keyed_by(config, jobs)¶
Resolve fields that can be keyed by project, etc.
gecko_taskgraph.transforms.bouncer_locations module¶
- gecko_taskgraph.transforms.bouncer_locations.make_task_worker(config, jobs)¶
gecko_taskgraph.transforms.bouncer_submission module¶
gecko_taskgraph.transforms.bouncer_submission_partners module¶
gecko_taskgraph.transforms.build module¶
Apply some defaults and minor modifications to the jobs defined in the build kind.
- gecko_taskgraph.transforms.build.enable_full_crashsymbols(config, jobs)¶
Enable full crashsymbols on jobs with ‘enable-full-crashsymbols’ set to True and on release branches, or on try
- gecko_taskgraph.transforms.build.mozconfig(config, jobs)¶
- gecko_taskgraph.transforms.build.resolve_keys(config, jobs)¶
- gecko_taskgraph.transforms.build.resolve_shipping_product(config, jobs)¶
- gecko_taskgraph.transforms.build.set_defaults(config, jobs)¶
Set defaults, including those that differ per worker implementation
- gecko_taskgraph.transforms.build.stub_installer(config, jobs)¶
- gecko_taskgraph.transforms.build.update_channel(config, jobs)¶
- gecko_taskgraph.transforms.build.use_artifact(config, jobs)¶
- gecko_taskgraph.transforms.build.use_profile_data(config, jobs)¶
gecko_taskgraph.transforms.build_attrs module¶
- gecko_taskgraph.transforms.build_attrs.set_build_attributes(config, jobs)¶
Set the build_platform and build_type attributes based on the job name. Although not all jobs using this transform are actual “builds”, the try option syntax treats them as such, and this arranges the attributes appropriately for that purpose.
- gecko_taskgraph.transforms.build_attrs.set_schedules_optimization(config, jobs)¶
Set the skip-unless-affected optimization based on the build platform.
gecko_taskgraph.transforms.build_fat_aar module¶
- gecko_taskgraph.transforms.build_fat_aar.set_fetches_and_locations(config, jobs)¶
Set defaults, including those that differ per worker implementation
gecko_taskgraph.transforms.build_lints module¶
Apply some defaults and minor modifications to the jobs defined in the build kind.
- gecko_taskgraph.transforms.build_lints.check_mozharness_perfherder_options(config, jobs)¶
Verify that multiple jobs don’t use the same perfherder bucket.
Build jobs record perfherder metrics by default. Perfherder metrics go to a bucket derived by the platform by default. The name can further be customized by the presence of “extra options” either defined in mozharness sub-configs or in an environment variable.
This linter tries to verify that no 2 jobs will send Perfherder metrics to the same bucket by looking for jobs not defining extra options when their platform or mozharness config are otherwise similar.
gecko_taskgraph.transforms.build_signing module¶
Transform the signing task into an actual task description.
- gecko_taskgraph.transforms.build_signing.add_signed_routes(config, jobs)¶
Add routes corresponding to the routes of the build task this corresponds to, with .signed inserted, for all gecko.v2 routes
- gecko_taskgraph.transforms.build_signing.define_upstream_artifacts(config, jobs)¶
gecko_taskgraph.transforms.cached_tasks module¶
- gecko_taskgraph.transforms.cached_tasks.cache_task(config, tasks)¶
- gecko_taskgraph.transforms.cached_tasks.format_task_digest(cached_task)¶
- gecko_taskgraph.transforms.cached_tasks.order_tasks(config, tasks)¶
Iterate image tasks in an order where parent tasks come first.
gecko_taskgraph.transforms.chunk_partners module¶
Chunk the partner repack tasks by subpartner and locale
- gecko_taskgraph.transforms.chunk_partners.chunk_partners(config, jobs)¶
gecko_taskgraph.transforms.code_review module¶
Add soft dependencies and configuration to code-review tasks.
- gecko_taskgraph.transforms.code_review.add_dependencies(config, jobs)¶
- gecko_taskgraph.transforms.code_review.add_phabricator_config(config, jobs)¶
gecko_taskgraph.transforms.condprof module¶
gecko_taskgraph.transforms.copy_attributes_from_dependent_task module¶
Transform the repackage task into an actual task description.
- gecko_taskgraph.transforms.copy_attributes_from_dependent_task.copy_attributes(config, jobs)¶
gecko_taskgraph.transforms.cross_channel module¶
Build a command to run mach l10n-cross-channel.
- gecko_taskgraph.transforms.cross_channel.build_command(config, jobs)¶
- gecko_taskgraph.transforms.cross_channel.resolve_keys(config, jobs)¶
gecko_taskgraph.transforms.diffoscope module¶
gecko_taskgraph.transforms.docker_image module¶
gecko_taskgraph.transforms.fetch module¶
- class gecko_taskgraph.transforms.fetch.FetchBuilder(schema: gecko_taskgraph.util.schema.Schema, builder)¶
Bases:
object
- gecko_taskgraph.transforms.fetch.configure_fetch(config, typ, name, fetch)¶
- gecko_taskgraph.transforms.fetch.create_chromium_fetch_task(config, name, fetch)¶
- gecko_taskgraph.transforms.fetch.create_fetch_url_task(config, name, fetch)¶
- gecko_taskgraph.transforms.fetch.create_git_fetch_task(config, name, fetch)¶
- gecko_taskgraph.transforms.fetch.fetch_builder(name, schema)¶
- gecko_taskgraph.transforms.fetch.make_task(config, jobs)¶
- gecko_taskgraph.transforms.fetch.process_fetch_job(config, jobs)¶
gecko_taskgraph.transforms.final_verify module¶
Transform the beetmover task into an actual task description.
- gecko_taskgraph.transforms.final_verify.add_command(config, tasks)¶
gecko_taskgraph.transforms.fxrecord module¶
- gecko_taskgraph.transforms.fxrecord.fxrecord(config, jobs)¶
gecko_taskgraph.transforms.geckodriver_signing module¶
gecko_taskgraph.transforms.github_sync module¶
- gecko_taskgraph.transforms.github_sync.sync_github(config, tasks)¶
Do transforms specific to github-sync tasks.
gecko_taskgraph.transforms.iris module¶
Take the base iris task definition and generate all of the actual test chunks for all combinations of test categories and test platforms.
- gecko_taskgraph.transforms.iris.add_notify_email(config, tasks)¶
- gecko_taskgraph.transforms.iris.fill_email_data(config, tasks)¶
- gecko_taskgraph.transforms.iris.make_iris_tasks(config, jobs)¶
gecko_taskgraph.transforms.l10n module¶
gecko_taskgraph.transforms.mac_dummy module¶
Add dependencies to dummy macosx64 tasks.
- gecko_taskgraph.transforms.mac_dummy.add_dependencies(config, jobs)¶
gecko_taskgraph.transforms.mac_notarization_poller module¶
Transform the mac notarization poller task into an actual task description.
- gecko_taskgraph.transforms.mac_notarization_poller.make_task_description(config, jobs)¶
gecko_taskgraph.transforms.mar_signing module¶
Transform the {partials,mar}-signing task into an actual task description.
- gecko_taskgraph.transforms.mar_signing.generate_complete_artifacts(job, kind)¶
- gecko_taskgraph.transforms.mar_signing.generate_partials_artifacts(job, release_history, platform, locale=None)¶
- gecko_taskgraph.transforms.mar_signing.make_task_description(config, jobs)¶
gecko_taskgraph.transforms.maybe_release module¶
- gecko_taskgraph.transforms.maybe_release.make_task_description(config, jobs)¶
gecko_taskgraph.transforms.merge_automation module¶
Transform the update generation task into an actual task description.
- gecko_taskgraph.transforms.merge_automation.add_payload_config(config, tasks)¶
- gecko_taskgraph.transforms.merge_automation.handle_keyed_by(config, tasks)¶
Resolve fields that can be keyed by platform, etc.
- gecko_taskgraph.transforms.merge_automation.update_labels(config, tasks)¶
gecko_taskgraph.transforms.name_sanity module¶
Generate labels for tasks without names, consistently. Uses attributes from primary-dependency.
- gecko_taskgraph.transforms.name_sanity.make_label(config, jobs)¶
Generate a sane label for a new task constructed from a dependency Using attributes from the dependent job and the current task kind
gecko_taskgraph.transforms.openh264 module¶
This transform is used to help populate mozharness options for openh264 jobs
- gecko_taskgraph.transforms.openh264.set_mh_options(config, jobs)¶
This transform sets the ‘openh264_rev’ attribute.
gecko_taskgraph.transforms.openh264_signing module¶
gecko_taskgraph.transforms.partials module¶
Transform the partials task into an actual task description.
- gecko_taskgraph.transforms.partials.identify_desired_signing_keys(project, product)¶
- gecko_taskgraph.transforms.partials.make_task_description(config, jobs)¶
gecko_taskgraph.transforms.partner_attribution module¶
Transform the partner attribution task into an actual task description.
- gecko_taskgraph.transforms.partner_attribution.add_command_arguments(config, tasks)¶
gecko_taskgraph.transforms.partner_attribution_beetmover module¶
gecko_taskgraph.transforms.partner_repack module¶
Transform the partner repack task into an actual task description.
- gecko_taskgraph.transforms.partner_repack.add_command_arguments(config, tasks)¶
- gecko_taskgraph.transforms.partner_repack.make_label(config, tasks)¶
- gecko_taskgraph.transforms.partner_repack.populate_repack_manifests_url(config, tasks)¶
- gecko_taskgraph.transforms.partner_repack.skip_unnecessary_platforms(config, tasks)¶
gecko_taskgraph.transforms.partner_signing module¶
Transform the signing task into an actual task description.
- gecko_taskgraph.transforms.partner_signing.define_upstream_artifacts(config, jobs)¶
- gecko_taskgraph.transforms.partner_signing.set_mac_label(config, jobs)¶
gecko_taskgraph.transforms.per_platform_dummy module¶
Transform the repackage task into an actual task description.
- gecko_taskgraph.transforms.per_platform_dummy.one_task_per_product_and_platform(config, jobs)¶
gecko_taskgraph.transforms.perftest module¶
This transform passes options from mach perftest to the corresponding task.
- gecko_taskgraph.transforms.perftest.handle_keyed_by_perftest(config, jobs)¶
- gecko_taskgraph.transforms.perftest.parse_perftest_metrics(config, jobs)¶
Parse the metrics into a dictionary immediately.
This way we can modify the extraOptions field (and others) entry through the transforms that come later. The metrics aren’t formatted until the end of the transforms.
- gecko_taskgraph.transforms.perftest.pass_perftest_options(config, jobs)¶
- gecko_taskgraph.transforms.perftest.setup_http3_tests(config, jobs)¶
- gecko_taskgraph.transforms.perftest.setup_perftest_browsertime_variants(config, jobs)¶
- gecko_taskgraph.transforms.perftest.setup_perftest_extra_options(config, jobs)¶
- gecko_taskgraph.transforms.perftest.setup_perftest_metrics(config, jobs)¶
- gecko_taskgraph.transforms.perftest.setup_perftest_test_date(config, jobs)¶
- gecko_taskgraph.transforms.perftest.split_btime_variants(config, jobs)¶
- gecko_taskgraph.transforms.perftest.split_perftest_variants(config, jobs)¶
- gecko_taskgraph.transforms.perftest.split_tests(config, jobs)¶
gecko_taskgraph.transforms.python_update module¶
Transform the repo-update task into an actual task description.
- gecko_taskgraph.transforms.python_update.resolve_keys(config, tasks)¶
gecko_taskgraph.transforms.release module¶
Transforms for release tasks
- gecko_taskgraph.transforms.release.run_on_releases(config, jobs)¶
Filter out jobs with run-on-releases set, and that don’t match the release_type paramater.
gecko_taskgraph.transforms.release_beetmover_signed_addons module¶
gecko_taskgraph.transforms.release_deps module¶
Add dependencies to release tasks.
- gecko_taskgraph.transforms.release_deps.add_dependencies(config, jobs)¶
gecko_taskgraph.transforms.release_flatpak_push module¶
gecko_taskgraph.transforms.release_flatpak_repackage module¶
- gecko_taskgraph.transforms.release_flatpak_repackage.format(config, tasks)¶
Apply format substitution to worker.env and worker.command.
gecko_taskgraph.transforms.release_generate_checksums module¶
Transform the checksums task into an actual task description.
- gecko_taskgraph.transforms.release_generate_checksums.handle_keyed_by(config, jobs)¶
Resolve fields that can be keyed by project, etc.
- gecko_taskgraph.transforms.release_generate_checksums.interpolate(config, jobs)¶
gecko_taskgraph.transforms.release_generate_checksums_beetmover module¶
gecko_taskgraph.transforms.release_generate_checksums_signing module¶
gecko_taskgraph.transforms.release_mark_as_shipped module¶
- gecko_taskgraph.transforms.release_mark_as_shipped.make_task_description(config, jobs)¶
gecko_taskgraph.transforms.release_msix_push module¶
gecko_taskgraph.transforms.release_notifications module¶
Add notifications via taskcluster-notify for release tasks
- class gecko_taskgraph.transforms.release_notifications.TitleCaseFormatter¶
Bases:
string.Formatter
Support title formatter for strings
- convert_field(value, conversion)¶
- gecko_taskgraph.transforms.release_notifications.add_notifications(config, jobs)¶
gecko_taskgraph.transforms.release_sign_and_push_langpacks module¶
gecko_taskgraph.transforms.release_snap_repackage module¶
- gecko_taskgraph.transforms.release_snap_repackage.format(config, tasks)¶
Apply format substitution to worker.env and worker.command.
gecko_taskgraph.transforms.release_started module¶
Add notifications via taskcluster-notify for release tasks
- gecko_taskgraph.transforms.release_started.add_notifications(config, jobs)¶
gecko_taskgraph.transforms.release_version_bump module¶
Transform the update generation task into an actual task description.
- gecko_taskgraph.transforms.release_version_bump.handle_keyed_by(config, tasks)¶
Resolve fields that can be keyed by platform, etc.
gecko_taskgraph.transforms.repackage module¶
gecko_taskgraph.transforms.repackage_l10n module¶
Transform the repackage task into an actual task description.
- gecko_taskgraph.transforms.repackage_l10n.split_locales(config, jobs)¶
gecko_taskgraph.transforms.repackage_partner module¶
gecko_taskgraph.transforms.repackage_routes module¶
Add indexes to repackage kinds
- gecko_taskgraph.transforms.repackage_routes.add_indexes(config, jobs)¶
gecko_taskgraph.transforms.repackage_signing module¶
gecko_taskgraph.transforms.repackage_signing_partner module¶
gecko_taskgraph.transforms.repo_update module¶
Transform the repo-update task into an actual task description.
- gecko_taskgraph.transforms.repo_update.resolve_keys(config, tasks)¶
gecko_taskgraph.transforms.reverse_chunk_deps module¶
Adjust dependencies to not exceed MAX_DEPENDENCIES
- gecko_taskgraph.transforms.reverse_chunk_deps.add_dependencies(config, jobs)¶
- gecko_taskgraph.transforms.reverse_chunk_deps.yield_job(orig_job, deps, count)¶
gecko_taskgraph.transforms.run_pgo_profile module¶
Apply some defaults and minor modifications to the pgo jobs.
- gecko_taskgraph.transforms.run_pgo_profile.run_profile_data(config, jobs)¶
gecko_taskgraph.transforms.scriptworker module¶
Transforms for adding appropriate scopes to scriptworker tasks.
- gecko_taskgraph.transforms.scriptworker.add_balrog_scopes(config, jobs)¶
gecko_taskgraph.transforms.scriptworker_canary module¶
Build a command to run mach release push-scriptworker-canaries.
- gecko_taskgraph.transforms.scriptworker_canary.build_command(config, jobs)¶
gecko_taskgraph.transforms.sentry module¶
- gecko_taskgraph.transforms.sentry.sentry(config, tasks)¶
Do transforms specific to github-sync tasks.
gecko_taskgraph.transforms.shippable_l10n_signing module¶
Transform the signing task into an actual task description.
- gecko_taskgraph.transforms.shippable_l10n_signing.define_upstream_artifacts(config, jobs)¶
- gecko_taskgraph.transforms.shippable_l10n_signing.make_signing_description(config, jobs)¶
gecko_taskgraph.transforms.signing module¶
gecko_taskgraph.transforms.source_checksums_signing module¶
gecko_taskgraph.transforms.source_test module¶
gecko_taskgraph.transforms.spidermonkey module¶
- gecko_taskgraph.transforms.spidermonkey.handle_keyed_by(config, jobs)¶
Resolve fields that can be keyed by platform, etc.
gecko_taskgraph.transforms.startup_test module¶
- gecko_taskgraph.transforms.startup_test.add_command(config, jobs)¶
gecko_taskgraph.transforms.strip_dependent_task module¶
FIXME
- gecko_taskgraph.transforms.strip_dependent_task.strip_dependent_task(config, jobs)¶
gecko_taskgraph.transforms.task module¶
gecko_taskgraph.transforms.try_job module¶
- gecko_taskgraph.transforms.try_job.set_job_try_name(config, jobs)¶
For a task which is governed by -j in try syntax, set the job_try_name attribute based on the job name.
gecko_taskgraph.transforms.update_verify module¶
Transform the beetmover task into an actual task description.
- gecko_taskgraph.transforms.update_verify.add_command(config, tasks)¶
gecko_taskgraph.transforms.update_verify_config module¶
gecko_taskgraph.transforms.upload_generated_sources module¶
Transform the upload-generated-files task description template, taskcluster/ci/upload-generated-sources/kind.yml, into an actual task description.
- gecko_taskgraph.transforms.upload_generated_sources.add_task_info(config, jobs)¶
gecko_taskgraph.transforms.upload_symbols module¶
Transform the upload-symbols task description template, taskcluster/ci/upload-symbols/job-template.yml into an actual task description.
- gecko_taskgraph.transforms.upload_symbols.check_nightlies(config, tasks)¶
Ensure that we upload symbols for all shippable builds, so that crash-stats can resolve any reports sent to it. Try may enable full symbols but not upload them.
Putting this check here (instead of the transforms for the build kind) lets us leverage the any not-for-build-platforms set in the update-symbols kind.
- gecko_taskgraph.transforms.upload_symbols.fill_template(config, tasks)¶
gecko_taskgraph.transforms.upstream_artifact_task module¶
Find upstream artifact task.
- gecko_taskgraph.transforms.upstream_artifact_task.find_upstream_artifact_task(config, jobs)¶