gecko_taskgraph.actions package¶

Submodules¶

gecko_taskgraph.actions.add_new_jobs module¶

gecko_taskgraph.actions.add_new_jobs.add_new_jobs_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.add_talos module¶

gecko_taskgraph.actions.add_talos.add_all_talos(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.backfill module¶

gecko_taskgraph.actions.backfill.add_all_browsertime(parameters, graph_config, input, task_group_id, task_id)¶

This action is normally scheduled by the backfill-all-browsertime action. The intent is to trigger all browsertime tasks for the current revision.

The push in which we want to schedule a new task is defined by the parameters object.

To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally

gecko_taskgraph.actions.backfill.add_backfill_suffix(regex, symbol, suffix)¶

gecko_taskgraph.actions.backfill.add_task_with_original_manifests(parameters, graph_config, input, task_group_id, task_id)¶

This action is normally scheduled by the backfill action. The intent is to schedule a test task with the test manifests from the original task (if available).

The push in which we want to schedule a new task is defined by the parameters object.

To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally

gecko_taskgraph.actions.backfill.backfill_action(parameters, graph_config, input, task_group_id, task_id)¶

This action takes a task ID and schedules it on previous pushes (via support action).

To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally

gecko_taskgraph.actions.backfill.backfill_all_browsertime(parameters, graph_config, input, task_group_id, task_id)¶

This action takes a revision and schedules it on previous pushes (via support action).

To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally

gecko_taskgraph.actions.backfill.do_not_modify(task)¶

gecko_taskgraph.actions.backfill.filter_raptor_jobs(full_task_graph, label_to_taskid)¶

gecko_taskgraph.actions.backfill.input_for_support_action(revision, task, times=1, retrigger=True)¶

Generate input for action to be scheduled.

Define what label to schedule with ‘label’. If it is a test task that uses explicit manifests add that information.

gecko_taskgraph.actions.backfill.new_label(label, tasks)¶

This is to handle the case when a previous push does not contain a specific task label and we try to find a label we can reuse.

For instance, we try to backfill chunk #3, however, a previous push does not contain such chunk, thus, we try to reuse another task/label.

gecko_taskgraph.actions.backfill.test_manifests_modifier(task, label, symbol, revision, test_manifests)¶: In the case of test tasks we can modify the test paths they execute.

gecko_taskgraph.actions.cancel module¶

gecko_taskgraph.actions.cancel.cancel_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.cancel_all module¶

gecko_taskgraph.actions.cancel_all.cancel_all_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.create_interactive module¶

gecko_taskgraph.actions.create_interactive.context(params)¶

gecko_taskgraph.actions.create_interactive.create_interactive_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.gecko_profile module¶

gecko_taskgraph.actions.gecko_profile.add_args_to_perf_command(payload_commands, extra_args=[])¶: Add custom command line args to a given command. :param payload_commands: the raw command as seen by taskcluster :param extra_args: array of args we want to inject

gecko_taskgraph.actions.gecko_profile.geckoprofile_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.isolate_test module¶

gecko_taskgraph.actions.isolate_test.create_isolate_failure_tasks(task_definition, failures, level, times)¶: Create tasks to re-run the original task plus tasks to test each failing test directory and individual path.

gecko_taskgraph.actions.isolate_test.get_failures(task_id)¶

Returns a dict containing properties containing a list of directories containing test failures and a separate list of individual test failures from the errorsummary.log artifact for the task.

Calls the helper function munge_test_path to attempt to find an appropriate test path to pass to the task in MOZHARNESS_TEST_PATHS. If no appropriate test path can be determined, nothing is returned.

gecko_taskgraph.actions.isolate_test.isolate_test_failures(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.merge_automation module¶

gecko_taskgraph.actions.merge_automation.is_release_promotion_available(parameters)¶

gecko_taskgraph.actions.merge_automation.merge_automation_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.openh264 module¶

gecko_taskgraph.actions.openh264.openh264_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.purge_caches module¶

gecko_taskgraph.actions.purge_caches.purge_caches_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.rebuild_cached_tasks module¶

gecko_taskgraph.actions.rebuild_cached_tasks.rebuild_cached_tasks_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.registry module¶

class gecko_taskgraph.actions.registry.Action(order, cb_name, permission, action_builder)¶

Bases: tuple

property action_builder¶: Alias for field number 3

property cb_name¶: Alias for field number 1

property order¶: Alias for field number 0

property permission¶: Alias for field number 2

gecko_taskgraph.actions.registry.hash_taskcluster_yml(filename)¶: Generate a hash of the given .taskcluster.yml. This is the first 10 digits of the sha256 of the file’s content, and is used by administrative scripts to create a hook based on this content.

gecko_taskgraph.actions.registry.is_json(data)¶: Return True, if data is a JSON serializable data structure.

gecko_taskgraph.actions.registry.read_taskcluster_yml(filename)¶: Load and parse .taskcluster.yml, memoized to save some time

gecko_taskgraph.actions.registry.register_callback_action(name, title, symbol, description, order=10000, context=[], available=<function <lambda>>, schema=None, permission='generic', cb_name=None)¶

Register an action callback that can be triggered from supporting user interfaces, such as Treeherder.

This function is to be used as a decorator for a callback that takes parameters as follows:

parameters:: Decision task parameters, see taskgraph.parameters.Parameters.
input:: Input matching specified JSON schema, None if no schema parameter is given to register_callback_action.
task_group_id:: The id of the task-group this was triggered for.
task_id and task`:: task identifier and task definition for task the action was triggered for, None if no context parameters was given to register_callback_action.

Parameters

name (str) – An identifier for this action, used by UIs to find the action.
title (str) – A human readable title for the action to be used as label on a button or text on a link for triggering the action.
symbol (str) – Treeherder symbol for the action callback, this is the symbol that the task calling your callback will be displayed as. This is usually 1-3 letters abbreviating the action title.
description (str) – A human readable description of the action in markdown. This will be display as tooltip and in dialog window when the action is triggered. This is a good place to describe how to use the action.
order (int) – Order of the action in menus, this is relative to the order of other actions declared.
context (list of dict) –
List of tag-sets specifying which tasks the action is can take as input. If no tag-sets is specified as input the action is related to the entire task-group, and won’t be triggered with a given task.

Otherwise, if context = [{'k': 'b', 'p': 'l'}, {'k': 't'}] will only be displayed in the context menu for tasks that has task.tags.k == 'b' && task.tags.p = 'l' or task.tags.k = 't'. Esentially, this allows filtering on task.tags.

If this is a function, it is given the decision parameters and must return a value of the form described above.
available (function) – An optional function that given decision parameters decides if the action is available. Defaults to a function that always returns True.
schema (dict) – JSON schema specifying input accepted by the action. This is optional and can be left null if no input is taken.
permission (string) – This defaults to ``generic` and needs to be set for actions that need additional permissions. It appears appears in ci-configuration and various role and hook names.
cb_name (string) – The name under which this function should be registered, defaulting to name. Unlike name, which can appear multiple times, cb_name must be unique among all registered callbacks.

Returns

To be used as decorator for the callback function.

Return type

function

gecko_taskgraph.actions.registry.render_actions_json(parameters, graph_config, decision_task_id)¶

Render JSON object for the public/actions.json artifact.

Parameters: parameters (taskgraph.parameters.Parameters) – Decision task parameters.
Returns: JSON object representation of the public/actions.json artifact.
Return type: dict

gecko_taskgraph.actions.registry.sanity_check_task_scope(callback, parameters, graph_config)¶: If this action is not generic, then verify that this task has the necessary scope to run the action. This serves as a backstop preventing abuse by running non-generic actions using generic hooks. While scopes should prevent serious damage from such abuse, it’s never a valid thing to do.

gecko_taskgraph.actions.registry.trigger_action_callback(task_group_id, task_id, input, callback, parameters, root, test=False)¶: Trigger action callback with the given inputs. If test is true, then run the action callback in testing mode, without actually creating tasks.

gecko_taskgraph.actions.release_promotion module¶

gecko_taskgraph.actions.release_promotion.get_flavors(graph_config, param)¶: Get all flavors with the given parameter enabled.

gecko_taskgraph.actions.release_promotion.get_partner_config(partner_url_config, github_token)¶

gecko_taskgraph.actions.release_promotion.get_required_signoffs(input, parameters)¶

gecko_taskgraph.actions.release_promotion.get_signoff_properties()¶

gecko_taskgraph.actions.release_promotion.get_signoff_urls(input, parameters)¶

gecko_taskgraph.actions.release_promotion.is_release_promotion_available(parameters)¶

gecko_taskgraph.actions.release_promotion.release_promotion_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.retrigger module¶

gecko_taskgraph.actions.retrigger.rerun_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.retrigger.retrigger_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.retrigger.retrigger_decision_action(parameters, graph_config, input, task_group_id, task_id)¶: For a single task, we try to just run exactly the same task once more. It’s quite possible that we don’t have the scopes to do so (especially for an action), but this is best-effort.

gecko_taskgraph.actions.retrigger.retrigger_multiple(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.retrigger_custom module¶

gecko_taskgraph.actions.retrigger_custom.basic_custom_retrigger_action_basic(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.retrigger_custom.extended_custom_retrigger_action(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.retrigger_custom.handle_custom_retrigger(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.run_missing_tests module¶

gecko_taskgraph.actions.run_missing_tests.run_missing_tests(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.scriptworker_canary module¶

gecko_taskgraph.actions.scriptworker_canary.scriptworker_canary(parameters, graph_config, input, task_group_id, task_id)¶

gecko_taskgraph.actions.util module¶

gecko_taskgraph.actions.util.add_args_to_command(cmd_parts, extra_args=[])¶: Add custom command line args to a given command. :param cmd_parts: the raw command as seen by taskcluster :param extra_args: array of args we want to add

gecko_taskgraph.actions.util.combine_task_graph_files(suffixes)¶

Combine task-graph-{suffix}.json files into a single task-graph.json file.

Since Chain of Trust verification requires a task-graph.json file that contains all children tasks, we can combine the various task-graph-0.json type files into a master task-graph.json file at the end.

Actions also look for various artifacts, so we combine those in a similar fashion.

In the case where there is only one suffix, we simply rename it to avoid the additional cost of uploading two copies of the same data.

gecko_taskgraph.actions.util.create_task_from_def(task_def, level, action_tag=None)¶: Create a new task from a definition rather than from a label that is already in the full-task-graph. The task definition will have {relative-datestamp’: ‘..’} rendered just like in a decision task. Use this for entirely new tasks or ones that change internals of the task. It is useful if you want to “edit” the full_task_graph and then hand it to this function. No dependencies will be scheduled. You must handle this yourself. Seeing how create_tasks handles it might prove helpful.

gecko_taskgraph.actions.util.create_tasks(graph_config, to_run, full_task_graph, label_to_taskid, params, decision_task_id, suffix='', modifier=<function <lambda>>, action_tag=None)¶

Create new tasks. The task definition will have {relative-datestamp’: ‘..’} rendered just like in a decision task. Action callbacks should use this function to create new tasks, allowing easy debugging with mach taskgraph action-callback –test. This builds up all required tasks to run in order to run the tasks requested.

Optionally this function takes a modifier function that is passed in each task before it is put into a new graph. It should return a valid task. Note that this is passed _all_ tasks in the graph, not just the set in to_run. You may want to skip modifying tasks not in your to_run list.

If suffix is given, then it is used to give unique names to the resulting artifacts. If you call this function multiple times in the same action, pass a different suffix each time to avoid overwriting artifacts.

If you wish to create the tasks in a new group, leave out decision_task_id.

Returns an updated label_to_taskid containing the new tasks

gecko_taskgraph.actions.util.fetch_graph_and_labels(parameters, graph_config)¶

gecko_taskgraph.actions.util.get_decision_task_id(project, push_id)¶

gecko_taskgraph.actions.util.get_parameters(decision_task_id)¶

gecko_taskgraph.actions.util.get_pushes(project, end_id, depth, full_response=False)¶

gecko_taskgraph.actions.util.get_pushes_from_params_input(parameters, input)¶

gecko_taskgraph.actions.util.get_tasks_with_downstream(labels, full_task_graph, label_to_taskid)¶

gecko_taskgraph.actions.util.relativize_datestamps(task_def)¶: Given a task definition as received from the queue, convert all datestamps to {relative_datestamp: ..} format, with the task creation time as “now”. The result is useful for handing to create_task.

gecko_taskgraph.actions.util.trigger_action(action_name, decision_task_id, task_id=None, input={})¶

gecko_taskgraph.actions.util.update_action_tag(task, graph, action_tag)¶

gecko_taskgraph.actions.util.update_dependencies(task, graph)¶

gecko_taskgraph.actions.util.update_parent(task, graph)¶

Module contents¶

gecko_taskgraph.actions.register_callback_action(name, title, symbol, description, order=10000, context=[], available=<function <lambda>>, schema=None, permission='generic', cb_name=None)¶

Register an action callback that can be triggered from supporting user interfaces, such as Treeherder.

This function is to be used as a decorator for a callback that takes parameters as follows:

parameters:: Decision task parameters, see taskgraph.parameters.Parameters.
input:: Input matching specified JSON schema, None if no schema parameter is given to register_callback_action.
task_group_id:: The id of the task-group this was triggered for.
task_id and task`:: task identifier and task definition for task the action was triggered for, None if no context parameters was given to register_callback_action.

Parameters

name (str) – An identifier for this action, used by UIs to find the action.
title (str) – A human readable title for the action to be used as label on a button or text on a link for triggering the action.
symbol (str) – Treeherder symbol for the action callback, this is the symbol that the task calling your callback will be displayed as. This is usually 1-3 letters abbreviating the action title.
description (str) – A human readable description of the action in markdown. This will be display as tooltip and in dialog window when the action is triggered. This is a good place to describe how to use the action.
order (int) – Order of the action in menus, this is relative to the order of other actions declared.
context (list of dict) –
List of tag-sets specifying which tasks the action is can take as input. If no tag-sets is specified as input the action is related to the entire task-group, and won’t be triggered with a given task.

Otherwise, if context = [{'k': 'b', 'p': 'l'}, {'k': 't'}] will only be displayed in the context menu for tasks that has task.tags.k == 'b' && task.tags.p = 'l' or task.tags.k = 't'. Esentially, this allows filtering on task.tags.

If this is a function, it is given the decision parameters and must return a value of the form described above.
available (function) – An optional function that given decision parameters decides if the action is available. Defaults to a function that always returns True.
schema (dict) – JSON schema specifying input accepted by the action. This is optional and can be left null if no input is taken.
permission (string) – This defaults to ``generic` and needs to be set for actions that need additional permissions. It appears appears in ci-configuration and various role and hook names.
cb_name (string) – The name under which this function should be registered, defaulting to name. Unlike name, which can appear multiple times, cb_name must be unique among all registered callbacks.

Returns

To be used as decorator for the callback function.

Return type

function

gecko_taskgraph.actions.render_actions_json(parameters, graph_config, decision_task_id)¶

Render JSON object for the public/actions.json artifact.

Parameters: parameters (taskgraph.parameters.Parameters) – Decision task parameters.
Returns: JSON object representation of the public/actions.json artifact.
Return type: dict

gecko_taskgraph.actions.trigger_action_callback(task_group_id, task_id, input, callback, parameters, root, test=False)¶: Trigger action callback with the given inputs. If test is true, then run the action callback in testing mode, without actually creating tasks.