taskgraph.actions package

Submodules

taskgraph.actions.add_new_jobs module

taskgraph.actions.add_new_jobs.add_new_jobs_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.add_talos module

taskgraph.actions.add_talos.add_all_talos(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.backfill module

taskgraph.actions.backfill.backfill_action(parameters, graph_config, input, task_group_id, task_id)
taskgraph.actions.backfill.remove_args_from_command(cmd_parts, preamble_length=0, args_to_ignore=[])

We need to remove all extra instances of command line arguments that are suite/job specific, like suite=jsreftest, subsuite=devtools and other ones like –total-chunk=X. :param cmd_parts: the raw command as seen by taskcluster :param preamble_length: the number of args to skip (usually python -u <name>) :param args_to_ignore: ignore specific args and their related values

taskgraph.actions.cancel module

taskgraph.actions.cancel.cancel_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.cancel_all module

taskgraph.actions.cancel_all.cancel_all_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.create_interactive module

taskgraph.actions.create_interactive.context(params)
taskgraph.actions.create_interactive.create_interactive_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.gecko_profile module

taskgraph.actions.gecko_profile.add_args_to_command(cmd_parts, extra_args=[])

Add custom command line args to a given command. :param cmd_parts: the raw command as seen by taskcluster :param extra_args: array of args we want to add

taskgraph.actions.gecko_profile.geckoprofile_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.isolate_test module

taskgraph.actions.isolate_test.create_isolate_failure_tasks(task_definition, failures, level, times)

Create tasks to re-run the original task plus tasks to test each failing test directory and individual path.

taskgraph.actions.isolate_test.get_failures(task_id)

Returns a dict containing properties containing a list of directories containing test failures and a separate list of individual test failures from the errorsummary.log artifact for the task.

Calls the helper function munge_test_path to attempt to find an appropriate test path to pass to the task in MOZHARNESS_TEST_PATHS. If no appropriate test path can be determined, nothing is returned.

taskgraph.actions.isolate_test.isolate_test_failures(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.openh264 module

taskgraph.actions.openh264.openh264_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.purge_caches module

taskgraph.actions.purge_caches.purge_caches_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.registry module

class taskgraph.actions.registry.Action(order, cb_name, generic, action_builder)

Bases: tuple

action_builder

Alias for field number 3

cb_name

Alias for field number 1

generic

Alias for field number 2

order

Alias for field number 0

taskgraph.actions.registry.hash_taskcluster_yml

Generate a hash of the given .taskcluster.yml. This is the first 10 digits of the sha256 of the file’s content, and is used by administrative scripts to create a hook based on this content.

taskgraph.actions.registry.is_json(data)

Return True, if data is a JSON serializable data structure.

taskgraph.actions.registry.read_taskcluster_yml

Load and parse .taskcluster.yml, memoized to save some time

taskgraph.actions.registry.register_callback_action(name, title, symbol, description, order=10000, context=[], available=<function <lambda>>, schema=None, generic=True, cb_name=None)

Register an action callback that can be triggered from supporting user interfaces, such as Treeherder.

This function is to be used as a decorator for a callback that takes parameters as follows:

parameters:
Decision task parameters, see taskgraph.parameters.Parameters.
input:
Input matching specified JSON schema, None if no schema parameter is given to register_callback_action.
task_group_id:
The id of the task-group this was triggered for.
task_id and task`:
task identifier and task definition for task the action was triggered for, None if no context parameters was given to register_callback_action.
Parameters:
  • name (str) – An identifier for this action, used by UIs to find the action.
  • title (str) – A human readable title for the action to be used as label on a button or text on a link for triggering the action.
  • symbol (str) – Treeherder symbol for the action callback, this is the symbol that the task calling your callback will be displayed as. This is usually 1-3 letters abbreviating the action title.
  • description (str) – A human readable description of the action in markdown. This will be display as tooltip and in dialog window when the action is triggered. This is a good place to describe how to use the action.
  • order (int) – Order of the action in menus, this is relative to the order of other actions declared.
  • context (list of dict) –

    List of tag-sets specifying which tasks the action is can take as input. If no tag-sets is specified as input the action is related to the entire task-group, and won’t be triggered with a given task.

    Otherwise, if context = [{'k': 'b', 'p': 'l'}, {'k': 't'}] will only be displayed in the context menu for tasks that has task.tags.k == 'b' && task.tags.p = 'l' or task.tags.k = 't'. Esentially, this allows filtering on task.tags.

    If this is a function, it is given the decision parameters and must return a value of the form described above.

  • available (function) – An optional function that given decision parameters decides if the action is available. Defaults to a function that always returns True.
  • schema (dict) – JSON schema specifying input accepted by the action. This is optional and can be left null if no input is taken.
  • generic (boolean) – Whether this is a generic action or has its own permissions.
  • cb_name (string) – The name under which this function should be registered, defaulting to name. This is used to generation actionPerm for non-generic hook actions, and thus appears in ci-configuration and various role and hook names. Unlike name, which can appear multiple times, cb_name must be unique among all registered callbacks.
Returns:

To be used as decorator for the callback function.

Return type:

function

taskgraph.actions.registry.render_actions_json(parameters, graph_config)

Render JSON object for the public/actions.json artifact.

Parameters:parameters (taskgraph.parameters.Parameters) – Decision task parameters.
Returns:JSON object representation of the public/actions.json artifact.
Return type:dict
taskgraph.actions.registry.sanity_check_task_scope(callback, parameters, graph_config)

If this action is not generic, then verify that this task has the necessary scope to run the action. This serves as a backstop preventing abuse by running non-generic actions using generic hooks. While scopes should prevent serious damage from such abuse, it’s never a valid thing to do.

taskgraph.actions.registry.trigger_action_callback(task_group_id, task_id, input, callback, parameters, root, test=False)

Trigger action callback with the given inputs. If test is true, then run the action callback in testing mode, without actually creating tasks.

taskgraph.actions.release_promotion module

taskgraph.actions.release_promotion.get_flavors(graph_config, param)

Get all flavors with the given parameter enabled.

taskgraph.actions.release_promotion.get_partner_config(partner_url_config, github_token)
taskgraph.actions.release_promotion.get_required_signoffs(input, parameters)
taskgraph.actions.release_promotion.get_signoff_properties()
taskgraph.actions.release_promotion.get_signoff_urls(input, parameters)
taskgraph.actions.release_promotion.is_release_promotion_available(parameters)
taskgraph.actions.release_promotion.release_promotion_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.retrigger module

taskgraph.actions.retrigger.rerun_action(parameters, graph_config, input, task_group_id, task_id)
taskgraph.actions.retrigger.retrigger_action(parameters, graph_config, input, task_group_id, task_id)
taskgraph.actions.retrigger.retrigger_decision_action(parameters, graph_config, input, task_group_id, task_id)

For a single task, we try to just run exactly the same task once more. It’s quite possible that we don’t have the scopes to do so (especially for an action), but this is best-effort.

taskgraph.actions.retrigger.retrigger_multiple(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.retrigger_mochitest module

taskgraph.actions.retrigger_mochitest.mochitest_retrigger_action(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.run_missing_tests module

taskgraph.actions.run_missing_tests.run_missing_tests(parameters, graph_config, input, task_group_id, task_id)

taskgraph.actions.util module

taskgraph.actions.util.add_args_to_command(cmd_parts, extra_args=[])

Add custom command line args to a given command. :param cmd_parts: the raw command as seen by taskcluster :param extra_args: array of args we want to add

taskgraph.actions.util.combine_task_graph_files(suffixes)

Combine task-graph-{suffix}.json files into a single task-graph.json file.

Since Chain of Trust verification requires a task-graph.json file that contains all children tasks, we can combine the various task-graph-0.json type files into a master task-graph.json file at the end.

Actions also look for various artifacts, so we combine those in a similar fashion.

In the case where there is only one suffix, we simply rename it to avoid the additional cost of uploading two copies of the same data.

taskgraph.actions.util.create_task_from_def(task_id, task_def, level)

Create a new task from a definition rather than from a label that is already in the full-task-graph. The task definition will have {relative-datestamp’: ‘..’} rendered just like in a decision task. Use this for entirely new tasks or ones that change internals of the task. It is useful if you want to “edit” the full_task_graph and then hand it to this function. No dependencies will be scheduled. You must handle this yourself. Seeing how create_tasks handles it might prove helpful.

taskgraph.actions.util.create_tasks(graph_config, to_run, full_task_graph, label_to_taskid, params, decision_task_id=None, suffix=u'', modifier=<function <lambda>>)

Create new tasks. The task definition will have {relative-datestamp’: ‘..’} rendered just like in a decision task. Action callbacks should use this function to create new tasks, allowing easy debugging with mach taskgraph action-callback –test. This builds up all required tasks to run in order to run the tasks requested.

Optionally this function takes a modifier function that is passed in each task before it is put into a new graph. It should return a valid task. Note that this is passed _all_ tasks in the graph, not just the set in to_run. You may want to skip modifying tasks not in your to_run list.

If suffix is given, then it is used to give unique names to the resulting artifacts. If you call this function multiple times in the same action, pass a different suffix each time to avoid overwriting artifacts.

If you wish to create the tasks in a new group, leave out decision_task_id.

Returns an updated label_to_taskid containing the new tasks

taskgraph.actions.util.fetch_graph_and_labels(parameters, graph_config)
taskgraph.actions.util.get_parameters(decision_task_id)
taskgraph.actions.util.relativize_datestamps(task_def)

Given a task definition as received from the queue, convert all datestamps to {relative_datestamp: ..} format, with the task creation time as “now”. The result is useful for handing to create_task.

taskgraph.actions.util.update_dependencies(task, graph)
taskgraph.actions.util.update_parent(task, graph)

Module contents

taskgraph.actions.register_callback_action(name, title, symbol, description, order=10000, context=[], available=<function <lambda>>, schema=None, generic=True, cb_name=None)

Register an action callback that can be triggered from supporting user interfaces, such as Treeherder.

This function is to be used as a decorator for a callback that takes parameters as follows:

parameters:
Decision task parameters, see taskgraph.parameters.Parameters.
input:
Input matching specified JSON schema, None if no schema parameter is given to register_callback_action.
task_group_id:
The id of the task-group this was triggered for.
task_id and task`:
task identifier and task definition for task the action was triggered for, None if no context parameters was given to register_callback_action.
Parameters:
  • name (str) – An identifier for this action, used by UIs to find the action.
  • title (str) – A human readable title for the action to be used as label on a button or text on a link for triggering the action.
  • symbol (str) – Treeherder symbol for the action callback, this is the symbol that the task calling your callback will be displayed as. This is usually 1-3 letters abbreviating the action title.
  • description (str) – A human readable description of the action in markdown. This will be display as tooltip and in dialog window when the action is triggered. This is a good place to describe how to use the action.
  • order (int) – Order of the action in menus, this is relative to the order of other actions declared.
  • context (list of dict) –

    List of tag-sets specifying which tasks the action is can take as input. If no tag-sets is specified as input the action is related to the entire task-group, and won’t be triggered with a given task.

    Otherwise, if context = [{'k': 'b', 'p': 'l'}, {'k': 't'}] will only be displayed in the context menu for tasks that has task.tags.k == 'b' && task.tags.p = 'l' or task.tags.k = 't'. Esentially, this allows filtering on task.tags.

    If this is a function, it is given the decision parameters and must return a value of the form described above.

  • available (function) – An optional function that given decision parameters decides if the action is available. Defaults to a function that always returns True.
  • schema (dict) – JSON schema specifying input accepted by the action. This is optional and can be left null if no input is taken.
  • generic (boolean) – Whether this is a generic action or has its own permissions.
  • cb_name (string) – The name under which this function should be registered, defaulting to name. This is used to generation actionPerm for non-generic hook actions, and thus appears in ci-configuration and various role and hook names. Unlike name, which can appear multiple times, cb_name must be unique among all registered callbacks.
Returns:

To be used as decorator for the callback function.

Return type:

function

taskgraph.actions.render_actions_json(parameters, graph_config)

Render JSON object for the public/actions.json artifact.

Parameters:parameters (taskgraph.parameters.Parameters) – Decision task parameters.
Returns:JSON object representation of the public/actions.json artifact.
Return type:dict
taskgraph.actions.trigger_action_callback(task_group_id, task_id, input, callback, parameters, root, test=False)

Trigger action callback with the given inputs. If test is true, then run the action callback in testing mode, without actually creating tasks.