gecko_taskgraph.actions package
Submodules
gecko_taskgraph.actions.add_new_jobs module
- gecko_taskgraph.actions.add_new_jobs.add_new_jobs_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.add_talos module
- gecko_taskgraph.actions.add_talos.add_all_talos(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.backfill module
- gecko_taskgraph.actions.backfill.add_all_browsertime(parameters, graph_config, input, task_group_id, task_id)
This action is normally scheduled by the backfill-all-browsertime action. The intent is to trigger all browsertime tasks for the current revision.
The push in which we want to schedule a new task is defined by the parameters object.
To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally
- gecko_taskgraph.actions.backfill.add_backfill_suffix(regex, symbol, suffix)
- gecko_taskgraph.actions.backfill.add_task_with_original_manifests(parameters, graph_config, input, task_group_id, task_id)
This action is normally scheduled by the backfill action. The intent is to schedule a test task with the test manifests from the original task (if available).
The push in which we want to schedule a new task is defined by the parameters object.
To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally
- gecko_taskgraph.actions.backfill.backfill_action(parameters, graph_config, input, task_group_id, task_id)
This action takes a task ID and schedules it on previous pushes (via support action).
To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally
- gecko_taskgraph.actions.backfill.backfill_all_browsertime(parameters, graph_config, input, task_group_id, task_id)
This action takes a revision and schedules it on previous pushes (via support action).
To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally
- gecko_taskgraph.actions.backfill.backfill_modifier(task, input)
- gecko_taskgraph.actions.backfill.do_not_modify(task)
- gecko_taskgraph.actions.backfill.filter_raptor_jobs(full_task_graph, label_to_taskid, project)
- gecko_taskgraph.actions.backfill.input_for_support_action(revision, task, times=1, retrigger=True)
Generate input for action to be scheduled.
Define what label to schedule with ‘label’. If it is a test task that uses explicit manifests add that information.
- gecko_taskgraph.actions.backfill.new_label(label, tasks)
This is to handle the case when a previous push does not contain a specific task label and we try to find a label we can reuse.
For instance, we try to backfill chunk #3, however, a previous push does not contain such chunk, thus, we try to reuse another task/label.
gecko_taskgraph.actions.cancel module
- gecko_taskgraph.actions.cancel.cancel_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.cancel_all module
- gecko_taskgraph.actions.cancel_all.cancel_all_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.confirm_failure module
- gecko_taskgraph.actions.confirm_failure.confirm_failures(parameters, graph_config, input, task_group_id, task_id)
- gecko_taskgraph.actions.confirm_failure.confirm_modifier(task, input)
- gecko_taskgraph.actions.confirm_failure.get_failures(task_id, task_definition)
Returns a dict containing properties containing a list of directories containing test failures and a separate list of individual test failures from the errorsummary.log artifact for the task.
Find test path to pass to the task in MOZHARNESS_TEST_PATHS. If no appropriate test path can be determined, nothing is returned.
- gecko_taskgraph.actions.confirm_failure.get_repeat_args(task_definition, failure_group)
gecko_taskgraph.actions.create_interactive module
- gecko_taskgraph.actions.create_interactive.context(params)
- gecko_taskgraph.actions.create_interactive.create_interactive_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.gecko_profile module
- gecko_taskgraph.actions.gecko_profile.add_args_to_perf_command(payload_commands, extra_args=[])
Add custom command line args to a given command. :param payload_commands: the raw command as seen by taskcluster :param extra_args: array of args we want to inject
- gecko_taskgraph.actions.gecko_profile.geckoprofile_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.googleplay module
- gecko_taskgraph.actions.googleplay.add_push_bundle(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.merge_automation module
- gecko_taskgraph.actions.merge_automation.is_release_promotion_available(parameters)
- gecko_taskgraph.actions.merge_automation.merge_automation_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.openh264 module
- gecko_taskgraph.actions.openh264.openh264_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.purge_caches module
- gecko_taskgraph.actions.purge_caches.purge_caches_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.raptor_extra_options module
- gecko_taskgraph.actions.raptor_extra_options.raptor_extra_options_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.rebuild_cached_tasks module
- gecko_taskgraph.actions.rebuild_cached_tasks.rebuild_cached_tasks_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.registry module
- class gecko_taskgraph.actions.registry.Action(order, cb_name, permission, action_builder)
Bases:
tuple
- action_builder
Alias for field number 3
- cb_name
Alias for field number 1
- order
Alias for field number 0
- permission
Alias for field number 2
- gecko_taskgraph.actions.registry.hash_taskcluster_yml(filename)
Generate a hash of the given .taskcluster.yml. This is the first 10 digits of the sha256 of the file’s content, and is used by administrative scripts to create a hook based on this content.
- gecko_taskgraph.actions.registry.is_json(data)
Return
True
, ifdata
is a JSON serializable data structure.
- gecko_taskgraph.actions.registry.read_taskcluster_yml(filename)
Load and parse .taskcluster.yml, memoized to save some time
- gecko_taskgraph.actions.registry.register_callback_action(name, title, symbol, description, order=10000, context=[], available=<function <lambda>>, schema=None, permission='generic', cb_name=None)
Register an action callback that can be triggered from supporting user interfaces, such as Treeherder.
This function is to be used as a decorator for a callback that takes parameters as follows:
parameters
:Decision task parameters, see
taskgraph.parameters.Parameters
.input
:Input matching specified JSON schema,
None
if noschema
parameter is given toregister_callback_action
.task_group_id
:The id of the task-group this was triggered for.
task_id
and task`:task identifier and task definition for task the action was triggered for,
None
if nocontext
parameters was given toregister_callback_action
.
- Parameters:
name (str) – An identifier for this action, used by UIs to find the action.
title (str) – A human readable title for the action to be used as label on a button or text on a link for triggering the action.
symbol (str) – Treeherder symbol for the action callback, this is the symbol that the task calling your callback will be displayed as. This is usually 1-3 letters abbreviating the action title.
description (str) – A human readable description of the action in markdown. This will be display as tooltip and in dialog window when the action is triggered. This is a good place to describe how to use the action.
order (int) – Order of the action in menus, this is relative to the
order
of other actions declared.context (list of dict) –
List of tag-sets specifying which tasks the action is can take as input. If no tag-sets is specified as input the action is related to the entire task-group, and won’t be triggered with a given task.
Otherwise, if
context = [{'k': 'b', 'p': 'l'}, {'k': 't'}]
will only be displayed in the context menu for tasks that hastask.tags.k == 'b' && task.tags.p = 'l'
ortask.tags.k = 't'
. Esentially, this allows filtering ontask.tags
.If this is a function, it is given the decision parameters and must return a value of the form described above.
available (function) – An optional function that given decision parameters decides if the action is available. Defaults to a function that always returns
True
.schema (dict) – JSON schema specifying input accepted by the action. This is optional and can be left
null
if no input is taken.permission (string) – This defaults to
generic
and needs to be set for actions that need additional permissions. It appears appears in ci-configuration and various role and hook names.cb_name (string) – The name under which this function should be registered, defaulting to name. Unlike name, which can appear multiple times, cb_name must be unique among all registered callbacks.
- Returns:
To be used as decorator for the callback function.
- Return type:
function
- gecko_taskgraph.actions.registry.render_actions_json(parameters, graph_config, decision_task_id)
Render JSON object for the
public/actions.json
artifact.- Parameters:
parameters (taskgraph.parameters.Parameters) – Decision task parameters.
- Returns:
JSON object representation of the
public/actions.json
artifact.- Return type:
dict
- gecko_taskgraph.actions.registry.sanity_check_task_scope(callback, parameters, graph_config)
If this action is not generic, then verify that this task has the necessary scope to run the action. This serves as a backstop preventing abuse by running non-generic actions using generic hooks. While scopes should prevent serious damage from such abuse, it’s never a valid thing to do.
- gecko_taskgraph.actions.registry.trigger_action_callback(task_group_id, task_id, input, callback, parameters, root, test=False)
Trigger action callback with the given inputs. If test is true, then run the action callback in testing mode, without actually creating tasks.
gecko_taskgraph.actions.release_promotion module
- gecko_taskgraph.actions.release_promotion.get_flavors(graph_config, param)
Get all flavors with the given parameter enabled.
- gecko_taskgraph.actions.release_promotion.get_partner_config(partner_url_config, github_token)
- gecko_taskgraph.actions.release_promotion.get_required_signoffs(input, parameters)
- gecko_taskgraph.actions.release_promotion.get_signoff_properties()
- gecko_taskgraph.actions.release_promotion.get_signoff_urls(input, parameters)
- gecko_taskgraph.actions.release_promotion.is_release_promotion_available(parameters)
- gecko_taskgraph.actions.release_promotion.release_promotion_action(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.retrigger module
- gecko_taskgraph.actions.retrigger.rerun_action(parameters, graph_config, input, task_group_id, task_id)
- gecko_taskgraph.actions.retrigger.retrigger_action(parameters, graph_config, input, task_group_id, task_id)
- gecko_taskgraph.actions.retrigger.retrigger_decision_action(parameters, graph_config, input, task_group_id, task_id)
For a single task, we try to just run exactly the same task once more. It’s quite possible that we don’t have the scopes to do so (especially for an action), but this is best-effort.
- gecko_taskgraph.actions.retrigger.retrigger_multiple(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.retrigger_custom module
- gecko_taskgraph.actions.retrigger_custom.basic_custom_retrigger_action_basic(parameters, graph_config, input, task_group_id, task_id)
- gecko_taskgraph.actions.retrigger_custom.extended_custom_retrigger_action(parameters, graph_config, input, task_group_id, task_id)
- gecko_taskgraph.actions.retrigger_custom.handle_custom_retrigger(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.run_missing_tests module
- gecko_taskgraph.actions.run_missing_tests.run_missing_tests(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.scriptworker_canary module
- gecko_taskgraph.actions.scriptworker_canary.scriptworker_canary(parameters, graph_config, input, task_group_id, task_id)
gecko_taskgraph.actions.side_by_side module
- gecko_taskgraph.actions.side_by_side.input_for_support_action(revision, base_revision, base_branch, task)
Generate input for action to be scheduled.
Define what label to schedule with ‘label’. If it is a test task that uses explicit manifests add that information.
- gecko_taskgraph.actions.side_by_side.side_by_side_action(parameters, graph_config, input, task_group_id, task_id)
This action does a side-by-side comparison between current revision and the revision entered manually or the latest revision that ran the pageload job (via support action).
To execute this action locally follow the documentation here: https://firefox-source-docs.mozilla.org/taskcluster/actions.html#testing-the-action-locally
- gecko_taskgraph.actions.side_by_side.side_by_side_modifier(task, input)
gecko_taskgraph.actions.util module
- gecko_taskgraph.actions.util.add_args_to_command(cmd_parts, extra_args=[])
Add custom command line args to a given command. :param cmd_parts: the raw command as seen by taskcluster :param extra_args: array of args we want to add
- gecko_taskgraph.actions.util.combine_task_graph_files(suffixes)
Combine task-graph-{suffix}.json files into a single task-graph.json file.
Since Chain of Trust verification requires a task-graph.json file that contains all children tasks, we can combine the various task-graph-0.json type files into a master task-graph.json file at the end.
Actions also look for various artifacts, so we combine those in a similar fashion.
In the case where there is only one suffix, we simply rename it to avoid the additional cost of uploading two copies of the same data.
- gecko_taskgraph.actions.util.create_task_from_def(task_def, level, action_tag=None)
Create a new task from a definition rather than from a label that is already in the full-task-graph. The task definition will have {relative-datestamp’: ‘..’} rendered just like in a decision task. Use this for entirely new tasks or ones that change internals of the task. It is useful if you want to “edit” the full_task_graph and then hand it to this function. No dependencies will be scheduled. You must handle this yourself. Seeing how create_tasks handles it might prove helpful.
- gecko_taskgraph.actions.util.create_tasks(graph_config, to_run, full_task_graph, label_to_taskid, params, decision_task_id, suffix='', modifier=<function <lambda>>, action_tag=None)
Create new tasks. The task definition will have {relative-datestamp’: ‘..’} rendered just like in a decision task. Action callbacks should use this function to create new tasks, allowing easy debugging with mach taskgraph action-callback –test. This builds up all required tasks to run in order to run the tasks requested.
Optionally this function takes a modifier function that is passed in each task before it is put into a new graph. It should return a valid task. Note that this is passed _all_ tasks in the graph, not just the set in to_run. You may want to skip modifying tasks not in your to_run list.
If suffix is given, then it is used to give unique names to the resulting artifacts. If you call this function multiple times in the same action, pass a different suffix each time to avoid overwriting artifacts.
If you wish to create the tasks in a new group, leave out decision_task_id.
Returns an updated label_to_taskid containing the new tasks
- gecko_taskgraph.actions.util.fetch_graph_and_labels(parameters, graph_config)
- gecko_taskgraph.actions.util.get_decision_task_id(project, push_id)
- gecko_taskgraph.actions.util.get_parameters(decision_task_id)
- gecko_taskgraph.actions.util.get_pushes(project, end_id, depth, full_response=False)
- gecko_taskgraph.actions.util.get_pushes_from_params_input(parameters, input)
- gecko_taskgraph.actions.util.get_tasks_with_downstream(labels, full_task_graph, label_to_taskid)
- gecko_taskgraph.actions.util.relativize_datestamps(task_def)
Given a task definition as received from the queue, convert all datestamps to {relative_datestamp: ..} format, with the task creation time as “now”. The result is useful for handing to
create_task
.
- gecko_taskgraph.actions.util.trigger_action(action_name, decision_task_id, task_id=None, input={})
- gecko_taskgraph.actions.util.update_action_tag(task, graph, action_tag)
- gecko_taskgraph.actions.util.update_dependencies(task, graph)
- gecko_taskgraph.actions.util.update_parent(task, graph)
Module contents
- gecko_taskgraph.actions.register_callback_action(name, title, symbol, description, order=10000, context=[], available=<function <lambda>>, schema=None, permission='generic', cb_name=None)
Register an action callback that can be triggered from supporting user interfaces, such as Treeherder.
This function is to be used as a decorator for a callback that takes parameters as follows:
parameters
:Decision task parameters, see
taskgraph.parameters.Parameters
.input
:Input matching specified JSON schema,
None
if noschema
parameter is given toregister_callback_action
.task_group_id
:The id of the task-group this was triggered for.
task_id
and task`:task identifier and task definition for task the action was triggered for,
None
if nocontext
parameters was given toregister_callback_action
.
- Parameters:
name (str) – An identifier for this action, used by UIs to find the action.
title (str) – A human readable title for the action to be used as label on a button or text on a link for triggering the action.
symbol (str) – Treeherder symbol for the action callback, this is the symbol that the task calling your callback will be displayed as. This is usually 1-3 letters abbreviating the action title.
description (str) – A human readable description of the action in markdown. This will be display as tooltip and in dialog window when the action is triggered. This is a good place to describe how to use the action.
order (int) – Order of the action in menus, this is relative to the
order
of other actions declared.context (list of dict) –
List of tag-sets specifying which tasks the action is can take as input. If no tag-sets is specified as input the action is related to the entire task-group, and won’t be triggered with a given task.
Otherwise, if
context = [{'k': 'b', 'p': 'l'}, {'k': 't'}]
will only be displayed in the context menu for tasks that hastask.tags.k == 'b' && task.tags.p = 'l'
ortask.tags.k = 't'
. Esentially, this allows filtering ontask.tags
.If this is a function, it is given the decision parameters and must return a value of the form described above.
available (function) – An optional function that given decision parameters decides if the action is available. Defaults to a function that always returns
True
.schema (dict) – JSON schema specifying input accepted by the action. This is optional and can be left
null
if no input is taken.permission (string) – This defaults to
generic
and needs to be set for actions that need additional permissions. It appears appears in ci-configuration and various role and hook names.cb_name (string) – The name under which this function should be registered, defaulting to name. Unlike name, which can appear multiple times, cb_name must be unique among all registered callbacks.
- Returns:
To be used as decorator for the callback function.
- Return type:
function
- gecko_taskgraph.actions.render_actions_json(parameters, graph_config, decision_task_id)
Render JSON object for the
public/actions.json
artifact.- Parameters:
parameters (taskgraph.parameters.Parameters) – Decision task parameters.
- Returns:
JSON object representation of the
public/actions.json
artifact.- Return type:
dict
- gecko_taskgraph.actions.trigger_action_callback(task_group_id, task_id, input, callback, parameters, root, test=False)
Trigger action callback with the given inputs. If test is true, then run the action callback in testing mode, without actually creating tasks.