mozbuild.controller package

Submodules

mozbuild.controller.building module

class mozbuild.controller.building.BuildDriver(*args, **kwargs)

Bases: MozbuildObject

Provides a high-level API for build actions.

build(metrics, what=None, jobs=0, job_size=0, directory=None, verbose=False, keep_going=False, mach_context=None, append_env=None)
configure(metrics, options=None, buildstatus_messages=False, line_handler=None, append_env=None)
install_tests()

Install test files.

class mozbuild.controller.building.BuildMonitor(topsrcdir, settings, log_manager, topobjdir=None, mozconfig=<object object>, virtualenv_name=None)

Bases: MozbuildObject

Monitors the output of the build.

ccache_stats(ccache=None)
finish()

Record the end of the build.

get_resource_usage()

Produce a data structure containing the low-level resource usage information.

This data structure can e.g. be serialized into JSON and saved for subsequent analysis.

If no resource usage is available, None is returned.

have_excessive_swapping()

Determine whether there was excessive swapping during the build.

Returns a tuple of (excessive, swap_in, swap_out). All values are None if no swap information is available.

have_high_finder_usage()

Determine whether there was high Finder CPU usage during the build.

Returns True if there was high Finder CPU usage, False if there wasn’t, or None if there is nothing to report.

property have_resource_usage

Whether resource usage is available.

init(warnings_path, terminal, metrics)

Create a new monitor.

warnings_path is a path of a warnings database to use.

log_resource_usage(usage)

Summarize the resource usage of this build in a log message.

on_line(line)

Consume a line of output from the build system.

This will parse the line for state and determine whether more action is needed.

Returns a BuildOutputResult instance.

In this named tuple, warning will be an object describing a new parsed warning. Otherwise it will be None.

state_changed indicates whether the build system changed state with this line. If the build system changed state, the caller may want to query this instance for the current state in order to update UI, etc.

message is either None, or the content of a message to be displayed to the user.

record_usage()
start()

Record the start of the build.

start_resource_recording()
stop_resource_recording()
class mozbuild.controller.building.BuildOutputManager(log_manager, monitor, footer)

Bases: OutputManager

Handles writing build output to a terminal, to logs, etc.

on_line(line)
class mozbuild.controller.building.BuildOutputResult(warning, state_changed, message)

Bases: tuple

message

Alias for field number 2

state_changed

Alias for field number 1

warning

Alias for field number 0

class mozbuild.controller.building.BuildProgressFooter(terminal, monitor)

Bases: Footer

Handles display of a build progress indicator in a terminal.

When mach builds inside a blessed-supported terminal, it will render progress information collected from a BuildMonitor. This class converts the state of BuildMonitor into terminal output.

draw()

Draws this footer in the terminal.

class mozbuild.controller.building.CCacheStats(output=None, has_machine_format=False)

Bases: object

Holds statistics from ccache.

Instances can be subtracted from each other to obtain differences. print() or str() the object to show a ccache -s like output of the captured stats.

ABSOLUTE_KEYS = {'cache_files', 'cache_max_size', 'cache_size'}
FORMAT_KEYS = {'cache_max_size', 'cache_size'}
GiB = 1073741824
KiB = 1024
MiB = 1048576
SKIP_LINES = ('cache directory', 'primary config', 'secondary config')
STATS_KEYS = [('stats_zeroed', ('stats zeroed', 'stats zero time')), ('stats_updated', 'stats updated'), ('cache_hit_direct', 'cache hit (direct)'), ('cache_hit_preprocessed', 'cache hit (preprocessed)'), ('cache_hit_rate', 'cache hit rate'), ('cache_miss', 'cache miss'), ('link', 'called for link'), ('preprocessing', 'called for preprocessing'), ('multiple', 'multiple source files'), ('stdout', 'compiler produced stdout'), ('no_output', 'compiler produced no output'), ('empty_output', 'compiler produced empty output'), ('failed', 'compile failed'), ('error', 'ccache internal error'), ('preprocessor_error', 'preprocessor error'), ('cant_use_pch', "can't use precompiled header"), ('compiler_missing', "couldn't find the compiler"), ('cache_file_missing', 'cache file missing'), ('bad_args', 'bad compiler arguments'), ('unsupported_lang', 'unsupported source language'), ('compiler_check_failed', 'compiler check failed'), ('autoconf', 'autoconf compile/link'), ('unsupported_code_directive', 'unsupported code directive'), ('unsupported_compiler_option', 'unsupported compiler option'), ('out_stdout', 'output to stdout'), ('out_device', 'output to a non-regular file'), ('no_input', 'no input file'), ('bad_extra_file', 'error hashing extra file'), ('num_cleanups', 'cleanups performed'), ('cache_files', 'files in cache'), ('cache_size', 'cache size'), ('cache_max_size', 'max cache size')]
STATS_KEYS_3_7_PLUS = {'autoconf_test': 'autoconf', 'bad_compiler_arguments': 'bad_args', 'cache_miss': 'cache_miss', 'cache_size_kibibyte': 'cache_size', 'called_for_link': 'link', 'called_for_preprocessing': 'preprocessing', 'cleanups_performed': 'num_cleanups', 'compile_failed': 'failed', 'compiler_check_failed': 'compiler_check_failed', 'compiler_produced_empty_output': 'empty_output', 'compiler_produced_no_output': 'no_output', 'compiler_produced_stdout': 'stdout', 'could_not_find_compiler': 'compiler_missing', 'could_not_use_precompiled_header': 'cant_use_pch', 'direct_cache_hit': 'cache_hit_direct', 'error_hashing_extra_file': 'bad_extra_file', 'files_in_cache': 'cache_files', 'internal_error': 'error', 'missing_cache_file': 'cache_file_missing', 'multiple_source_files': 'multiple', 'no_input_file': 'no_input', 'output_to_a_non_file': 'out_device', 'output_to_stdout': 'out_stdout', 'preprocessed_cache_hit': 'cache_hit_preprocessed', 'preprocessor_error': 'preprocessor_error', 'stats_updated_timestamp': 'stats_updated', 'stats_zeroed_timestamp': 'stats_zeroed', 'unsupported_code_directive': 'unsupported_code_directive', 'unsupported_compiler_option': 'unsupported_compiler_option', 'unsupported_source_language': 'unsupported_lang'}
static check_version_3_7_or_newer(ccache)
hit_rate_message()
hit_rates()
class mozbuild.controller.building.OutputManager(log_manager, footer)

Bases: LoggingMixin

Handles writing job output to a terminal or log.

refresh()
write_line(line)
class mozbuild.controller.building.StaticAnalysisFooter(terminal, monitor)

Bases: Footer

Handles display of a static analysis progress indicator in a terminal.

draw()

Draws this footer in the terminal.

class mozbuild.controller.building.StaticAnalysisOutputManager(log_manager, monitor, footer)

Bases: OutputManager

Handles writing static analysis output to a terminal or file.

on_line(line)
write(path, output_format)
class mozbuild.controller.building.TerminalLoggingHandler

Bases: Handler

Custom logging handler that works with terminal window dressing.

This class should probably live elsewhere, like the mach core. Consider this a proving ground for its usefulness.

emit(record)

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

flush()

Ensure all logging output has been flushed.

This version does nothing and is intended to be implemented by subclasses.

class mozbuild.controller.building.TierStatus(resources, metrics)

Bases: object

Represents the state and progress of tier traversal.

The build system is organized into linear phases called tiers. Each tier executes in the order it was defined, 1 at a time.

begin_tier(tier)

Record that execution of a tier has begun.

finish_tier(tier)

Record that execution of a tier has finished.

set_tiers(tiers)

Record the set of known tiers.

mozbuild.controller.building.record_cargo_timings(resource_monitor, timings_path)

mozbuild.controller.clobber module

This module contains code for managing clobbering of the tree.

class mozbuild.controller.clobber.Clobberer(topsrcdir, topobjdir, substs=None)

Bases: object

clobber_cause()

Obtain the cause why a clobber is required.

This reads the cause from the CLOBBER file.

This returns a list of lines describing why the clobber was required. Each line is stripped of leading and trailing whitespace.

clobber_needed()

Returns a bool indicating whether a tree clobber is required.

collect_subdirs(root, exclude)

Gathers a list of subdirectories excluding specified items.

delete_dirs(root, paths_to_delete)

Deletes the given subdirectories in an optimal way.

have_winrm()
maybe_do_clobber(cwd, allow_auto=False, fh=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>)

Perform a clobber if it is required. Maybe.

This is the API the build system invokes to determine if a clobber is needed and to automatically perform that clobber if we can.

This returns a tuple of (bool, bool, str). The elements are:

  • Whether a clobber was/is required.

  • Whether a clobber was performed.

  • The reason why the clobber failed or could not be performed. This will be None if no clobber is required or if we clobbered without error.

remove_objdir(full=True)

Remove the object directory.

full controls whether to fully delete the objdir. If False, some directories (e.g. Visual Studio Project Files) will not be deleted.

Module contents