Talos is a cross-platform Python performance testing framework that is specifically for Firefox on desktop. New performance tests should be added to the newer framework mozperftest unless there are limitations there (highly unlikely) that make it absolutely necessary to add them to Talos. Talos is named after the bronze automaton from Greek myth.
Talos tests are run in a similar manner to xpcshell and mochitests. They are started via
mach talos-test. A python script
information is recorded in a text log file, e.g.
browser_output.txt, and then processed
into the JSON format supported by Perfherder.
Talos bugs can be filed in Testing::Talos.
Examples of current Talos runs can be found in Treeherder by searching for “Talos”.
If none are immediately available, then scroll to the bottom of the page and load more test
runs. The tests all share a group symbol starting with a
T, for example
T(c d damp g1) or
Running Talos Locally¶
Running tests locally is most likely only useful for debugging what is going on in a test, as the test output is only reported as raw JSON. The CLI is documented via:
./mach talos-test --help
To quickly try out the
./mach talos-test command, the following can be run to do a
single run of the DevTools’ simple netmonitor test.
# Run the "simple.netmonitor" test very quickly with 1 cycle, and 1 page cycle. ./mach talos-test --activeTests damp --subtests simple.netmonitor --cycles 1 --tppagecycles 1
--print-tests are two helpful command flags to
figure out what suites and tests are available to run.
# Print out the suites: ./mach talos-test --print-suites # Available suites: # bcv (basic_compositor_video) # chromez (about_preferences_basic:tresize:about_newtab_with_snippets) # dromaeojs (dromaeo_css:kraken) # flex (tart_flex:ts_paint_flex) # ... # Run all of the tests in the "bcv" test suite: ./mach talos-test --suite bcv # Print out the tests: ./mach talos-test --print-tests # Available tests: # ================ # # a11yr # ----- # This test ensures basic a11y tables and permutations do not cause # performance regressions. # # about_newtab_with_snippets # -------------------------- # Load about ActivityStream (about:home and about:newtab) with snippets enabled # # ... # Run the tests in "a11yr" listed above ./mach talos-test --activeTests a11yr
Running Talos on Try¶
Talos runs can be generated through the mach try fuzzy finder:
./mach try fuzzy
The following is an example output at the time of this writing. Refine the query for the platform and test suites of your choosing.
| test-windows10-64-qr/opt-talos-bcv-swr-e10s | test-linux64-shippable/opt-talos-webgl-e10s | test-linux64-shippable/opt-talos-other-e10s | test-linux64-shippable-qr/opt-talos-g5-e10s | test-linux64-shippable-qr/opt-talos-g4-e10s | test-linux64-shippable-qr/opt-talos-g3-e10s | test-linux64-shippable-qr/opt-talos-g1-e10s | test-windows10-64/opt-talos-webgl-gli-e10s | test-linux64-shippable/opt-talos-tp5o-e10s | test-linux64-shippable/opt-talos-svgr-e10s | test-linux64-shippable/opt-talos-flex-e10s | test-linux64-shippable/opt-talos-damp-e10s > test-windows7-32/opt-talos-webgl-gli-e10s | test-linux64-shippable/opt-talos-bcv-e10s | test-linux64-shippable/opt-talos-g5-e10s | test-linux64-shippable/opt-talos-g4-e10s | test-linux64-shippable/opt-talos-g3-e10s | test-linux64-shippable/opt-talos-g1-e10s | test-linux64-qr/opt-talos-bcv-swr-e10s For more shortcuts, see mach help try fuzzy and man fzf select: <tab>, accept: <enter>, cancel: <ctrl-c>, select-all: <ctrl-a>, cursor-up: <up>, cursor-down: <down> 1379/2967 > talos