Raptor

Raptor is a performance-testing framework for running browser pageload and browser benchmark tests. Raptor is cross-browser compatible and is currently running in production on Firefox Desktop, Firefox Android GeckoView, Fenix, Reference Browser, Chromium, Chrome, Chrome for Android, Safari, and Safari Technology Preview.

• Contact: Dave Hunt [:davehunt]

• Source code: https://searchfox.org/mozilla-central/source/testing/raptor

• Good first bugs: https://codetribute.mozilla.org/projects/automation?project%3DRaptor

Raptor currently supports three test types: 1) page-load performance tests, 2) standard benchmark-performance tests, and 3) “scenario”-based tests, such as power, CPU, and memory-usage measurements on Android and Desktop.

Locally, Raptor can be invoked with the following command:

./mach raptor

• Raptor Browsertime
• Debugging
• Contributing
• Metrics Gathered

Raptor tests

The following documents all testing we have for Raptor.

Benchmarks

Standard benchmarks are third-party tests (i.e. Speedometer) that we have integrated into Raptor to run per-commit in our production CI. To update any of these benchmarks, see Updating Benchmark Tests.

ares6

• Command to Run Locally

./mach raptor -t ares6

Owner: :jandem and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox, chrome, safari

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/ares6.toml#16

• lower is better: true

• page cycles: 4

• page timeout: 270000

• subtest lower is better: true

• subtest unit: ms

• test url: http://<host>:<port>/ARES-6/index.html?raptor

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-ares6	❌	❌	❌	❌
browsertime-benchmark-firefox-ares6	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	✅	✅	❌	❌
browsertime-benchmark-safari-ares6	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-ares6	❌	❌	❌	❌
browsertime-benchmark-firefox-ares6	✅	✅	❌	❌

assorted-dom

• Command to Run Locally

./mach raptor -t assorted-dom

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, chrome, safari

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/assorted-dom.toml#15

• lower is better: true

• page cycles: 1

• page timeout: 60000

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/assorted-dom

• repository revision: 61332db584026b73e37066d717a162825408c36b

• screen capture: true

• test url: http://<host>:<port>/assorted/driver.html?raptor

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-assorted-dom	❌	❌	❌	❌
browsertime-benchmark-firefox-assorted-dom	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	✅	✅	❌	❌
browsertime-benchmark-safari-assorted-dom	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-assorted-dom	❌	❌	❌	❌
browsertime-benchmark-firefox-assorted-dom	✅	✅	❌	❌

jetstream2

• Command to Run Locally

./mach raptor -t jetstream2

Owner: :jandem and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox, chrome, safari, custom-car

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/jetstream2.toml#16

• lower is better: false

• page cycles: 4

• page timeout: 2000000

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/JetStream2

• repository revision: 61332db584026b73e37066d717a162825408c36b

• subtest lower is better: false

• subtest unit: score

• test url: http://<host>:<port>/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-jetstream2-chrome-m-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-chrome-m-jetstream2-nofis	❌	❌	❌	❌
browsertime-benchmark-jetstream2-cstm-car-m-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-cstm-car-m-jetstream2-nofis	❌	❌	❌	❌
browsertime-benchmark-jetstream2-fenix-jetstream2	✅	❌	❌	❌
browsertime-benchmark-jetstream2-fenix-jetstream2-nofis	✅	❌	❌	❌
browsertime-benchmark-jetstream2-geckoview-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-geckoview-jetstream2-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-jetstream2-fenix-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-fenix-jetstream2-nofis	❌	❌	❌	❌
browsertime-benchmark-jetstream2-geckoview-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-geckoview-jetstream2-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-jetstream2	❌	❌	❌	❌
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-jetstream2	❌	❌	❌	❌
browsertime-benchmark-custom-car-jetstream2	❌	❌	❌	❌
browsertime-benchmark-firefox-jetstream2	✅	❌	❌	❌
browsertime-benchmark-safari-jetstream2	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌
browsertime-benchmark-safari-jetstream2	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-jetstream2	❌	❌	❌	❌
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌

Owner: :jandem and SpiderMonkey Team

• alert threshold: 2.0

• apps: fenix, geckoview, chrome-m, cstm-car-m

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/jetstream2-mobile.toml#16

• lower is better: false

• page cycles: 4

• page timeout: 2000000

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/JetStream2

• repository revision: 61332db584026b73e37066d717a162825408c36b

• subtest lower is better: false

• subtest unit: score

• test url: http://<host>:<port>/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-jetstream2-chrome-m-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-chrome-m-jetstream2-nofis	❌	❌	❌	❌
browsertime-benchmark-jetstream2-cstm-car-m-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-cstm-car-m-jetstream2-nofis	❌	❌	❌	❌
browsertime-benchmark-jetstream2-fenix-jetstream2	✅	❌	❌	❌
browsertime-benchmark-jetstream2-fenix-jetstream2-nofis	✅	❌	❌	❌
browsertime-benchmark-jetstream2-geckoview-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-geckoview-jetstream2-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-jetstream2-fenix-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-fenix-jetstream2-nofis	❌	❌	❌	❌
browsertime-benchmark-jetstream2-geckoview-jetstream2	❌	❌	❌	❌
browsertime-benchmark-jetstream2-geckoview-jetstream2-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-jetstream2	❌	❌	❌	❌
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-jetstream2	❌	❌	❌	❌
browsertime-benchmark-custom-car-jetstream2	❌	❌	❌	❌
browsertime-benchmark-firefox-jetstream2	✅	❌	❌	❌
browsertime-benchmark-safari-jetstream2	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌
browsertime-benchmark-safari-jetstream2	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-jetstream2	❌	❌	❌	❌
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌

matrix-react-bench

• Command to Run Locally

./mach raptor -t matrix-react-bench

Owner: :jandem and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/matrix-react-bench.toml#16

• lower is better: true

• page cycles: 30

• page timeout: 2000000

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/matrix-react-bench

• repository revision: 61332db584026b73e37066d717a162825408c36b

• subtest lower is better: true

• subtest unit: ms

• test url: http://<host>:<port>/matrix_demo.html

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-matrix-react-bench	❌	❌	❌	❌
browsertime-benchmark-firefox-matrix-react-bench	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-matrix-react-bench	❌	❌	❌	❌
browsertime-benchmark-firefox-matrix-react-bench	✅	✅	❌	❌

motionmark-1-3

• Command to Run Locally

./mach raptor -t motionmark-1-3

Owner: Graphics Team

• alert threshold: 2.0

• apps: firefox, chrome, safari, custom-car

• browser cycles: 1

• cold: true

• expected: pass

• expose browser profiler: true

• expose chrome trace: true

• gecko profile features: stackwalk,js,cpu,java,processcpu,memory

• gecko profile interval: 1

• gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate

• host from parent: false

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-desktop.toml#13

• lower is better: false

• page cycles: 1

• page timeout: 600000

• repository: https://github.com/webkit/motionmark

• repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb

• subtest lower is better: false

• subtest unit: score

• suite name: MotionMark

• support class: motionmark-1-3.py

• test script: motionmark-1-3.js

• test url: http://<host>:<port>/MotionMark/developer.html?warmup-length=2000&warmup-frame-count=30&first-frame-minimum-length=0&test-interval=30&display=minimal&tiles=big&controller=ramp&system-frame-rate=60&frame-rate=60&time-measurement=performance

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	❌	❌	❌
browsertime-benchmark-safari-motionmark-1-3	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-1-3	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌

Owner: Graphics Team

• alert threshold: 2.0

• apps: fenix, geckoview, chrome-m, cstm-car-m

• browser cycles: 1

• cold: true

• expected: pass

• expose browser profiler: true

• gecko profile features: stackwalk,js,cpu,java,processcpu,memory

• gecko profile interval: 1

• gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate

• host from parent: false

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-mobile.toml#12

• lower is better: false

• page cycles: 1

• page timeout: 600000

• repository: https://github.com/webkit/motionmark

• repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb

• subtest lower is better: false

• subtest unit: score

• suite name: MotionMark

• support class: motionmark-1-3.py

• test script: motionmark-1-3.js

• test url: http://<host>:<port>/MotionMark/developer.html?warmup-length=2000&warmup-frame-count=30&first-frame-minimum-length=0&test-interval=30&display=minimal&tiles=big&controller=ramp&system-frame-rate=60&frame-rate=60&time-measurement=performance

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	❌	❌	❌
browsertime-benchmark-safari-motionmark-1-3	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-1-3	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌

motionmark-htmlsuite-1-3

• Command to Run Locally

./mach raptor -t motionmark-htmlsuite-1-3

Owner: Graphics Team

• alert threshold: 2.0

• apps: firefox, chrome, safari, custom-car

• browser cycles: 1

• cold: true

• expected: pass

• expose browser profiler: true

• expose chrome trace: true

• gecko profile features: stackwalk,js,cpu,java,processcpu,memory

• gecko profile interval: 1

• gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate

• host from parent: false

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-desktop.toml#29

• lower is better: false

• page cycles: 1

• page timeout: 600000

• repository: https://github.com/webkit/motionmark

• repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb

• subtest lower is better: false

• subtest unit: score

• suite name: HTML suite

• support class: motionmark-1-3.py

• test script: motionmark-1-3.js

• test url: http://<host>:<port>/MotionMark/developer.html?warmup-length=2000&warmup-frame-count=30&first-frame-minimum-length=0&test-interval=30&display=minimal&tiles=big&controller=ramp&system-frame-rate=60&frame-rate=60&time-measurement=performance

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	❌	❌	❌
browsertime-benchmark-safari-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌

Owner: Graphics Team

• alert threshold: 2.0

• apps: fenix, geckoview, chrome-m, cstm-car-m

• browser cycles: 1

• cold: true

• expected: pass

• expose browser profiler: true

• gecko profile features: stackwalk,js,cpu,java,processcpu,memory

• gecko profile interval: 1

• gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate

• host from parent: false

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-mobile.toml#28

• lower is better: false

• page cycles: 1

• page timeout: 600000

• repository: https://github.com/webkit/motionmark

• repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb

• subtest lower is better: false

• subtest unit: score

• suite name: HTML suite

• support class: motionmark-1-3.py

• test script: motionmark-1-3.js

• test url: http://<host>:<port>/MotionMark/developer.html?warmup-length=2000&warmup-frame-count=30&first-frame-minimum-length=0&test-interval=30&display=minimal&tiles=big&controller=ramp&system-frame-rate=60&frame-rate=60&time-measurement=performance

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	❌	❌	❌
browsertime-benchmark-safari-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌

speedometer

• Command to Run Locally

./mach raptor -t speedometer

Owner: SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox, chrome, safari, safari-tp, custom-car

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile features: stackwalk,js,cpu,processcpu,ipcmessages,nomarkerstacks

• gecko profile interval: 1

• gecko profile threads: GeckoMain,Compositor,Renderer,TaskController,StyleThread

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-desktop.toml#19

• lower is better: false

• page cycles: 5

• page timeout: 600000

• subtest lower is better: true

• subtest unit: ms

• test url: http://<host>:<port>/Speedometer/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m	✅	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix	✅	✅	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-p6-13-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-s24-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	❌	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌

Owner: SpiderMonkey Team

• alert threshold: 2.0

• apps: fenix, geckoview, refbrow, chrome-m, cstm-car-m

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile features: stackwalk,js,cpu,processcpu,ipcmessages,nomarkerstacks

• gecko profile interval: 1

• gecko profile threads: GeckoMain,Compositor,Renderer,TaskController,StyleThread

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-mobile.toml#19

• lower is better: false

• page cycles: 5

• page timeout: 420000

• subtest lower is better: true

• subtest unit: ms

• test url: http://<host>:<port>/Speedometer/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m	✅	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix	✅	✅	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-p6-13-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-s24-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	❌	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌

speedometer3

• Command to Run Locally

./mach raptor -t speedometer3

Owner: Performance Team

• alert threshold: 2.0

• apps: firefox, chrome, safari, safari-tp, custom-car

• browser cycles: 5

• cold: true

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile features: stackwalk,js,cpu,processcpu,ipcmessages,nomarkerstacks

• gecko profile interval: 1

• gecko profile threads: GeckoMain,Compositor,Renderer,TaskController,StyleThread

• host from parent: false

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-desktop.toml#21

• lower is better: false

• page cycles: 5

• page timeout: 600000

• subtest lower is better: true

• subtest unit: ms

• support class: speedometer3.py

• test script: speedometer3.js

• test url: http://<host>:<port>/Speedometer3/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix	✅	✅	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-p6-13-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-s24-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	❌	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer3	❌	❌	❌	❌
browsertime-benchmark-safari-tp-speedometer3	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-windows10-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	❌	❌	❌

test-windows11-64-2009-hw-ref-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	❌	❌	❌

test-windows11-64-2009-hw-ref/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

Owner: Performance Team

• alert threshold: 2.0

• apps: fenix, geckoview, refbrow, chrome-m, cstm-car-m

• browser cycles: 5

• cold: true

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile features: stackwalk,js,cpu,processcpu,ipcmessages,nomarkerstacks

• gecko profile interval: 1

• gecko profile threads: GeckoMain,Compositor,Renderer,TaskController,StyleThread

• host from parent: false

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-mobile.toml#21

• lower is better: false

• page cycles: 5

• page timeout: 420000

• repository: https://github.com/WebKit/Speedometer

• repository revision: 8d67f28d0281ac4330f283495b7f48286654ad7d

• subtest lower is better: true

• subtest unit: ms

• support class: speedometer3.py

• test script: speedometer3.js

• test url: http://<host>:<port>/index.html

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix	✅	✅	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-p6-13-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

test-android-hw-s24-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-macosx1400-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	❌	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer3	❌	❌	❌	❌
browsertime-benchmark-safari-tp-speedometer3	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-windows10-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	❌	❌	❌

test-windows11-64-2009-hw-ref-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	❌	❌	❌

test-windows11-64-2009-hw-ref/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

stylebench

• Command to Run Locally

./mach raptor -t stylebench

Owner: :emelio and Layout Team

• alert threshold: 2.0

• apps: firefox, chrome, safari

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/stylebench.toml#16

• lower is better: false

• page cycles: 5

• page timeout: 140000

• subtest lower is better: true

• subtest unit: ms

• test url: http://<host>:<port>/StyleBench/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-stylebench	❌	❌	❌	❌
browsertime-benchmark-firefox-stylebench	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	✅	✅	❌	❌
browsertime-benchmark-safari-stylebench	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-stylebench	❌	❌	❌	❌
browsertime-benchmark-firefox-stylebench	✅	✅	❌	❌

sunspider

• Command to Run Locally

./mach raptor -t sunspider

Owner: :jandem and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox, chrome, safari

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/sunspider.toml#10

• lower is better: true

• page cycles: 5

• page timeout: 55000

• test url: http://<host>:<port>/SunSpider/sunspider-1.0.1/sunspider-1.0.1/driver.html?raptor

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-sunspider	❌	❌	❌	❌
browsertime-benchmark-firefox-sunspider	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	✅	✅	❌	❌
browsertime-benchmark-safari-sunspider	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-sunspider	❌	❌	❌	❌
browsertime-benchmark-firefox-sunspider	✅	✅	❌	❌

twitch-animation

• Command to Run Locally

./mach raptor -t twitch-animation

Owner: :jrmuizel

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/twitch-animation.toml#17

• lower is better: true

• page cycles: 1

• page timeout: 2000000

• perfstats: true

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/twitch-animation

• repository revision: 61332db584026b73e37066d717a162825408c36b

• subtest lower is better: true

• subtest unit: ms

• test url: http://<host>:<port>/index.html

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	✅	✅	❌	❌

unity-webgl

• Command to Run Locally

./mach raptor -t unity-webgl

Owner: :jgilbert and Graphics(gfx) Team

• alert threshold: 2.0

• apps: firefox, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/unity-webgl-desktop.toml#14

• lower is better: false

• page cycles: 5

• page timeout: 420000

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/unity-webgl

• repository revision: 61332db584026b73e37066d717a162825408c36b

• test url: http://<host>:<port>/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-chrome-m	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-refbrow	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-refbrow	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-unity-webgl	❌	❌	❌	❌
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-unity-webgl	❌	❌	❌	❌
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

Owner: :jgilbert and Graphics(gfx) Team

• alert threshold: 2.0

• apps: geckoview, refbrow, fenix, chrome-m

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/unity-webgl-mobile.toml#14

• lower is better: false

• page cycles: 1

• page timeout: 420000

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/unity-webgl

• repository revision: 61332db584026b73e37066d717a162825408c36b

• test url: http://<host>:<port>/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-chrome-m	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-refbrow	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-fenix	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-refbrow	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-unity-webgl	❌	❌	❌	❌
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-unity-webgl	❌	❌	❌	❌
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

wasm-godot

• Command to Run Locally

./mach raptor -t wasm-godot

Owner: :lth and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox, chrome, safari

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot.toml#11

• lower is better: true

• newtab per cycle: true

• page cycles: 5

• page timeout: 120000

• test url: http://localhost:<port>/wasm-godot/index.html

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-chrome-wasm-godot	❌	❌	❌	❌
browsertime-benchmark-wasm-firefox-wasm-godot	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	✅	❌	❌	❌
browsertime-benchmark-wasm-safari-wasm-godot	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-chrome-wasm-godot	❌	❌	❌	❌
browsertime-benchmark-wasm-firefox-wasm-godot	✅	❌	❌	❌

wasm-godot-baseline

• Command to Run Locally

./mach raptor -t wasm-godot-baseline

Owner: :lth and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot-baseline.toml#14

• lower is better: true

• newtab per cycle: true

• page cycles: 5

• page timeout: 120000

• preferences: javascript.options.wasm_baselinejit=true javascript.options.wasm_optimizingjit=false

• test url: http://localhost:<port>/wasm-godot/index.html

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	✅	❌	❌	❌

wasm-godot-optimizing

• Command to Run Locally

./mach raptor -t wasm-godot-optimizing

Owner: :lth and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot-optimizing.toml#14

• lower is better: true

• newtab per cycle: true

• page cycles: 5

• page timeout: 120000

• preferences: javascript.options.wasm_baselinejit=false javascript.options.wasm_optimizingjit=true

• test url: http://localhost:<port>/wasm-godot/index.html

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	✅	❌	❌	❌

wasm-misc

• Command to Run Locally

./mach raptor -t wasm-misc

Owner: :lth and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc.toml#14

• lower is better: true

• page cycles: 5

• page timeout: 1200000

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/wasm-misc

• repository revision: 61332db584026b73e37066d717a162825408c36b

• test url: http://<host>:<port>/index.html?raptor

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-chrome-wasm-misc	❌	❌	❌	❌
browsertime-benchmark-wasm-firefox-wasm-misc	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-chrome-wasm-misc	❌	❌	❌	❌
browsertime-benchmark-wasm-firefox-wasm-misc	✅	❌	❌	❌

wasm-misc-baseline

• Command to Run Locally

./mach raptor -t wasm-misc-baseline

Owner: :lth and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc-baseline.toml#13

• lower is better: true

• page cycles: 5

• page timeout: 1200000

• preferences: javascript.options.wasm_baselinejit=true javascript.options.wasm_optimizingjit=false

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/wasm-misc

• repository revision: 61332db584026b73e37066d717a162825408c36b

• test url: http://<host>:<port>/index.html?raptor

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	✅	❌	❌	❌

wasm-misc-optimizing

• Command to Run Locally

./mach raptor -t wasm-misc-optimizing

Owner: :lth and SpiderMonkey Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc-optimizing.toml#13

• lower is better: true

• page cycles: 5

• page timeout: 1200000

• preferences: javascript.options.wasm_baselinejit=false javascript.options.wasm_optimizingjit=true

• repository: https://github.com/mozilla/perf-automation

• repository path: benchmarks/wasm-misc

• repository revision: 61332db584026b73e37066d717a162825408c36b

• test url: http://<host>:<port>/index.html?raptor

• type: benchmark

• unit: ms

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	✅	❌	❌	❌

webaudio

• Command to Run Locally

./mach raptor -t webaudio

Owner: :padenot and Media Team

• alert threshold: 2.0

• apps: firefox, chrome, safari

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/webaudio.toml#10

• lower is better: true

• page cycles: 5

• page timeout: 360000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• test url: http://<host>:<port>/webaudio/index.html?raptor

• type: benchmark

• unit: score

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-webaudio	❌	❌	❌	❌
browsertime-benchmark-firefox-webaudio	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	✅	✅	❌	❌
browsertime-benchmark-safari-webaudio	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-webaudio	❌	❌	❌	❌
browsertime-benchmark-firefox-webaudio	✅	✅	❌	❌

youtube-playback-av1-sfr

• Command to Run Locally

./mach raptor -t youtube-playback-av1-sfr

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, geckoview, fenix, refbrow, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#24

• lower is better: true

• page cycles: 1

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-av1-test&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-av1-sfr	✅	❌	❌	❌
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis	✅	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-av1-sfr	❌	❌	❌	❌
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-av1-sfr	✅	❌	❌	❌

youtube-playback-h264-1080p30

• Command to Run Locally

./mach raptor -t youtube-playback-h264-1080p30

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#28

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-h264-test&tests=18&raptor=true&muted=true&command=run&exclude=1,2

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

youtube-playback-h264-1080p60

• Command to Run Locally

./mach raptor -t youtube-playback-h264-1080p60

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#34

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&tests=46&muted=true&command=run&exclude=1,2

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

youtube-playback-h264-720p60

• Command to Run Locally

./mach raptor -t youtube-playback-h264-720p60

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, chrome, geckoview, fenix, refbrow, chrome-m

• browsertime args: –pageCompleteWaitTime=10 –pageCompleteCheckPollTimeout=10 –browsertime.benchmark_wait_time=500

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#82

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• support class: browsertime_benchmark.py

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&tests=38&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-mobile-chrome-m-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-chrome-m-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-h264-720p60	✅	✅	❌	❌
browsertime-power-mobile-fenix-youtube-playback-h264-720p60-nofis	✅	✅	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-mobile-fenix-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌

test-android-hw-p6-13-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-mobile-chrome-m-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-chrome-m-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌

test-android-hw-s24-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-mobile-chrome-m-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-chrome-m-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-h264-720p60	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-h264-720p60-nofis	❌	❌	❌	❌

youtube-playback-h264-full-1080p30

• Command to Run Locally

./mach raptor -t youtube-playback-h264-full-1080p30

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#40

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-h264-test&tests=18&raptor=true&muted=true&command=run&fullscreen=true&exclude=1,2

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

youtube-playback-h264-full-1080p60

• Command to Run Locally

./mach raptor -t youtube-playback-h264-full-1080p60

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#55

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&tests=46&muted=true&command=run&fullscreen=true&exclude=1,2

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

youtube-playback-h264-sfr

• Command to Run Locally

./mach raptor -t youtube-playback-h264-sfr

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, geckoview, fenix, refbrow, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#70

• lower is better: true

• page cycles: 1

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-h264-test&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-geckoview-youtube-playback-h264-sfr	❌	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis	❌	❌	❌	❌
browsertime-mobile-refbrow-youtube-playback-h264-sfr	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-geckoview-youtube-playback-h264-sfr	❌	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis	❌	❌	❌	❌
browsertime-mobile-refbrow-youtube-playback-h264-sfr	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	✅	❌	❌	❌

youtube-playback-hfr

• Command to Run Locally

./mach raptor -t youtube-playback-hfr

Owner: PerfTest Team

• alert on: H2641080p60fps@1X_dropped_frames, H264720p60fps@1X_dropped_frames

• alert threshold: 2.0

• apps: firefox, geckoview, fenix, refbrow, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#74

• lower is better: true

• page cycles: 1

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-hfr	✅	❌	❌	❌
browsertime-mobile-fenix-youtube-playback-hfr-nofis	✅	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-hfr	❌	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-hfr-nofis	❌	❌	❌	❌
browsertime-mobile-refbrow-youtube-playback-hfr	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-hfr	❌	❌	❌	❌
browsertime-mobile-fenix-youtube-playback-hfr-nofis	❌	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-hfr	❌	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-hfr-nofis	❌	❌	❌	❌
browsertime-mobile-refbrow-youtube-playback-hfr	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	✅	❌	❌	❌

youtube-playback-v9-1080p30

• Command to Run Locally

./mach raptor -t youtube-playback-v9-1080p30

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#96

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-vp9-test&raptor=true&tests=18&muted=true&command=run&exclude=1,2

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

youtube-playback-v9-1080p60

• Command to Run Locally

./mach raptor -t youtube-playback-v9-1080p60

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#102

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&tests=14&muted=true&command=run&exclude=1,2

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

youtube-playback-v9-full-1080p30

• Command to Run Locally

./mach raptor -t youtube-playback-v9-full-1080p30

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#108

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-vp9-test&raptor=true&tests=18&muted=true&command=run&fullscreen=true&exclude=1,2

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

youtube-playback-v9-full-1080p60

• Command to Run Locally

./mach raptor -t youtube-playback-v9-full-1080p60

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox

• expected: pass

• expose browser profiler: true

• gather cpuTime: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#123

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&tests=14&muted=true&command=run&fullscreen=true&exclude=1,2

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

youtube-playback-vp9-720p60

• Command to Run Locally

./mach raptor -t youtube-playback-vp9-720p60

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, chrome, geckoview, fenix, refbrow, chrome-m

• browsertime args: –pageCompleteWaitTime=10 –pageCompleteCheckPollTimeout=10 –browsertime.benchmark_wait_time=500

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#89

• lower is better: true

• page cycles: 20

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• support class: browsertime_benchmark.py

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&tests=6&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-mobile-chrome-m-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-chrome-m-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-vp9-720p60	✅	✅	❌	❌
browsertime-power-mobile-fenix-youtube-playback-vp9-720p60-nofis	✅	✅	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-mobile-fenix-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌

test-android-hw-p6-13-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-mobile-chrome-m-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-chrome-m-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌

test-android-hw-s24-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-mobile-chrome-m-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-chrome-m-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-fenix-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-vp9-720p60	❌	❌	❌	❌
browsertime-power-mobile-geckoview-youtube-playback-vp9-720p60-nofis	❌	❌	❌	❌

youtube-playback-vp9-sfr

• Command to Run Locally

./mach raptor -t youtube-playback-vp9-sfr

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, geckoview, fenix, refbrow, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#138

• lower is better: true

• page cycles: 1

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-vp9-test&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-android-hw-a55-14-0-aarch64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-vp9-sfr	✅	❌	❌	❌
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis	✅	❌	❌	❌

test-android-hw-a55-14-0-aarch64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-vp9-sfr	❌	❌	❌	❌
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis	❌	❌	❌	❌

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	✅	❌	❌	❌

youtube-playback-widevine-h264-sfr

• Command to Run Locally

./mach raptor -t youtube-playback-widevine-h264-sfr

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, geckoview, fenix, refbrow, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#142

• lower is better: true

• page cycles: 1

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-widevine-sfr-h264-test&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	✅	❌	❌	❌

youtube-playback-widevine-hfr

• Command to Run Locally

./mach raptor -t youtube-playback-widevine-hfr

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, geckoview, fenix, refbrow, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#156

• lower is better: true

• page cycles: 1

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-widevine-hfr-test&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	✅	❌	❌	❌

youtube-playback-widevine-vp9-sfr

• Command to Run Locally

./mach raptor -t youtube-playback-widevine-vp9-sfr

Owner: PerfTest Team

• alert threshold: 2.0

• apps: firefox, geckoview, fenix, refbrow, chrome

• expected: pass

• expose browser profiler: true

• gecko profile interval: 1000

• gecko profile threads: MediaPlayback

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#170

• lower is better: true

• page cycles: 1

• page timeout: 2700000

• preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false

• subtest lower is better: true

• subtest unit: score

• test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-widevine-sfr-vp9-test&raptor=true&exclude=1,2&muted=true&command=run

• type: benchmark

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-clang-trunk-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	✅	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	✅	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	✅	❌	❌	❌

Custom

Browsertime tests that use a custom pageload test script. These use the pageload type, but may have other intentions.

addMab1

• Command to Run Locally

./mach raptor -t addMab1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small ArrayBuffers to IndexedDB in one transaction

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=1

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#30

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	✅	✅	❌	❌

addMabN

• Command to Run Locally

./mach raptor -t addMabN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small ArrayBuffers to IndexedDB independently

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=0

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#34

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	✅	✅	❌	❌

addMar1

• Command to Run Locally

./mach raptor -t addMar1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small Arrays to IndexedDB in one transaction

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Array –browsertime.atomic=1

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#38

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	✅	✅	❌	❌

addMarN

• Command to Run Locally

./mach raptor -t addMarN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small Arrays to IndexedDB independently

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Array –browsertime.atomic=0

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#42

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	✅	✅	❌	❌

addMbl1

• Command to Run Locally

./mach raptor -t addMbl1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small Blobs to IndexedDB in one transaction

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Blob –browsertime.atomic=1

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#46

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	✅	✅	❌	❌

addMblN

• Command to Run Locally

./mach raptor -t addMblN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small Blobs to IndexedDB independently

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Blob –browsertime.atomic=0

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#50

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	✅	✅	❌	❌

addkAB1

• Command to Run Locally

./mach raptor -t addkAB1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large ArrayBuffers to IndexedDB in one transaction

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=1

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#54

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	✅	✅	❌	❌

addkABN

• Command to Run Locally

./mach raptor -t addkABN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large ArrayBuffers to IndexedDB independently

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=0

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#58

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	✅	✅	❌	❌

addkAR1

• Command to Run Locally

./mach raptor -t addkAR1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large JS Arrays to IndexedDB in one transaction

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Array –browsertime.atomic=1

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#62

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	✅	✅	❌	❌

addkARN

• Command to Run Locally

./mach raptor -t addkARN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large JS Arrays to IndexedDB independently

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Array –browsertime.atomic=0

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#66

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	✅	✅	❌	❌

addkBL1

• Command to Run Locally

./mach raptor -t addkBL1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large Blobs to IndexedDB in one transaction

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Blob –browsertime.atomic=1

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#70

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	✅	✅	❌	❌

addkBLN

• Command to Run Locally

./mach raptor -t addkBLN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large Blobs to IndexedDB independently

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Blob –browsertime.atomic=0

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#74

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• test script: indexeddb_write.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	✅	✅	❌	❌

browsertime

• Command to Run Locally

./mach raptor -t browsertime

Owner: PerfTest Team

Description: Used to run vanilla browsertime tests through raptor. For example: ./mach raptor -t browsertime –browsertime-arg url=https://www.sitespeed.io –browsertime-arg iterations=3

• alert on: fcp, loadtime

• alert threshold: 2.0

• apps: firefox

• browser cycles: 1

• expected: pass

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-custom.toml#17

• lower is better: true

• measure: fnbpaint, fcp, dcf, loadtime

• page cycles: 1

• page timeout: 60000

• playback: mitmproxy

• playback pageset manifest: null.manifest

• playback version: 11.0.0

• support class: browsertime_pageload.py

• test script: None

• test url: None

• type: pageload

• unit: ms

• use live sites: true

connect

• Command to Run Locally

./mach raptor -t connect

Owner: Network Team

Description: Verifies that connection time on a pageload is zero when a speculative connection has first been made

• alert change type: absolute

• alert on: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange

• alert threshold: 10.0

• apps: firefox

• browser cycles: 3

• cold: true

• custom data: true

• expected: pass

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-speculative.toml#18

• lower is better: true

• page cycles: 1

• page timeout: 60000

• test script: speculative-connect.js

• test url: None

• type: pageload

• unit: ms

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	✅	❌	❌

constant-regression

• Command to Run Locally

./mach raptor -t constant-regression

Owner: PerfTest Team

Description: Generates a constant value that can be changed to induce a regression.

• alert threshold: 2.0

• apps: firefox, chrome, safari, geckoview, fenix

• browser cycles: 1

• browsertime args: –browsertime.constant_value=1500

• custom data: true

• expected: pass

• fore window: 1

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-regression-test.toml#23

• lower is better: true

• max back window: 3

• min back window: 2

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• subtest unit: ms

• test script: constant_regression_test.js

• test summary: flatten

• test url: None

• type: pageload

• unit: score

• use live sites: true

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-regression-tests-firefox-constant-regression	❌	✅	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-regression-tests-firefox-constant-regression	❌	✅	❌	❌

getkeyrng

• Command to Run Locally

./mach raptor -t getkeyrng

Owner: DOM Lifecycle, Workers and Storage Team

Description: Measures the speed of key-range based reads of IndexedDB

• alert threshold: 2.0

• apps: firefox, chrome, safari

• browser cycles: 1

• custom data: true

• expected: pass

• gecko profile entries: 131072000

• gecko profile features: js,stackwalk,cpu,memory

• gecko profile threads: GeckoMain,DOM Worker,IndexedDB

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#78

• lower is better: true

• measure: cpuTime

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• playback: mitmproxy

• playback pageset manifest: mitm8-linux-firefox-example.manifest

• subtest unit: ms

• support class: browsertime_pageload.py

• test script: indexeddb_getkeyrange.js

• test summary: flatten

• test url: https://www.example.com

• type: pageload

• unit: ms

• use live sites: false

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	✅	✅	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	✅	✅	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	✅	✅	❌	❌

h3-download

• Command to Run Locally

./mach raptor -t h3-download

Owner: Network Team

Description: Measures HTTP/3 download throughput with a local server

• alert threshold: 2.0

• apps: firefox, chrome, custom-car

• browser cycles: 1

• browsertime args: –browsertime.test_type=h3_download –browsertime.iterations=10 –firefox.preference=network.http.http3.enable:true

• custom data: true

• expected: pass

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-network-bench.toml#28

• lower is better: false

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• subtest lower is better: false

• subtest name filters: tests/,iterations/

• subtest unit: mbps

• support class: network_bench.py

• test script: network_bench.js

• test url: None

• type: pageload

• unit: mbps

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-chrome-h3-download	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

test-windows11-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

test-windows11-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

test-windows11-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-chrome-h3-download	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-download-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download-300M_80ms-0	❌	❌	❌	❌

h3-upload

• Command to Run Locally

./mach raptor -t h3-upload

Owner: Network Team

Description: Measures HTTP/3 file upload throughput with a local server

• alert threshold: 2.0

• apps: firefox, chrome, custom-car

• browser cycles: 1

• browsertime args: –browsertime.test_type=h3_upload –browsertime.iterations=10 –firefox.preference=network.http.http3.enable:true

• custom data: true

• expected: pass

• gecko profile interval: 1

• link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-network-bench.toml#24

• lower is better: false

• output timeout: 2000000

• page cycles: 1

• page timeout: 1800000

• subtest lower is better: false

• subtest name filters: tests/,iterations/

• subtest unit: mbps

• support class: network_bench.py

• test script: network_bench.js

• test url: None

• type: pageload

• unit: mbps

• Test Task:

test-linux1804-64-nightlyasrelease-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-1	❌	❌	❌	❌

test-linux1804-64-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-1	❌	❌	❌	❌

test-linux1804-64-shippable-qr/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-chrome-h3-upload	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-100M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-100M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-100M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-100M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-10M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-10M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-10M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-10M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-1M_400ms-0.01	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-1M_400ms-0.1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-1M_400ms-0.5	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-1M_400ms-1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_80ms-0.01	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_80ms-0.1	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_80ms-0.5	❌	❌	❌	❌
browsertime-network-bench-chrome-h3-upload-300M_80ms-1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-100M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-100M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-100M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-100M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-10M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-10M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-10M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-10M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-1M_400ms-0.01	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-1M_400ms-0.1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-1M_400ms-0.5	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-1M_400ms-1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_80ms-0.01	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_80ms-0.1	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_80ms-0.5	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload-300M_80ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-1	❌	❌	❌	❌

test-macosx1470-64-nightlyasrelease/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-1	❌	❌	❌	❌

test-macosx1470-64-shippable/opt

Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-100M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-10M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-1M_400ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_40ms-1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.01	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.1	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-0.5	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload-300M_80ms-1	❌	❌	❌	❌

test-macosx1470-64/opt

Test Name	mozilla-central	autoland