Raptor¶

Raptor is a performance-testing framework for running browser pageload and browser benchmark tests. Raptor is cross-browser compatible and is currently running in production on Firefox Desktop, Firefox Android GeckoView, Fenix, Reference Browser, Chromium, Chrome, Chrome for Android, Safari, and Safari Technology Preview.

Contact: Dave Hunt [:davehunt]
Source code: https://searchfox.org/mozilla-central/source/testing/raptor
Good first bugs: https://codetribute.mozilla.org/projects/automation?project%3DRaptor

Raptor currently supports three test types: 1) page-load performance tests, 2) standard benchmark-performance tests, and 3) “scenario”-based tests, such as power, CPU, and memory-usage measurements on Android (and desktop?).

Locally, Raptor can be invoked with the following command:

./mach raptor

Raptor tests ¶

The following documents all testing we have for Raptor.

Benchmarks ¶

Standard benchmarks are third-party tests (i.e. Speedometer) that we have integrated into Raptor to run per-commit in our production CI. To update any of these benchmarks, see Updating Benchmark Tests.

ares6

Command to Run Locally

./mach raptor -t ares6

Owner: :jandem and SpiderMonkey Team

alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/ares6.toml#16
lower is better: true
page cycles: 4
page timeout: 270000
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/ARES-6/index.html?raptor
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-ares6	❌	❌	❌	❌
browsertime-benchmark-firefox-ares6	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	✅	✅	❌	❌
browsertime-benchmark-safari-ares6	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-ares6	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-ares6	❌	❌	❌	❌
browsertime-benchmark-firefox-ares6	✅	✅	❌	❌

assorted-dom

Command to Run Locally

./mach raptor -t assorted-dom

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/assorted-dom.toml#15
lower is better: true
page cycles: 1
page timeout: 60000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/assorted-dom
repository revision: 61332db584026b73e37066d717a162825408c36b
screen capture: true
test url: http://<host>:<port>/assorted/driver.html?raptor
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-assorted-dom	❌	❌	❌	❌
browsertime-benchmark-firefox-assorted-dom	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	✅	✅	❌	❌
browsertime-benchmark-safari-assorted-dom	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-assorted-dom	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-assorted-dom	❌	❌	❌	❌
browsertime-benchmark-firefox-assorted-dom	✅	✅	❌	❌

jetstream2

Command to Run Locally

./mach raptor -t jetstream2

Owner: :jandem and SpiderMonkey Team

alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/jetstream2.toml#16
lower is better: false
page cycles: 4
page timeout: 2000000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/JetStream2
repository revision: 61332db584026b73e37066d717a162825408c36b
subtest lower is better: false
subtest unit: score
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: score
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-jetstream2	❌	❌	❌	❌
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌
browsertime-benchmark-safari-jetstream2	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-jetstream2	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-jetstream2	❌	❌	❌	❌
browsertime-benchmark-firefox-jetstream2	✅	✅	❌	❌

matrix-react-bench

Command to Run Locally

./mach raptor -t matrix-react-bench

Owner: :jandem and SpiderMonkey Team

alert threshold: 2.0
apps: firefox, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/matrix-react-bench.toml#16
lower is better: true
page cycles: 30
page timeout: 2000000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/matrix-react-bench
repository revision: 61332db584026b73e37066d717a162825408c36b
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/matrix_demo.html
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-matrix-react-bench	❌	❌	❌	❌
browsertime-benchmark-firefox-matrix-react-bench	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-matrix-react-bench	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-matrix-react-bench	❌	❌	❌	❌
browsertime-benchmark-firefox-matrix-react-bench	✅	✅	❌	❌

motionmark-1-3

Command to Run Locally

./mach raptor -t motionmark-1-3

Owner: Graphics Team

alert threshold: 2.0
apps: firefox, chrome, safari, custom-car
browser cycles: 1
cold: true
expected: pass
expose browser profiler: true
expose chrome trace: true
gecko profile features: stackwalk,js,cpu,java,processcpu,memory
gecko profile interval: 1
gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-desktop.toml#12
lower is better: false
page cycles: 1
page timeout: 600000
repository: https://github.com/webkit/motionmark
repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb
suite name: MotionMark
support class: motionmark-1-3.py
test script: motionmark-1-3.js
test url: http://<host>:<port>/MotionMark/developer.html?warmup-length=2000&warmup-frame-count=30&first-frame-minimum-length=0&test-interval=30&display=minimal&tiles=big&controller=ramp&system-frame-rate=60&frame-rate=60&time-measurement=performance
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-1-3	❌	❌	❌	❌

**test-macosx1400-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-1-3	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌

Owner: Graphics Team

alert threshold: 2.0
apps: fenix, geckoview, chrome-m, cstm-car-m
browser cycles: 1
cold: true
expected: pass
expose browser profiler: true
gecko profile features: stackwalk,js,cpu,java,processcpu,memory
gecko profile interval: 1
gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-mobile.toml#11
lower is better: false
page cycles: 1
page timeout: 600000
repository: https://github.com/webkit/motionmark
repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb
suite name: MotionMark
support class: motionmark-1-3.py
test script: motionmark-1-3.js
test url: http://<host>:<port>/MotionMark/developer.html?warmup-length=2000&warmup-frame-count=30&first-frame-minimum-length=0&test-interval=30&display=minimal&tiles=big&controller=ramp&system-frame-rate=60&frame-rate=60&time-measurement=performance
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-1-3	❌	❌	❌	❌

**test-macosx1400-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-1-3	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-1-3	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-1-3	✅	✅	❌	❌

motionmark-htmlsuite-1-3

Command to Run Locally

./mach raptor -t motionmark-htmlsuite-1-3

Owner: Graphics Team

alert threshold: 2.0
apps: firefox, chrome, safari, custom-car
browser cycles: 1
cold: true
expected: pass
expose browser profiler: true
expose chrome trace: true
gecko profile features: stackwalk,js,cpu,java,processcpu,memory
gecko profile interval: 1
gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-desktop.toml#27
lower is better: false
page cycles: 1
page timeout: 600000
repository: https://github.com/webkit/motionmark
repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb
suite name: HTML suite
support class: motionmark-1-3.py
test script: motionmark-1-3.js
test url: http://<host>:<port>/MotionMark/developer.html?warmup-length=2000&warmup-frame-count=30&first-frame-minimum-length=0&test-interval=30&display=minimal&tiles=big&controller=ramp&system-frame-rate=60&frame-rate=60&time-measurement=performance
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-macosx1400-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌

Owner: Graphics Team

alert threshold: 2.0
apps: fenix, geckoview, chrome-m, cstm-car-m
browser cycles: 1
cold: true
expected: pass
expose browser profiler: true
gecko profile features: stackwalk,js,cpu,java,processcpu,memory
gecko profile interval: 1
gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-mobile.toml#26
lower is better: false
page cycles: 1
page timeout: 600000
repository: https://github.com/webkit/motionmark
repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb
suite name: HTML suite
support class: motionmark-1-3.py
test script: motionmark-1-3.js
test url: http://<host>:<port>/MotionMark/developer.html?warmup-length=2000&warmup-frame-count=30&first-frame-minimum-length=0&test-interval=30&display=minimal&tiles=big&controller=ramp&system-frame-rate=60&frame-rate=60&time-measurement=performance
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis	❌	❌	❌	❌
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-macosx1400-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌
browsertime-benchmark-safari-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3	❌	❌	❌	❌
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3	✅	✅	❌	❌

speedometer

Command to Run Locally

./mach raptor -t speedometer

Owner: SpiderMonkey Team

alert threshold: 2.0
apps: firefox, chrome, safari, safari-tp, custom-car
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-desktop.toml#17
lower is better: false
page cycles: 5
page timeout: 600000
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/Speedometer/index.html?raptor
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	✅	✅	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	✅	✅	❌	❌

**test-android-hw-p6-13-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-s24-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer	❌	❌	❌	❌

**test-macosx1400-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	❌	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌

Owner: SpiderMonkey Team

alert threshold: 2.0
apps: fenix, geckoview, refbrow, chrome-m, cstm-car-m
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-mobile.toml#17
lower is better: false
page cycles: 5
page timeout: 420000
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/Speedometer/index.html?raptor
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	✅	✅	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	✅	✅	❌	❌

**test-android-hw-p6-13-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-s24-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer-mobile-geckoview-nofis	❌	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer	❌	❌	❌	❌

**test-macosx1400-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	❌	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	✅	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer	✅	✅	❌	❌

speedometer3

Command to Run Locally

./mach raptor -t speedometer3

Owner: Performance Team

alert threshold: 2.0
apps: firefox, chrome, safari, safari-tp, custom-car
browser cycles: 5
cold: true
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-desktop.toml#19
lower is better: false
page cycles: 5
page timeout: 600000
subtest lower is better: true
subtest unit: ms
support class: speedometer3.py
test script: speedometer3.js
test url: http://<host>:<port>/Speedometer3/index.html?raptor
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	✅	✅	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	✅	✅	❌	❌

**test-android-hw-p6-13-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	✅	❌	❌	❌

**test-android-hw-s24-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-macosx1400-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	❌	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer3	❌	❌	❌	❌
browsertime-benchmark-safari-tp-speedometer3	✅	❌	❌	❌

**test-windows10-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

Owner: Performance Team

alert threshold: 2.0
apps: fenix, geckoview, refbrow, chrome-m, cstm-car-m
browser cycles: 5
cold: true
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-mobile.toml#19
lower is better: false
page cycles: 5
page timeout: 420000
repository: https://github.com/WebKit/Speedometer
repository revision: 8d67f28d0281ac4330f283495b7f48286654ad7d
subtest lower is better: true
subtest unit: ms
support class: speedometer3.py
test script: speedometer3.js
test url: http://<host>:<port>/index.html
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	✅	✅	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	❌	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	✅	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	✅	✅	❌	❌

**test-android-hw-p6-13-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	✅	❌	❌	❌

**test-android-hw-s24-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-speedometer3-mobile-geckoview-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-macosx1400-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	❌	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌
browsertime-benchmark-safari-speedometer3	❌	❌	❌	❌
browsertime-benchmark-safari-tp-speedometer3	✅	❌	❌	❌

**test-windows10-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-speedometer3	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-speedometer3	❌	❌	❌	❌
browsertime-benchmark-custom-car-speedometer3	✅	❌	❌	❌
browsertime-benchmark-firefox-speedometer3	✅	✅	❌	❌

stylebench

Command to Run Locally

./mach raptor -t stylebench

Owner: :emelio and Layout Team

alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/stylebench.toml#16
lower is better: false
page cycles: 5
page timeout: 140000
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/StyleBench/index.html?raptor
type: benchmark
unit: score
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-stylebench	❌	❌	❌	❌
browsertime-benchmark-firefox-stylebench	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	✅	✅	❌	❌
browsertime-benchmark-safari-stylebench	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-stylebench	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-stylebench	❌	❌	❌	❌
browsertime-benchmark-firefox-stylebench	✅	✅	❌	❌

sunspider

Command to Run Locally

./mach raptor -t sunspider

Owner: :jandem and SpiderMonkey Team

alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/sunspider.toml#10
lower is better: true
page cycles: 5
page timeout: 55000
test url: http://<host>:<port>/SunSpider/sunspider-1.0.1/sunspider-1.0.1/driver.html?raptor
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-sunspider	❌	❌	❌	❌
browsertime-benchmark-firefox-sunspider	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	✅	✅	❌	❌
browsertime-benchmark-safari-sunspider	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-sunspider	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-sunspider	❌	❌	❌	❌
browsertime-benchmark-firefox-sunspider	✅	✅	❌	❌

twitch-animation

Command to Run Locally

./mach raptor -t twitch-animation

Owner: :jrmuizel

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/twitch-animation.toml#17
lower is better: true
page cycles: 1
page timeout: 2000000
perfstats: true
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/twitch-animation
repository revision: 61332db584026b73e37066d717a162825408c36b
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/index.html
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-twitch-animation	✅	✅	❌	❌

unity-webgl

Command to Run Locally

./mach raptor -t unity-webgl

Owner: :jgilbert and Graphics(gfx) Team

alert threshold: 2.0
apps: firefox, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/unity-webgl-desktop.toml#14
lower is better: false
page cycles: 5
page timeout: 420000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/unity-webgl
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-unity-webgl	❌	❌	❌	❌
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-unity-webgl	❌	❌	❌	❌
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

Owner: :jgilbert and Graphics(gfx) Team

alert threshold: 2.0
apps: geckoview, refbrow, fenix, chrome-m
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/unity-webgl-mobile.toml#14
lower is better: false
page cycles: 1
page timeout: 420000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/unity-webgl
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: score
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	❌	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-fenix-nofis	✅	❌	❌	❌
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-unity-webgl	❌	❌	❌	❌
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-unity-webgl	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-unity-webgl	❌	❌	❌	❌
browsertime-benchmark-firefox-unity-webgl	✅	❌	❌	❌

wasm-godot

Command to Run Locally

./mach raptor -t wasm-godot

Owner: :lth and SpiderMonkey Team

alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot.toml#11
lower is better: true
newtab per cycle: true
page cycles: 5
page timeout: 120000
test url: http://localhost:<port>/wasm-godot/index.html
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-chrome-wasm-godot	❌	❌	❌	❌
browsertime-benchmark-wasm-firefox-wasm-godot	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	✅	❌	❌	❌
browsertime-benchmark-wasm-safari-wasm-godot	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-chrome-wasm-godot	❌	❌	❌	❌
browsertime-benchmark-wasm-firefox-wasm-godot	✅	❌	❌	❌

wasm-godot-baseline

Command to Run Locally

./mach raptor -t wasm-godot-baseline

Owner: :lth and SpiderMonkey Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot-baseline.toml#14
lower is better: true
newtab per cycle: true
page cycles: 5
page timeout: 120000
preferences: javascript.options.wasm_baselinejit=true javascript.options.wasm_optimizingjit=false
test url: http://localhost:<port>/wasm-godot/index.html
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-baseline	✅	❌	❌	❌

wasm-godot-optimizing

Command to Run Locally

./mach raptor -t wasm-godot-optimizing

Owner: :lth and SpiderMonkey Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot-optimizing.toml#14
lower is better: true
newtab per cycle: true
page cycles: 5
page timeout: 120000
preferences: javascript.options.wasm_baselinejit=false javascript.options.wasm_optimizingjit=true
test url: http://localhost:<port>/wasm-godot/index.html
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing	✅	❌	❌	❌

wasm-misc

Command to Run Locally

./mach raptor -t wasm-misc

Owner: :lth and SpiderMonkey Team

alert threshold: 2.0
apps: firefox, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc.toml#14
lower is better: true
page cycles: 5
page timeout: 1200000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/wasm-misc
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-chrome-wasm-misc	❌	❌	❌	❌
browsertime-benchmark-wasm-firefox-wasm-misc	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-chrome-wasm-misc	❌	❌	❌	❌
browsertime-benchmark-wasm-firefox-wasm-misc	✅	❌	❌	❌

wasm-misc-baseline

Command to Run Locally

./mach raptor -t wasm-misc-baseline

Owner: :lth and SpiderMonkey Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc-baseline.toml#13
lower is better: true
page cycles: 5
page timeout: 1200000
preferences: javascript.options.wasm_baselinejit=true javascript.options.wasm_optimizingjit=false
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/wasm-misc
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-baseline	✅	❌	❌	❌

wasm-misc-optimizing

Command to Run Locally

./mach raptor -t wasm-misc-optimizing

Owner: :lth and SpiderMonkey Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc-optimizing.toml#13
lower is better: true
page cycles: 5
page timeout: 1200000
preferences: javascript.options.wasm_baselinejit=false javascript.options.wasm_optimizingjit=true
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/wasm-misc
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: ms
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing	✅	❌	❌	❌

webaudio

Command to Run Locally

./mach raptor -t webaudio

Owner: :padenot and Media Team

alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/webaudio.toml#10
lower is better: true
page cycles: 5
page timeout: 360000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
test url: http://<host>:<port>/webaudio/index.html?raptor
type: benchmark
unit: score
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-webaudio	❌	❌	❌	❌
browsertime-benchmark-firefox-webaudio	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	✅	✅	❌	❌
browsertime-benchmark-safari-webaudio	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-firefox-webaudio	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-benchmark-chrome-webaudio	❌	❌	❌	❌
browsertime-benchmark-firefox-webaudio	✅	✅	❌	❌

youtube-playback-av1-sfr

Command to Run Locally

./mach raptor -t youtube-playback-av1-sfr

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#24
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-av1-test&raptor=true&exclude=1,2&muted=true&command=run
type: benchmark
unit: score
use live sites: true
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis	✅	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-av1-sfr	✅	❌	❌	❌

youtube-playback-h264-1080p30

Command to Run Locally

./mach raptor -t youtube-playback-h264-1080p30

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#28
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-h264-test&tests=18&raptor=true&muted=true&command=run&exclude=1,2
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p30	❌	❌	❌	❌

youtube-playback-h264-1080p60

Command to Run Locally

./mach raptor -t youtube-playback-h264-1080p60

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#34
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&tests=46&muted=true&command=run&exclude=1,2
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-1080p60	❌	❌	❌	❌

youtube-playback-h264-full-1080p30

Command to Run Locally

./mach raptor -t youtube-playback-h264-full-1080p30

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#40
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-h264-test&tests=18&raptor=true&muted=true&command=run&fullscreen=true&exclude=1,2
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p30	❌	❌	❌	❌

youtube-playback-h264-full-1080p60

Command to Run Locally

./mach raptor -t youtube-playback-h264-full-1080p60

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#55
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&tests=46&muted=true&command=run&fullscreen=true&exclude=1,2
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-h264-full-1080p60	❌	❌	❌	❌

youtube-playback-h264-sfr

Command to Run Locally

./mach raptor -t youtube-playback-h264-sfr

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#70
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-h264-test&raptor=true&exclude=1,2&muted=true&command=run
type: benchmark
unit: score
use live sites: true
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-h264-sfr	✅	❌	❌	❌

youtube-playback-hfr

Command to Run Locally

./mach raptor -t youtube-playback-hfr

Owner: PerfTest Team

alert on: H2641080p60fps@1X_dropped_frames, H264720p60fps@1X_dropped_frames
alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#74
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&exclude=1,2&muted=true&command=run
type: benchmark
unit: score
use live sites: true
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-hfr-nofis	❌	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-hfr-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-hfr-nofis	✅	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-hfr-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-hfr-nofis	❌	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-hfr-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-hfr-nofis	✅	❌	❌	❌
browsertime-mobile-geckoview-youtube-playback-hfr-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-hfr	✅	❌	❌	❌

youtube-playback-v9-1080p30

Command to Run Locally

./mach raptor -t youtube-playback-v9-1080p30

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#82
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-vp9-test&raptor=true&tests=18&muted=true&command=run&exclude=1,2
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p30	❌	❌	❌	❌

youtube-playback-v9-1080p60

Command to Run Locally

./mach raptor -t youtube-playback-v9-1080p60

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#88
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&tests=14&muted=true&command=run&exclude=1,2
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-1080p60	❌	❌	❌	❌

youtube-playback-v9-full-1080p30

Command to Run Locally

./mach raptor -t youtube-playback-v9-full-1080p30

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#94
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-vp9-test&raptor=true&tests=18&muted=true&command=run&fullscreen=true&exclude=1,2
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p30	❌	❌	❌	❌

youtube-playback-v9-full-1080p60

Command to Run Locally

./mach raptor -t youtube-playback-v9-full-1080p60

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#109
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-hfr-test&raptor=true&tests=14&muted=true&command=run&fullscreen=true&exclude=1,2
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-power-firefox-youtube-playback-v9-full-1080p60	❌	❌	❌	❌

youtube-playback-vp9-sfr

Command to Run Locally

./mach raptor -t youtube-playback-vp9-sfr

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#124
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-sfr-vp9-test&raptor=true&exclude=1,2&muted=true&command=run
type: benchmark
unit: score
use live sites: true
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis	✅	❌	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis	✅	❌	❌	❌

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-vp9-sfr	✅	❌	❌	❌

youtube-playback-widevine-h264-sfr

Command to Run Locally

./mach raptor -t youtube-playback-widevine-h264-sfr

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#128
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-widevine-sfr-h264-test&raptor=true&exclude=1,2&muted=true&command=run
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-h264-sfr	✅	❌	❌	❌

youtube-playback-widevine-hfr

Command to Run Locally

./mach raptor -t youtube-playback-widevine-hfr

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#142
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-widevine-hfr-test&raptor=true&exclude=1,2&muted=true&command=run
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-hfr	✅	❌	❌	❌

youtube-playback-widevine-vp9-sfr

Command to Run Locally

./mach raptor -t youtube-playback-widevine-vp9-sfr

Owner: PerfTest Team

alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#156
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false
subtest lower is better: true
subtest unit: score
test url: https://prod.youtube-test.prod.webservices.mozgcp.net/main.html?test_type=playbackperf-widevine-sfr-vp9-test&raptor=true&exclude=1,2&muted=true&command=run
type: benchmark
unit: score
use live sites: true
Test Task:

**test-linux1804-64-clang-trunk-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-firefox-youtube-playback-widevine-vp9-sfr	✅	❌	❌	❌

Custom ¶

Browsertime tests that use a custom pageload test script. These use the pageload type, but may have other intentions.

addMab1

Command to Run Locally

./mach raptor -t addMab1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small ArrayBuffers to IndexedDB in one transaction

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#30
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMab1	✅	✅	❌	❌

addMabN

Command to Run Locally

./mach raptor -t addMabN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small ArrayBuffers to IndexedDB independently

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#34
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMabN	✅	✅	❌	❌

addMar1

Command to Run Locally

./mach raptor -t addMar1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small Arrays to IndexedDB in one transaction

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Array –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#38
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMar1	✅	✅	❌	❌

addMarN

Command to Run Locally

./mach raptor -t addMarN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small Arrays to IndexedDB independently

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Array –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#42
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMarN	✅	✅	❌	❌

addMbl1

Command to Run Locally

./mach raptor -t addMbl1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small Blobs to IndexedDB in one transaction

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Blob –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#46
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMbl1	✅	✅	❌	❌

addMblN

Command to Run Locally

./mach raptor -t addMblN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send many small Blobs to IndexedDB independently

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Blob –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#50
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addMblN	✅	✅	❌	❌

addkAB1

Command to Run Locally

./mach raptor -t addkAB1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large ArrayBuffers to IndexedDB in one transaction

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#54
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAB1	✅	✅	❌	❌

addkABN

Command to Run Locally

./mach raptor -t addkABN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large ArrayBuffers to IndexedDB independently

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#58
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkABN	✅	✅	❌	❌

addkAR1

Command to Run Locally

./mach raptor -t addkAR1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large JS Arrays to IndexedDB in one transaction

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Array –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#62
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkAR1	✅	✅	❌	❌

addkARN

Command to Run Locally

./mach raptor -t addkARN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large JS Arrays to IndexedDB independently

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Array –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#66
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkARN	✅	✅	❌	❌

addkBL1

Command to Run Locally

./mach raptor -t addkBL1

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large Blobs to IndexedDB in one transaction

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Blob –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#70
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBL1	✅	✅	❌	❌

addkBLN

Command to Run Locally

./mach raptor -t addkBLN

Owner: DOM Lifecycle, Workers and Storage Team

Description: Use add API to send a couple of large Blobs to IndexedDB independently

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Blob –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#74
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-addkBLN	✅	✅	❌	❌

connect

Command to Run Locally

./mach raptor -t connect

Owner: Network Team

Description: Verifies that connection time on a pageload is zero when a speculative connection has first been made

alert change type: absolute
alert on: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange
alert threshold: 10.0
apps: firefox
browser cycles: 3
cold: true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-speculative.toml#18
lower is better: true
page cycles: 1
page timeout: 60000
test script: speculative-connect.js
test url: None
type: pageload
unit: ms
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-speculative-firefox-connect	❌	✅	❌	❌

constant-regression

Command to Run Locally

./mach raptor -t constant-regression

Owner: PerfTest Team

Description: Generates a constant value that can be changed to induce a regression.

alert threshold: 2.0
apps: firefox, chrome, safari, geckoview, fenix
browser cycles: 1
browsertime args: –browsertime.constant_value=1500
custom data: true
expected: pass
fore window: 1
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-regression-test.toml#23
lower is better: true
max back window: 3
min back window: 2
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest unit: ms
test script: constant_regression_test.js
test summary: flatten
test url: None
type: pageload
unit: score
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-regression-tests-firefox-constant-regression	❌	✅	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-regression-tests-firefox-constant-regression	❌	✅	❌	❌

getkeyrng

Command to Run Locally

./mach raptor -t getkeyrng

Owner: DOM Lifecycle, Workers and Storage Team

Description: Measures the speed of key-range based reads of IndexedDB

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#78
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
support class: browsertime_pageload.py
test script: indexeddb_getkeyrange.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	✅	✅	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-indexeddb-firefox-getkeyrng	✅	✅	❌	❌

h3-download

Command to Run Locally

./mach raptor -t h3-download

Owner: Network Team

Description: Measures HTTP/3 download throughput with a local server

alert threshold: 2.0
apps: firefox, chrome, custom-car
browser cycles: 1
browsertime args: –browsertime.download_iterations=10 –firefox.preference=network.http.http3.enable:true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-network-bench.toml#28
lower is better: false
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest lower is better: false
subtest name filters: tests/,iterations/
subtest unit: mbps
support class: network_bench.py
test script: download.js
test url: None
type: pageload
unit: mbps
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-chrome-h3-download	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-chrome-h3-download	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-download	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-download	❌	❌	❌	❌

h3-upload

Command to Run Locally

./mach raptor -t h3-upload

Owner: Network Team

Description: Measures HTTP/3 file upload throughput with a local server

alert threshold: 2.0
apps: firefox, chrome, custom-car
browser cycles: 1
browsertime args: –browsertime.upload_iterations=10 –firefox.preference=network.http.http3.enable:true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-network-bench.toml#24
lower is better: false
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest lower is better: false
subtest name filters: tests/,iterations/
subtest unit: mbps
support class: network_bench.py
test script: upload.js
test url: None
type: pageload
unit: mbps
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-chrome-h3-upload	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-network-bench-chrome-h3-upload	❌	❌	❌	❌
browsertime-network-bench-custom-car-h3-upload	❌	❌	❌	❌
browsertime-network-bench-firefox-h3-upload	❌	❌	❌	❌

process-switch

Command to Run Locally

./mach raptor -t process-switch

Owner: PerfTest Team

Description: Measures process switch time

accept zero vismet: true
alert on: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange
alert threshold: 2.0
apps: firefox, chrome
browser cycles: 25
browsertime args: –pageCompleteWaitTime=1000 –pageCompleteCheckInactivity=true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-process-switch.toml#20
lower is better: true
measure: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange
page cycles: 25
page timeout: 60000
playback: mitmproxy
playback pageset manifest: mitm5-linux-firefox-proc-switch.manifest
playback version: 5.1.1
support class: browsertime_pageload.py
test script: process_switch.js
test url: https://mozilla.seanfeng.dev/files/red.html,https://mozilla.pettay.fi/moztests/blue.html
type: pageload
unit: ms
use live sites: false
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	✅	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	✅	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-custom-firefox-process-switch	✅	❌	❌	❌

throttled

Command to Run Locally

./mach raptor -t throttled

Owner: Network Team

Description: Pageload test using a throttled network environment

alert on: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange
alert threshold: 2.0
apps: firefox
browser cycles: 3
cold: true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-throttled-pageload.toml#17
lower is better: true
page cycles: 1
page timeout: 60000
test script: throttled_pageload.js
test url: None
type: pageload
unit: ms
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-throttled-firefox-throttled	❌	❌	❌	❌

tp6-bench

Command to Run Locally

./mach raptor -t tp6-bench

Owner: PerfTest Team

Description: A pageload benchmark test that quickly runs through a selection of the pages available to produce a single overall metric for fcp, lcp, and loadtime. The metrics from all pages get summarized with the geomean, and mean.

alert threshold: 2.0
apps: firefox, chrome, chromium, safari
browser cycles: 1
cold: true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-tp6-bench.toml#19
lower is better: true
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: None
playback version: 8.1.1
subtest unit: ms
support class: browsertime_tp6_bench.py
test script: browsertime_tp6_bench.js
test summary: flatten
test url: None
type: pageload
unit: ms
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-pageload-benchmark-firefox-tp6-bench	❌	✅	❌	❌

upload

Command to Run Locally

./mach raptor -t upload

Owner: Network Team

Description: Measures HTTP/2 file upload throughput with a remote server

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.upload_iterations=10 –browsertime.server_url=https://uploadtest-381620.uc.r.appspot.com –firefox.preference=network.http.http3.enable:false –chrome.args disable-quic
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-upload.toml#16
lower is better: false
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest lower is better: false
subtest name filters: tests/,iterations/
subtest unit: mbps
test script: upload.js
test url: None
type: pageload
unit: mbps
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-chrome-upload	❌	❌	❌	❌
browsertime-upload-firefox-upload	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload	✅	✅	❌	❌
browsertime-upload-safari-upload	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-chrome-upload	❌	❌	❌	❌
browsertime-upload-firefox-upload	✅	✅	❌	❌

upload-h3

Command to Run Locally

./mach raptor -t upload-h3

Owner: Network Team

Description: Measures HTTP/3 file upload throughput with a remote server

alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.upload_iterations=10 –browsertime.server_url=https://uploadtest-381620.uc.r.appspot.com –firefox.preference=network.http.http3.enable:true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-upload.toml#31
lower is better: false
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest lower is better: false
subtest name filters: tests/,iterations/
subtest unit: mbps
test script: upload.js
test url: None
type: pageload
unit: mbps
use live sites: true
Test Task:

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload-h3	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload-h3	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-chrome-upload-h3	❌	❌	❌	❌
browsertime-upload-firefox-upload-h3	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload-h3	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload-h3	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload-h3	✅	✅	❌	❌
browsertime-upload-safari-upload-h3	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload-h3	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-firefox-upload-h3	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-upload-chrome-upload-h3	❌	❌	❌	❌
browsertime-upload-firefox-upload-h3	✅	✅	❌	❌

vpl-av1

Command to Run Locally

./mach raptor -t vpl-av1

Owner: :aosmond and the Media Team

Description: Measures AV1 1080p video playback latency.

alert on: estimatedFirstFrameLatency, estimatedAnyFrameLatency
alert threshold: 2.0
apps: chrome, chrome-m, geckoview, fenix, firefox, safari
benchmark webserver: true
browser cycles: 20
expected: pass
gecko profile features: stackwalk,js,cpu,java,processcpu,screenshots,memory
gecko profile interval: 1
gecko profile threads: BackgroundThreadPool,Compositor,decoder,GeckoMain,gmp,graph,grph,InotifyEventThread,IPDL Background,media,ModuleProcessThread,PacerThread,RemVidChild,RenderBackend,Renderer,SwComposite,TextureUpdate,ImageBridgeChld
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-video-playback-latency.toml#36
lower is better: true
measure: firstFrame, secondFrame, lastFrame, estimatedFirstFrameLatency, estimatedAnyFrameLatency
newtab per cycle: true
page cycles: 20
page timeout: 20000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/video-playback-latency
repository revision: e3d0295c15cfdd6f32e6d4270de0ed5d6c1632cd
support class: video_playback_latency.py
test url: http://<host>:<port>/av1_1080p.html
type: pageload
unit: ms
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-mobile-fenix-vpl-av1-nofis	❌	❌	❌	❌
browsertime-video-playback-latency-mobile-geckoview-vpl-av1-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-mobile-chrome-m-vpl-av1-nofis	❌	❌	❌	❌
browsertime-video-playback-latency-mobile-fenix-vpl-av1-nofis	✅	✅	❌	❌
browsertime-video-playback-latency-mobile-geckoview-vpl-av1-nofis	✅	✅	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-mobile-fenix-vpl-av1-nofis	❌	❌	❌	❌
browsertime-video-playback-latency-mobile-geckoview-vpl-av1-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-mobile-chrome-m-vpl-av1-nofis	❌	❌	❌	❌
browsertime-video-playback-latency-mobile-fenix-vpl-av1-nofis	✅	✅	❌	❌
browsertime-video-playback-latency-mobile-geckoview-vpl-av1-nofis	✅	✅	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-chrome-vpl-av1	❌	❌	❌	❌
browsertime-video-playback-latency-firefox-vpl-av1	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-av1	✅	✅	❌	❌

**test-windows11-64-2009-hw-ref-shippable/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-chrome-vpl-av1	❌	❌	❌	❌
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

**test-windows11-64-2009-hw-ref/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-chrome-vpl-av1	❌	❌	❌	❌
browsertime-video-playback-latency-firefox-vpl-av1	❌	❌	❌	❌

vpl-h264

Command to Run Locally

./mach raptor -t vpl-h264

Owner: :aosmond and the Media Team

Description: Measures H264 1080p video playback latency.

alert on: estimatedFirstFrameLatency, estimatedAnyFrameLatency
alert threshold: 2.0
apps: chrome, chrome-m, geckoview, fenix, firefox, safari
benchmark webserver: true
browser cycles: 20
expected: pass
gecko profile features: stackwalk,js,cpu,java,processcpu,screenshots,memory
gecko profile interval: 1
gecko profile threads: BackgroundThreadPool,Compositor,decoder,GeckoMain,gmp,graph,grph,InotifyEventThread,IPDL Background,media,ModuleProcessThread,PacerThread,RemVidChild,RenderBackend,Renderer,SwComposite,TextureUpdate,ImageBridgeChld
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-video-playback-latency.toml#30
lower is better: true
measure: firstFrame, secondFrame, lastFrame, estimatedFirstFrameLatency, estimatedAnyFrameLatency
newtab per cycle: true
page cycles: 20
page timeout: 20000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/video-playback-latency
repository revision: e3d0295c15cfdd6f32e6d4270de0ed5d6c1632cd
support class: video_playback_latency.py
test url: http://<host>:<port>/h264_1080p.html
type: pageload
unit: ms
Test Task:

**test-android-hw-a51-11-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-mobile-fenix-vpl-h264-nofis	❌	❌	❌	❌
browsertime-video-playback-latency-mobile-geckoview-vpl-h264-nofis	❌	❌	❌	❌

**test-android-hw-a51-11-0-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-mobile-chrome-m-vpl-h264-nofis	❌	❌	❌	❌
browsertime-video-playback-latency-mobile-fenix-vpl-h264-nofis	✅	✅	❌	❌
browsertime-video-playback-latency-mobile-geckoview-vpl-h264-nofis	✅	✅	❌	❌

**test-android-hw-a55-14-0-aarch64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-mobile-fenix-vpl-h264-nofis	❌	❌	❌	❌
browsertime-video-playback-latency-mobile-geckoview-vpl-h264-nofis	❌	❌	❌	❌

**test-android-hw-a55-14-0-android-aarch64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-mobile-chrome-m-vpl-h264-nofis	❌	❌	❌	❌
browsertime-video-playback-latency-mobile-fenix-vpl-h264-nofis	✅	✅	❌	❌
browsertime-video-playback-latency-mobile-geckoview-vpl-h264-nofis	✅	✅	❌	❌

**test-linux1804-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-h264	❌	❌	❌	❌

**test-linux1804-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-h264	❌	❌	❌	❌

**test-linux1804-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-chrome-vpl-h264	❌	❌	❌	❌
browsertime-video-playback-latency-firefox-vpl-h264	✅	✅	❌	❌

**test-macosx1015-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-h264	❌	❌	❌	❌

**test-macosx1015-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-h264	❌	❌	❌	❌

**test-macosx1015-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-h264	✅	✅	❌	❌
browsertime-video-playback-latency-safari-vpl-h264	❌	❌	❌	❌

**test-windows11-64-2009-hw-ref-shippable/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-chrome-vpl-h264	❌	❌	❌	❌
browsertime-video-playback-latency-firefox-vpl-h264	❌	❌	❌	❌

**test-windows11-64-2009-hw-ref/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-h264	❌	❌	❌	❌

**test-windows11-64-nightlyasrelease-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-h264	❌	❌	❌	❌

**test-windows11-64-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-firefox-vpl-h264	❌	❌	❌	❌

**test-windows11-64-shippable-qr/opt**¶
Test Name	mozilla-central	autoland	mozilla-release	mozilla-beta
browsertime-video-playback-latency-chrome-vpl-h264	❌	❌	❌	❌

Raptor¶

Raptor tests¶

Benchmarks¶

Custom¶

Raptor tests ¶

Benchmarks ¶

Custom ¶