Raptor¶
Raptor is a performance-testing framework for running browser pageload and browser benchmark tests. Raptor is cross-browser compatible and is currently running in production on Firefox Desktop, Firefox Android GeckoView, Fenix, Reference Browser, Chromium, Chrome, Chrome for Android, Safari, and Safari Technology Preview.
Contact: Dave Hunt [:davehunt]
Source code: https://searchfox.org/mozilla-central/source/testing/raptor
Good first bugs: https://codetribute.mozilla.org/projects/automation?project%3DRaptor
Raptor currently supports three test types: 1) page-load performance tests, 2) standard benchmark-performance tests, and 3) “scenario”-based tests, such as power, CPU, and memory-usage measurements on Android (and desktop?).
Locally, Raptor can be invoked with the following command:
./mach raptor
Raptor tests¶
The following documents all testing we have for Raptor.
Benchmarks¶
Standard benchmarks are third-party tests (i.e. Speedometer) that we have integrated into Raptor to run per-commit in our production CI. To update any of these benchmarks, see Updating Benchmark Tests.
ares6
Command to Run Locally
./mach raptor -t ares6
Owner: :jandem and SpiderMonkey Team
alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/ares6.toml#16
lower is better: true
page cycles: 4
page timeout: 270000
subtest lower is better: true
subtest unit: ms
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-ares6 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-ares6 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-ares6 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-ares6 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-ares6 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-ares6 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-ares6 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-ares6 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-ares6 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-ares6 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-ares6 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-ares6 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-ares6 |
✅ |
✅ |
❌ |
❌ |
assorted-dom
Command to Run Locally
./mach raptor -t assorted-dom
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/assorted-dom.toml#15
lower is better: true
page cycles: 1
page timeout: 60000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/assorted-dom
repository revision: 61332db584026b73e37066d717a162825408c36b
screen capture: true
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-assorted-dom |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-assorted-dom |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-assorted-dom |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-assorted-dom |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-assorted-dom |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-assorted-dom |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-assorted-dom |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-assorted-dom |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-assorted-dom |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-assorted-dom |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-assorted-dom |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-assorted-dom |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-assorted-dom |
✅ |
✅ |
❌ |
❌ |
jetstream2
Command to Run Locally
./mach raptor -t jetstream2
Owner: :jandem and SpiderMonkey Team
alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/jetstream2.toml#16
lower is better: false
page cycles: 4
page timeout: 2000000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/JetStream2
repository revision: 61332db584026b73e37066d717a162825408c36b
subtest lower is better: false
subtest unit: score
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-jetstream2 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-jetstream2 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-jetstream2 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-jetstream2 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-jetstream2 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-jetstream2 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-jetstream2 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-jetstream2 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-jetstream2 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-jetstream2 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-jetstream2 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-jetstream2 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-jetstream2 |
✅ |
✅ |
❌ |
❌ |
matrix-react-bench
Command to Run Locally
./mach raptor -t matrix-react-bench
Owner: :jandem and SpiderMonkey Team
alert threshold: 2.0
apps: firefox, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/matrix-react-bench.toml#16
lower is better: true
page cycles: 30
page timeout: 2000000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/matrix-react-bench
repository revision: 61332db584026b73e37066d717a162825408c36b
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/matrix_demo.html
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-matrix-react-bench |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-matrix-react-bench |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-matrix-react-bench |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-matrix-react-bench |
✅ |
✅ |
❌ |
❌ |
motionmark-1-3
Command to Run Locally
./mach raptor -t motionmark-1-3
Owner: Graphics Team
alert threshold: 2.0
apps: firefox, chrome, safari, custom-car
browser cycles: 1
cold: true
expected: pass
expose browser profiler: true
expose chrome trace: true
gecko profile features: stackwalk,js,cpu,java,processcpu,memory
gecko profile interval: 1
gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-desktop.toml#12
lower is better: false
page cycles: 1
page timeout: 600000
repository: https://github.com/webkit/motionmark
repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb
suite name: MotionMark
support class: motionmark-1-3.py
test script: motionmark-1-3.js
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-1-3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-1-3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-1-3 |
✅ |
✅ |
❌ |
❌ |
Owner: Graphics Team
alert threshold: 2.0
apps: fenix, geckoview, chrome-m, cstm-car-m
browser cycles: 1
cold: true
expected: pass
expose browser profiler: true
gecko profile features: stackwalk,js,cpu,java,processcpu,memory
gecko profile interval: 1
gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-mobile.toml#11
lower is better: false
page cycles: 1
page timeout: 600000
repository: https://github.com/webkit/motionmark
repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb
suite name: MotionMark
support class: motionmark-1-3.py
test script: motionmark-1-3.js
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-fenix-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-1-3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-1-3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-1-3 |
✅ |
✅ |
❌ |
❌ |
motionmark-htmlsuite-1-3
Command to Run Locally
./mach raptor -t motionmark-htmlsuite-1-3
Owner: Graphics Team
alert threshold: 2.0
apps: firefox, chrome, safari, custom-car
browser cycles: 1
cold: true
expected: pass
expose browser profiler: true
expose chrome trace: true
gecko profile features: stackwalk,js,cpu,java,processcpu,memory
gecko profile interval: 1
gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-desktop.toml#27
lower is better: false
page cycles: 1
page timeout: 600000
repository: https://github.com/webkit/motionmark
repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb
suite name: HTML suite
support class: motionmark-1-3.py
test script: motionmark-1-3.js
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
✅ |
✅ |
❌ |
❌ |
Owner: Graphics Team
alert threshold: 2.0
apps: fenix, geckoview, chrome-m, cstm-car-m
browser cycles: 1
cold: true
expected: pass
expose browser profiler: true
gecko profile features: stackwalk,js,cpu,java,processcpu,memory
gecko profile interval: 1
gecko profile threads: GeckoMain,Compositor,Renderer,SwComposite,RenderBackend,SceneBuilder,WrWorker,CanvasWorkers,TextureUpdate
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/motionmark-1-3-mobile.toml#26
lower is better: false
page cycles: 1
page timeout: 600000
repository: https://github.com/webkit/motionmark
repository revision: be2a5fea89b6ef411b053ebeb95a6302b3dc0ecb
suite name: HTML suite
support class: motionmark-1-3.py
test script: motionmark-1-3.js
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-motionmark-1-3-chrome-m-motionmark-htmlsuite-1-3-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-cstm-car-m-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-fenix-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-motionmark-1-3-geckoview-motionmark-htmlsuite-1-3-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-motionmark-htmlsuite-1-3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-motionmark-htmlsuite-1-3 |
✅ |
✅ |
❌ |
❌ |
speedometer
Command to Run Locally
./mach raptor -t speedometer
Owner: SpiderMonkey Team
alert threshold: 2.0
apps: firefox, chrome, safari, safari-tp, custom-car
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-desktop.toml#17
lower is better: false
page cycles: 5
page timeout: 600000
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/Speedometer/index.html?raptor
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-fenix-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-fenix-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer |
✅ |
✅ |
❌ |
❌ |
Owner: SpiderMonkey Team
alert threshold: 2.0
apps: fenix, geckoview, refbrow, chrome-m, cstm-car-m
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-mobile.toml#17
lower is better: false
page cycles: 5
page timeout: 420000
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/Speedometer/index.html?raptor
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-fenix-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-fenix-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-cstm-car-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer |
✅ |
✅ |
❌ |
❌ |
speedometer3
Command to Run Locally
./mach raptor -t speedometer3
Owner: Performance Team
alert threshold: 2.0
apps: firefox, chrome, safari, safari-tp, custom-car
browser cycles: 5
cold: true
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-desktop.toml#19
lower is better: false
page cycles: 5
page timeout: 600000
subtest lower is better: true
subtest unit: ms
support class: speedometer3.py
test script: speedometer3.js
test url: http://<host>:<port>/Speedometer3/index.html?raptor
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer3 |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-safari-tp-speedometer3 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer3 |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer3 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer3 |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Owner: Performance Team
alert threshold: 2.0
apps: fenix, geckoview, refbrow, chrome-m, cstm-car-m
browser cycles: 5
cold: true
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1
host from parent: false
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/speedometer-mobile.toml#19
lower is better: false
page cycles: 5
page timeout: 420000
repository: https://github.com/WebKit/Speedometer
repository revision: 8d67f28d0281ac4330f283495b7f48286654ad7d
subtest lower is better: true
subtest unit: ms
support class: speedometer3.py
test script: speedometer3.js
test url: http://<host>:<port>/index.html
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-speedometer3-mobile-chrome-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-cstm-car-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-fenix-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-speedometer3-mobile-geckoview-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer3 |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-safari-tp-speedometer3 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer3 |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer3 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-speedometer3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-speedometer3 |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-custom-car-speedometer3 |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-speedometer3 |
✅ |
✅ |
❌ |
❌ |
stylebench
Command to Run Locally
./mach raptor -t stylebench
Owner: :emelio and Layout Team
alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/stylebench.toml#16
lower is better: false
page cycles: 5
page timeout: 140000
subtest lower is better: true
subtest unit: ms
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-stylebench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-stylebench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-stylebench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-stylebench |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-stylebench |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-stylebench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-stylebench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-stylebench |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-stylebench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-stylebench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-stylebench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-stylebench |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-stylebench |
✅ |
✅ |
❌ |
❌ |
sunspider
Command to Run Locally
./mach raptor -t sunspider
Owner: :jandem and SpiderMonkey Team
alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/sunspider.toml#10
lower is better: true
page cycles: 5
page timeout: 55000
test url: http://<host>:<port>/SunSpider/sunspider-1.0.1/sunspider-1.0.1/driver.html?raptor
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-sunspider |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-sunspider |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-sunspider |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-sunspider |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-sunspider |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-sunspider |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-sunspider |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-sunspider |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-sunspider |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-sunspider |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-sunspider |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-sunspider |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-sunspider |
✅ |
✅ |
❌ |
❌ |
twitch-animation
Command to Run Locally
./mach raptor -t twitch-animation
Owner: :jrmuizel
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/twitch-animation.toml#17
lower is better: true
page cycles: 1
page timeout: 2000000
perfstats: true
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/twitch-animation
repository revision: 61332db584026b73e37066d717a162825408c36b
subtest lower is better: true
subtest unit: ms
test url: http://<host>:<port>/index.html
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-twitch-animation |
✅ |
✅ |
❌ |
❌ |
unity-webgl
Command to Run Locally
./mach raptor -t unity-webgl
Owner: :jgilbert and Graphics(gfx) Team
alert threshold: 2.0
apps: firefox, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/unity-webgl-desktop.toml#14
lower is better: false
page cycles: 5
page timeout: 420000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/unity-webgl
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-unity-webgl-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-fenix-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-unity-webgl-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-fenix-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-unity-webgl |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-unity-webgl |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-unity-webgl |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-unity-webgl |
✅ |
❌ |
❌ |
❌ |
Owner: :jgilbert and Graphics(gfx) Team
alert threshold: 2.0
apps: geckoview, refbrow, fenix, chrome-m
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/unity-webgl-mobile.toml#14
lower is better: false
page cycles: 1
page timeout: 420000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/unity-webgl
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-unity-webgl-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-fenix-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-unity-webgl-mobile-fenix-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-unity-webgl-mobile-chrome-m-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-fenix-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-unity-webgl-mobile-geckoview-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-unity-webgl |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-unity-webgl |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-unity-webgl |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-unity-webgl |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-unity-webgl |
✅ |
❌ |
❌ |
❌ |
wasm-godot
Command to Run Locally
./mach raptor -t wasm-godot
Owner: :lth and SpiderMonkey Team
alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot.toml#11
lower is better: true
newtab per cycle: true
page cycles: 5
page timeout: 120000
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-chrome-wasm-godot |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-wasm-firefox-wasm-godot |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot |
✅ |
❌ |
❌ |
❌ |
browsertime-benchmark-wasm-safari-wasm-godot |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-chrome-wasm-godot |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-wasm-firefox-wasm-godot |
✅ |
❌ |
❌ |
❌ |
wasm-godot-baseline
Command to Run Locally
./mach raptor -t wasm-godot-baseline
Owner: :lth and SpiderMonkey Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot-baseline.toml#14
lower is better: true
newtab per cycle: true
page cycles: 5
page timeout: 120000
preferences: javascript.options.wasm_baselinejit=true javascript.options.wasm_optimizingjit=false
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-baseline |
✅ |
❌ |
❌ |
❌ |
wasm-godot-optimizing
Command to Run Locally
./mach raptor -t wasm-godot-optimizing
Owner: :lth and SpiderMonkey Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-godot-optimizing.toml#14
lower is better: true
newtab per cycle: true
page cycles: 5
page timeout: 120000
preferences: javascript.options.wasm_baselinejit=false javascript.options.wasm_optimizingjit=true
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-godot-optimizing |
✅ |
❌ |
❌ |
❌ |
wasm-misc
Command to Run Locally
./mach raptor -t wasm-misc
Owner: :lth and SpiderMonkey Team
alert threshold: 2.0
apps: firefox, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc.toml#14
lower is better: true
page cycles: 5
page timeout: 1200000
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/wasm-misc
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-chrome-wasm-misc |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-wasm-firefox-wasm-misc |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-chrome-wasm-misc |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-wasm-firefox-wasm-misc |
✅ |
❌ |
❌ |
❌ |
wasm-misc-baseline
Command to Run Locally
./mach raptor -t wasm-misc-baseline
Owner: :lth and SpiderMonkey Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc-baseline.toml#13
lower is better: true
page cycles: 5
page timeout: 1200000
preferences: javascript.options.wasm_baselinejit=true javascript.options.wasm_optimizingjit=false
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/wasm-misc
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-baseline |
✅ |
❌ |
❌ |
❌ |
wasm-misc-optimizing
Command to Run Locally
./mach raptor -t wasm-misc-optimizing
Owner: :lth and SpiderMonkey Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/wasm-misc-optimizing.toml#13
lower is better: true
page cycles: 5
page timeout: 1200000
preferences: javascript.options.wasm_baselinejit=false javascript.options.wasm_optimizingjit=true
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/wasm-misc
repository revision: 61332db584026b73e37066d717a162825408c36b
test url: http://<host>:<port>/index.html?raptor
type: benchmark
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-wasm-firefox-wasm-misc-optimizing |
✅ |
❌ |
❌ |
❌ |
webaudio
Command to Run Locally
./mach raptor -t webaudio
Owner: :padenot and Media Team
alert threshold: 2.0
apps: firefox, chrome, safari
expected: pass
expose browser profiler: true
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/webaudio.toml#10
lower is better: true
page cycles: 5
page timeout: 360000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
type: benchmark
unit: score
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-webaudio |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-webaudio |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-webaudio |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-webaudio |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-webaudio |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-webaudio |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-webaudio |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-webaudio |
✅ |
✅ |
❌ |
❌ |
browsertime-benchmark-safari-webaudio |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-webaudio |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-firefox-webaudio |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-benchmark-chrome-webaudio |
❌ |
❌ |
❌ |
❌ |
browsertime-benchmark-firefox-webaudio |
✅ |
✅ |
❌ |
❌ |
youtube-playback-av1-sfr
Command to Run Locally
./mach raptor -t youtube-playback-av1-sfr
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#24
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-av1-sfr-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-av1-sfr |
✅ |
❌ |
❌ |
❌ |
youtube-playback-h264-1080p30
Command to Run Locally
./mach raptor -t youtube-playback-h264-1080p30
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#28
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p30 |
❌ |
❌ |
❌ |
❌ |
youtube-playback-h264-1080p60
Command to Run Locally
./mach raptor -t youtube-playback-h264-1080p60
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#34
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-1080p60 |
❌ |
❌ |
❌ |
❌ |
youtube-playback-h264-full-1080p30
Command to Run Locally
./mach raptor -t youtube-playback-h264-full-1080p30
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#40
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
youtube-playback-h264-full-1080p60
Command to Run Locally
./mach raptor -t youtube-playback-h264-full-1080p60
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#55
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-h264-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
youtube-playback-h264-sfr
Command to Run Locally
./mach raptor -t youtube-playback-h264-sfr
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#70
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-geckoview-youtube-playback-h264-sfr-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-h264-sfr |
✅ |
❌ |
❌ |
❌ |
youtube-playback-hfr
Command to Run Locally
./mach raptor -t youtube-playback-hfr
Owner: PerfTest Team
alert on: H2641080p60fps@1X_dropped_frames, H264720p60fps@1X_dropped_frames
alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#74
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-hfr-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-mobile-geckoview-youtube-playback-hfr-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-hfr-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-mobile-geckoview-youtube-playback-hfr-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-hfr-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-mobile-geckoview-youtube-playback-hfr-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-hfr-nofis |
✅ |
❌ |
❌ |
❌ |
browsertime-mobile-geckoview-youtube-playback-hfr-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-hfr |
✅ |
❌ |
❌ |
❌ |
youtube-playback-v9-1080p30
Command to Run Locally
./mach raptor -t youtube-playback-v9-1080p30
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#82
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p30 |
❌ |
❌ |
❌ |
❌ |
youtube-playback-v9-1080p60
Command to Run Locally
./mach raptor -t youtube-playback-v9-1080p60
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#88
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-1080p60 |
❌ |
❌ |
❌ |
❌ |
youtube-playback-v9-full-1080p30
Command to Run Locally
./mach raptor -t youtube-playback-v9-full-1080p30
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#94
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p30 |
❌ |
❌ |
❌ |
❌ |
youtube-playback-v9-full-1080p60
Command to Run Locally
./mach raptor -t youtube-playback-v9-full-1080p60
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox
expected: pass
expose browser profiler: true
gather cpuTime: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#109
lower is better: true
page cycles: 20
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false full-screen-api.allow-trusted-requests-only=false full-screen-api.warning.timeout=0
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-power-firefox-youtube-playback-v9-full-1080p60 |
❌ |
❌ |
❌ |
❌ |
youtube-playback-vp9-sfr
Command to Run Locally
./mach raptor -t youtube-playback-vp9-sfr
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#124
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-mobile-fenix-youtube-playback-vp9-sfr-nofis |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-vp9-sfr |
✅ |
❌ |
❌ |
❌ |
youtube-playback-widevine-h264-sfr
Command to Run Locally
./mach raptor -t youtube-playback-widevine-h264-sfr
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#128
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-h264-sfr |
✅ |
❌ |
❌ |
❌ |
youtube-playback-widevine-hfr
Command to Run Locally
./mach raptor -t youtube-playback-widevine-hfr
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#142
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-hfr |
✅ |
❌ |
❌ |
❌ |
youtube-playback-widevine-vp9-sfr
Command to Run Locally
./mach raptor -t youtube-playback-widevine-vp9-sfr
Owner: PerfTest Team
alert threshold: 2.0
apps: firefox, geckoview, fenix, refbrow, chrome
expected: pass
expose browser profiler: true
gecko profile interval: 1000
gecko profile threads: MediaPlayback
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/benchmarks/youtube-playback.toml#156
lower is better: true
page cycles: 1
page timeout: 2700000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false media.eme.enabled=true media.gmp-manager.updateEnabled=true media.eme.require-app-approval=false
subtest lower is better: true
subtest unit: score
type: benchmark
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-firefox-youtube-playback-widevine-vp9-sfr |
✅ |
❌ |
❌ |
❌ |
Custom¶
Browsertime tests that use a custom pageload test script. These use the pageload type, but may have other intentions.
addMab1
Command to Run Locally
./mach raptor -t addMab1
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send many small ArrayBuffers to IndexedDB in one transaction
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#30
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMab1 |
✅ |
✅ |
❌ |
❌ |
addMabN
Command to Run Locally
./mach raptor -t addMabN
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send many small ArrayBuffers to IndexedDB independently
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#34
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMabN |
✅ |
✅ |
❌ |
❌ |
addMar1
Command to Run Locally
./mach raptor -t addMar1
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send many small Arrays to IndexedDB in one transaction
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Array –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#38
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMar1 |
✅ |
✅ |
❌ |
❌ |
addMarN
Command to Run Locally
./mach raptor -t addMarN
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send many small Arrays to IndexedDB independently
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Array –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#42
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMarN |
✅ |
✅ |
❌ |
❌ |
addMbl1
Command to Run Locally
./mach raptor -t addMbl1
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send many small Blobs to IndexedDB in one transaction
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Blob –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#46
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMbl1 |
✅ |
✅ |
❌ |
❌ |
addMblN
Command to Run Locally
./mach raptor -t addMblN
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send many small Blobs to IndexedDB independently
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=16384 –browsertime.chunk_size=1024 –browsertime.buffer_type=Blob –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#50
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addMblN |
✅ |
✅ |
❌ |
❌ |
addkAB1
Command to Run Locally
./mach raptor -t addkAB1
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send a couple of large ArrayBuffers to IndexedDB in one transaction
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#54
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAB1 |
✅ |
✅ |
❌ |
❌ |
addkABN
Command to Run Locally
./mach raptor -t addkABN
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send a couple of large ArrayBuffers to IndexedDB independently
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=ArrayBuffer –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#58
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkABN |
✅ |
✅ |
❌ |
❌ |
addkAR1
Command to Run Locally
./mach raptor -t addkAR1
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send a couple of large JS Arrays to IndexedDB in one transaction
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Array –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#62
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkAR1 |
✅ |
✅ |
❌ |
❌ |
addkARN
Command to Run Locally
./mach raptor -t addkARN
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send a couple of large JS Arrays to IndexedDB independently
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Array –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#66
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkARN |
✅ |
✅ |
❌ |
❌ |
addkBL1
Command to Run Locally
./mach raptor -t addkBL1
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send a couple of large Blobs to IndexedDB in one transaction
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Blob –browsertime.atomic=1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#70
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBL1 |
✅ |
✅ |
❌ |
❌ |
addkBLN
Command to Run Locally
./mach raptor -t addkBLN
Owner: DOM Lifecycle, Workers and Storage Team
Description: Use add API to send a couple of large Blobs to IndexedDB independently
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.iterations=128 –browsertime.chunk_size=1048576 –browsertime.buffer_type=Blob –browsertime.atomic=0
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#74
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
test script: indexeddb_write.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-addkBLN |
✅ |
✅ |
❌ |
❌ |
browsertime
Command to Run Locally
./mach raptor -t browsertime
Owner: PerfTest Team
Description: Used to run vanilla browsertime tests through raptor. For example: ./mach raptor -t browsertime –browsertime-arg url=https://www.sitespeed.io –browsertime-arg iterations=3
alert threshold: 2.0
apps: firefox
browser cycles: 1
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-custom.toml#17
lower is better: true
measure: fnbpaint, fcp, dcf, loadtime
page cycles: 1
page timeout: 60000
playback: mitmproxy
playback pageset manifest: null.manifest
playback version: 8.1.1
support class: browsertime_pageload.py
test script: None
test url: None
type: pageload
unit: ms
use live sites: true
connect
Command to Run Locally
./mach raptor -t connect
Owner: Network Team
Description: Verifies that connection time on a pageload is zero when a speculative connection has first been made
alert change type: absolute
alert on: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange
alert threshold: 10.0
apps: firefox
browser cycles: 3
cold: true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-speculative.toml#18
lower is better: true
page cycles: 1
page timeout: 60000
test script: speculative-connect.js
test url: None
type: pageload
unit: ms
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-speculative-firefox-connect |
❌ |
✅ |
❌ |
❌ |
constant-regression
Command to Run Locally
./mach raptor -t constant-regression
Owner: PerfTest Team
Description: Generates a constant value that can be changed to induce a regression.
alert threshold: 2.0
apps: firefox, chrome, safari, geckoview, fenix
browser cycles: 1
browsertime args: –browsertime.constant_value=1500
custom data: true
expected: pass
fore window: 1
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-regression-test.toml#23
lower is better: true
max back window: 3
min back window: 2
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest unit: ms
test script: constant_regression_test.js
test summary: flatten
test url: None
type: pageload
unit: score
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-regression-tests-firefox-constant-regression |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-regression-tests-firefox-constant-regression |
❌ |
✅ |
❌ |
❌ |
getkeyrng
Command to Run Locally
./mach raptor -t getkeyrng
Owner: DOM Lifecycle, Workers and Storage Team
Description: Measures the speed of key-range based reads of IndexedDB
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
custom data: true
expected: pass
gecko profile entries: 131072000
gecko profile features: js,stackwalk,cpu,memory
gecko profile threads: GeckoMain,DOM Worker,IndexedDB
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-indexeddb.toml#78
lower is better: true
measure: cpuTime
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: mitm8-linux-firefox-example.manifest
subtest unit: ms
support class: browsertime_pageload.py
test script: indexeddb_getkeyrange.js
test summary: flatten
test url: https://www.example.com
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-indexeddb-firefox-getkeyrng |
✅ |
✅ |
❌ |
❌ |
h3-download
Command to Run Locally
./mach raptor -t h3-download
Owner: Network Team
Description: Measures HTTP/3 download throughput with a local server
alert threshold: 2.0
apps: firefox, chrome, custom-car
browser cycles: 1
browsertime args: –browsertime.download_iterations=10 –firefox.preference=network.http.http3.enable:true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-network-bench.toml#28
lower is better: false
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest lower is better: false
subtest name filters: tests/,iterations/
subtest unit: mbps
support class: network_bench.py
test script: download.js
test url: None
type: pageload
unit: mbps
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-chrome-h3-download |
❌ |
❌ |
❌ |
❌ |
browsertime-network-bench-custom-car-h3-download |
❌ |
❌ |
❌ |
❌ |
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-chrome-h3-download |
❌ |
❌ |
❌ |
❌ |
browsertime-network-bench-custom-car-h3-download |
❌ |
❌ |
❌ |
❌ |
browsertime-network-bench-firefox-h3-download |
❌ |
❌ |
❌ |
❌ |
h3-upload
Command to Run Locally
./mach raptor -t h3-upload
Owner: Network Team
Description: Measures HTTP/3 file upload throughput with a local server
alert threshold: 2.0
apps: firefox, chrome, custom-car
browser cycles: 1
browsertime args: –browsertime.upload_iterations=10 –firefox.preference=network.http.http3.enable:true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-network-bench.toml#24
lower is better: false
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest lower is better: false
subtest name filters: tests/,iterations/
subtest unit: mbps
support class: network_bench.py
test script: upload.js
test url: None
type: pageload
unit: mbps
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-chrome-h3-upload |
❌ |
❌ |
❌ |
❌ |
browsertime-network-bench-custom-car-h3-upload |
❌ |
❌ |
❌ |
❌ |
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-network-bench-chrome-h3-upload |
❌ |
❌ |
❌ |
❌ |
browsertime-network-bench-custom-car-h3-upload |
❌ |
❌ |
❌ |
❌ |
browsertime-network-bench-firefox-h3-upload |
❌ |
❌ |
❌ |
❌ |
process-switch
Command to Run Locally
./mach raptor -t process-switch
Owner: PerfTest Team
Description: Measures process switch time
accept zero vismet: true
alert on: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange
alert threshold: 2.0
apps: firefox, chrome
browser cycles: 25
browsertime args: –pageCompleteWaitTime=1000 –pageCompleteCheckInactivity=true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-process-switch.toml#20
lower is better: true
measure: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange, LastVisualChange
page cycles: 25
page timeout: 60000
playback: mitmproxy
playback pageset manifest: mitm5-linux-firefox-proc-switch.manifest
playback version: 5.1.1
support class: browsertime_pageload.py
test script: process_switch.js
test url: https://mozilla.seanfeng.dev/files/red.html,https://mozilla.pettay.fi/moztests/blue.html
type: pageload
unit: ms
use live sites: false
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
✅ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-custom-firefox-process-switch |
✅ |
❌ |
❌ |
❌ |
sample-python-support
Command to Run Locally
./mach raptor -t sample-python-support
Owner: PerfTest Team
Description: A sample test that uses a python support file to modify the test command.
alert threshold: 2.0
apps: firefox
browser cycles: 1
browsertime args: –browsertime.constant_value={replace-with-constant-value}
custom data: true
expected: pass
fore window: 1
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-sample-python-support.toml#22
lower is better: true
max back window: 3
min back window: 2
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest unit: ms
support class: sample_python_support.py
test script: constant_regression_test.js
test summary: flatten
test url: None
type: pageload
unit: score
use live sites: true
throttled
Command to Run Locally
./mach raptor -t throttled
Owner: Network Team
Description: Pageload test using a throttled network environment
alert on: fcp, loadtime, ContentfulSpeedIndex, PerceptualSpeedIndex, SpeedIndex, FirstVisualChange
alert threshold: 2.0
apps: firefox
browser cycles: 3
cold: true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-throttled-pageload.toml#17
lower is better: true
page cycles: 1
page timeout: 60000
test script: throttled_pageload.js
test url: None
type: pageload
unit: ms
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-throttled-firefox-throttled |
❌ |
❌ |
❌ |
❌ |
tp6-bench
Command to Run Locally
./mach raptor -t tp6-bench
Owner: PerfTest Team
Description: A pageload benchmark test that quickly runs through a selection of the pages available to produce a single overall metric for fcp, lcp, and loadtime. The metrics from all pages get summarized with the geomean, and mean.
alert threshold: 2.0
apps: firefox, chrome, chromium, safari
browser cycles: 1
cold: true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-tp6-bench.toml#19
lower is better: true
output timeout: 2000000
page cycles: 1
page timeout: 1800000
playback: mitmproxy
playback pageset manifest: None
playback version: 8.1.1
subtest unit: ms
support class: browsertime_tp6_bench.py
test script: browsertime_tp6_bench.js
test summary: flatten
test url: None
type: pageload
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-pageload-benchmark-firefox-tp6-bench |
❌ |
✅ |
❌ |
❌ |
upload
Command to Run Locally
./mach raptor -t upload
Owner: Network Team
Description: Measures HTTP/2 file upload throughput with a remote server
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.upload_iterations=10 –browsertime.server_url=https://uploadtest-381620.uc.r.appspot.com –firefox.preference=network.http.http3.enable:false –chrome.args disable-quic
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-upload.toml#16
lower is better: false
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest lower is better: false
subtest name filters: tests/,iterations/
subtest unit: mbps
test script: upload.js
test url: None
type: pageload
unit: mbps
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-chrome-upload |
❌ |
❌ |
❌ |
❌ |
browsertime-upload-firefox-upload |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload |
✅ |
✅ |
❌ |
❌ |
browsertime-upload-safari-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-chrome-upload |
❌ |
❌ |
❌ |
❌ |
browsertime-upload-firefox-upload |
✅ |
✅ |
❌ |
❌ |
upload-h3
Command to Run Locally
./mach raptor -t upload-h3
Owner: Network Team
Description: Measures HTTP/3 file upload throughput with a remote server
alert threshold: 2.0
apps: firefox, chrome, safari
browser cycles: 1
browsertime args: –browsertime.upload_iterations=10 –browsertime.server_url=https://uploadtest-381620.uc.r.appspot.com –firefox.preference=network.http.http3.enable:true
custom data: true
expected: pass
gecko profile interval: 1
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-upload.toml#31
lower is better: false
output timeout: 2000000
page cycles: 1
page timeout: 1800000
subtest lower is better: false
subtest name filters: tests/,iterations/
subtest unit: mbps
test script: upload.js
test url: None
type: pageload
unit: mbps
use live sites: true
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload-h3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload-h3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-chrome-upload-h3 |
❌ |
❌ |
❌ |
❌ |
browsertime-upload-firefox-upload-h3 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload-h3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload-h3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload-h3 |
✅ |
✅ |
❌ |
❌ |
browsertime-upload-safari-upload-h3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload-h3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-firefox-upload-h3 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-upload-chrome-upload-h3 |
❌ |
❌ |
❌ |
❌ |
browsertime-upload-firefox-upload-h3 |
✅ |
✅ |
❌ |
❌ |
vpl-av1
Command to Run Locally
./mach raptor -t vpl-av1
Owner: :aosmond and the Media Team
Description: Measures AV1 1080p video playback latency.
alert on: estimatedFirstFrameLatency, estimatedAnyFrameLatency
alert threshold: 2.0
apps: chrome, chrome-m, geckoview, fenix, firefox, safari
benchmark webserver: true
browser cycles: 20
expected: pass
gecko profile features: stackwalk,js,cpu,java,processcpu,screenshots,memory
gecko profile interval: 1
gecko profile threads: BackgroundThreadPool,Compositor,decoder,GeckoMain,gmp,graph,grph,InotifyEventThread,IPDL Background,media,ModuleProcessThread,PacerThread,RemVidChild,RenderBackend,Renderer,SwComposite,TextureUpdate,ImageBridgeChld
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-video-playback-latency.toml#36
lower is better: true
measure: firstFrame, secondFrame, lastFrame, estimatedFirstFrameLatency, estimatedAnyFrameLatency
newtab per cycle: true
page cycles: 20
page timeout: 20000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/video-playback-latency
repository revision: e3d0295c15cfdd6f32e6d4270de0ed5d6c1632cd
support class: video_playback_latency.py
test url: http://<host>:<port>/av1_1080p.html
type: pageload
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-mobile-fenix-vpl-av1-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-geckoview-vpl-av1-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-mobile-chrome-m-vpl-av1-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-fenix-vpl-av1-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-geckoview-vpl-av1-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-mobile-fenix-vpl-av1-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-geckoview-vpl-av1-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-mobile-chrome-m-vpl-av1-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-fenix-vpl-av1-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-geckoview-vpl-av1-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-chrome-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-firefox-vpl-av1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-av1 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-chrome-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-chrome-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-firefox-vpl-av1 |
❌ |
❌ |
❌ |
❌ |
vpl-h264
Command to Run Locally
./mach raptor -t vpl-h264
Owner: :aosmond and the Media Team
Description: Measures H264 1080p video playback latency.
alert on: estimatedFirstFrameLatency, estimatedAnyFrameLatency
alert threshold: 2.0
apps: chrome, chrome-m, geckoview, fenix, firefox, safari
benchmark webserver: true
browser cycles: 20
expected: pass
gecko profile features: stackwalk,js,cpu,java,processcpu,screenshots,memory
gecko profile interval: 1
gecko profile threads: BackgroundThreadPool,Compositor,decoder,GeckoMain,gmp,graph,grph,InotifyEventThread,IPDL Background,media,ModuleProcessThread,PacerThread,RemVidChild,RenderBackend,Renderer,SwComposite,TextureUpdate,ImageBridgeChld
link searchfox: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/custom/browsertime-video-playback-latency.toml#30
lower is better: true
measure: firstFrame, secondFrame, lastFrame, estimatedFirstFrameLatency, estimatedAnyFrameLatency
newtab per cycle: true
page cycles: 20
page timeout: 20000
preferences: media.autoplay.default=0 media.autoplay.ask-permission=false media.autoplay.blocking_policy=0 media.allowed-to-play.enabled=true media.block-autoplay-until-in-foreground=false
repository: https://github.com/mozilla/perf-automation
repository path: benchmarks/video-playback-latency
repository revision: e3d0295c15cfdd6f32e6d4270de0ed5d6c1632cd
support class: video_playback_latency.py
test url: http://<host>:<port>/h264_1080p.html
type: pageload
unit: ms
Test Task:
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-mobile-fenix-vpl-h264-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-geckoview-vpl-h264-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-mobile-chrome-m-vpl-h264-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-fenix-vpl-h264-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-geckoview-vpl-h264-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-mobile-fenix-vpl-h264-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-geckoview-vpl-h264-nofis |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-mobile-chrome-m-vpl-h264-nofis |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-fenix-vpl-h264-nofis |
✅ |
✅ |
❌ |
❌ |
browsertime-video-playback-latency-mobile-geckoview-vpl-h264-nofis |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-chrome-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-firefox-vpl-h264 |
✅ |
✅ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-h264 |
✅ |
✅ |
❌ |
❌ |
browsertime-video-playback-latency-safari-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-chrome-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
browsertime-video-playback-latency-firefox-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-firefox-vpl-h264 |
❌ |
❌ |
❌ |
❌ |
Test Name |
mozilla-central |
autoland |
mozilla-release |
mozilla-beta |
---|---|---|---|---|
browsertime-video-playback-latency-chrome-vpl-h264 |
❌ |
❌ |
❌ |
❌ |