Mozperftest

Mozperftest can be used to run performance tests.

• Running a performance test
  • Running locally
  • Running in the CI
• Running a performance tool
  • Running locally
• Performance scripts
  • XPCShell
    • XPCShell Tests in CI
  • Mochitest
    • Mochitest Android Tests
    • Mochitest Tests in CI
    • Mochitest Android Tests in CI
  • Custom Script
    • Custom Shell Scripts
  • Alert
  • Browsertime
  • Hooks
• Developing in mozperftest
  • Architecture overview
  • Layer
  • Important layers
  • Coding style
  • Landing patches
• Vision
  • Next steps

The following documents all testing we have for mozperftest. If the owner does not specify the Usage and Description, it’s marked N/A.

browser/base/content/test

Performance tests from the ‘browser/base/content/test’ folder.

perftest_browser_xhtml_dom.js

owner:

Browser Front-end team

name:

Dom-size

Measures the size of the DOM

browser/components/translations/tests/browser

Performance tests for Translations models on Firefox Desktop

browser_translations_perf_es_en.js

owner:

Translations Team

name:

Full-Page Translation (Spanish to English)

Default options:

--perfherder
--perfherder-metrics name:engine-init-time,unit:ms,shouldAlert:True,lowerIsBetter:True, name:words-per-second,unit:WPS,shouldAlert:True,lowerIsBetter:False, name:tokens-per-second,unit:TPS,shouldAlert:True,lowerIsBetter:False, name:peak-memory-usage,unit:MiB,shouldAlert:True,lowerIsBetter:True, name:stabilized-memory-usage,unit:MiB,shouldAlert:True,lowerIsBetter:True, name:total-translation-time,unit:s,shouldAlert:True,lowerIsBetter:True
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Tests the speed of Full Page Translations using the Spanish-to-English model.

dom/serviceworkers/test/performance

Performance tests running through Mochitest for Service Workers

test_caching.html

owner:

DOM LWS

name:

Service Worker Caching

Default options:

--perfherder
--perfherder-metrics name:No cache,unit:ms,shouldAlert:True, name:Cached,unit:ms,shouldAlert:True, name:No cache again,unit:ms,shouldAlert:True
--verbose
--manifest perftest.toml
--manifest-flavor plain

Test service worker caching.

test_fetch.html

owner:

DOM LWS

name:

Service Worker Fetch

Default options:

--perfherder
--perfherder-metrics name:Cold fetch,unit:ms,shouldAlert:True, name:Undisturbed fetch,unit:ms,shouldAlert:True, name:Intercepted fetch,unit:ms,shouldAlert:True, name:Liberated fetch,unit:ms,shouldAlert:True, name:Undisturbed XHR,unit:ms,shouldAlert:True, name:Intercepted XHR,unit:ms,shouldAlert:True, name:Liberated XHR,unit:ms,shouldAlert:True
--verbose
--manifest perftest.toml
--manifest-flavor plain

Test cold and warm fetches.

test_registration.html

owner:

DOM LWS

name:

Service Worker Registration

Default options:

--perfherder
--perfherder-metrics name:Registration,unit:ms,shouldAlert:True, name:Registration Internals,unit:ms,shouldAlert:True, name:Activation,unit:ms,shouldAlert:True, name:Unregistration,unit:ms,shouldAlert:True
--verbose
--manifest perftest.toml
--manifest-flavor plain

Test registration, activation, and unregistration.

test_update.html

owner:

DOM LWS

name:

Service Worker Update

Default options:

--perfherder
--perfherder-metrics name:Vacuous update,unit:ms,shouldAlert:True, name:Server update,unit:ms,shouldAlert:True, name:Main callback,unit:ms,shouldAlert:True, name:SW callback,unit:ms,shouldAlert:True, name:Update internals,unit:ms,shouldAlert:True
--verbose
--manifest perftest.toml
--manifest-flavor plain

Test updating.

dom/webgpu/tests/mochitest

Performance tests from the ‘dom/webgpu/tests/mochitest’ folder.

test_queue_write_perf.html

owner:

Graphics Team

name:

Queue Write

Default options:

--perfherder
--perfherder-metrics name:writeBuffer Time,unit:ms, name:writeTexture Time,unit:ms
--manifest perftest.toml
--manifest-flavor plain

Test the performance of Queue.writeBuffer and Queue.writeTexture

intl/benchmarks/test/xpcshell

Performance tests running through XPCShell for Intl code

perftest_dateTimeFormat.js

owner:

Internationalization Team

name:

Intl.DateTimeFormat

tags:

intl,ecma402

Default options:

--perfherder
--perfherder-metrics name:Intl.DateTimeFormat constructor iterations,unit:iterations, name:Intl.DateTimeFormat constructor accumulatedTime,unit:ms, name:Intl.DateTimeFormat constructor perCallTime,unit:ms, name:Intl.DateTimeFormat.prototype.format iterations,unit:iterations, name:Intl.DateTimeFormat.prototype.format accumulatedTime,unit:ms, name:Intl.DateTimeFormat.prototype.format perCallTime,unit:ms
--verbose

Test the speed of the Intl.DateTimeFormat implementation.

perftest_locale.js

owner:

Internationalization Team

name:

Intl.Locale

tags:

intl,ecma402

Default options:

--perfherder
--perfherder-metrics name:Intl.Locale constructor iterations,unit:iterations, name:Intl.Locale constructor accumulatedTime,unit:ms, name:Intl.Locale constructor perCallTime,unit:ms, name:Intl.Locale.prototype accessors iterations,unit:iterations, name:Intl.Locale.prototype accessors accumulatedTime,unit:ms, name:Intl.Locale.prototype accessors perCallTime,unit:ms, name:Intl.Locale.maximize operation iterations,unit:iterations, name:Intl.Locale.maximize operation accumulatedTime,unit:ms, name:Intl.Locale.maximize operation perCallTime,unit:ms
--verbose

Test the speed of the Intl.Locale implementation.

perftest_numberFormat.js

owner:

Internationalization Team

name:

Intl.NumberFormat

tags:

intl,ecma402

Default options:

--perfherder
--perfherder-metrics name:Intl.NumberFormat constructor iterations,unit:iterations, name:Intl.NumberFormat constructor accumulatedTime,unit:ms, name:Intl.NumberFormat constructor perCallTime,unit:ms, name:Intl.NumberFormat.prototype.format iterations,unit:iterations, name:Intl.NumberFormat.prototype.format accumulatedTime,unit:ms, name:Intl.NumberFormat.prototype.format perCallTime,unit:ms, name:Intl.NumberFormat.prototype.formatToParts iterations,unit:iterations, name:Intl.NumberFormat.prototype.formatToParts accumulatedTime,unit:ms, name:Intl.NumberFormat.prototype.formatToParts perCallTime,unit:ms
--verbose

Test the speed of the Intl.NumberFormat implementation.

perftest_pluralRules.js

owner:

Internationalization Team

name:

Intl.PluralRules

tags:

intl,ecma402

Default options:

--perfherder
--perfherder-metrics name:Intl.PluralRules constructor iterations,unit:iterations, name:Intl.PluralRules constructor accumulatedTime,unit:ms, name:Intl.PluralRules constructor perCallTime,unit:ms, name:Intl.PluralRules.prototype.select iterations,unit:iterations, name:Intl.PluralRules.prototype.select accumulatedTime,unit:ms, name:Intl.PluralRules.prototype.select perCallTime,unit:ms, name:Intl.PluralRules pluralCategories iterations,unit:iterations, name:Intl.PluralRules pluralCategories accumulatedTime,unit:ms, name:Intl.PluralRules pluralCategories perCallTime,unit:ms
--verbose

Test the speed of the Intl.PluralRules implementation.

netwerk/test/perf

Performance tests from the ‘network/test/perf’ folder.

perftest_http3_cloudflareblog.js

owner:

Network Team

name:

cloudflare

User-journey live site test for Cloudflare blog.

perftest_http3_controlled.js

owner:

Network Team

name:

controlled

tags:

throttlable

User-journey live site test for controlled server

perftest_http3_facebook_scroll.js

owner:

Network Team

name:

facebook-scroll

Measures the number of requests per second after a scroll.

perftest_http3_google_image.js

owner:

Network Team

name:

g-image

Measures the number of images per second after a scroll.

perftest_http3_google_search.js

owner:

Network Team

name:

g-search

User-journey live site test for google search

perftest_http3_lucasquicfetch.js

owner:

Network Team

name:

lq-fetch

Measures the amount of time it takes to load a set of images.

perftest_http3_youtube_watch.js

owner:

Network Team

name:

youtube-noscroll

Measures quality of the video being played.

perftest_http3_youtube_watch_scroll.js

owner:

Network Team

name:

youtube-scroll

Measures quality of the video being played.

netwerk/test/unit

Performance tests from the ‘netwerk/test/unit’ folder.

test_http3_perf.js

owner:

Network Team

name:

http3 raw

tags:

network,http3,quic

Default options:

--perfherder
--perfherder-metrics name:speed,unit:bps
--xpcshell-cycles 13
--verbose
--try-platform linux, mac

XPCShell tests that verifies the lib integration against a local server

testing/performance

Performance tests from the ‘testing/performance’ folder.

perftest_bbc_link.js

owner:

Performance Team

name:

BBC Link

Measures time to load BBC homepage

perftest_facebook.js

owner:

Performance Team

name:

Facebook

Measures time to log in to Facebook

perftest_jsconf_cold.js

owner:

Performance Team

name:

JSConf (cold)

Measures time to load JSConf page (cold)

perftest_jsconf_warm.js

owner:

Performance Team

name:

JSConf (warm)

Measures time to load JSConf page (warm)

perftest_politico_link.js

owner:

Performance Team

name:

Politico Link

Measures time to load Politico homepage

perftest_youtube_link.js

owner:

Performance Team

name:

YouTube Link

Measures time to load YouTube video

perftest_pageload.js

owner:

Performance Team

name:

pageload

Measures time to load mozilla page

perftest_perfstats.js

owner:

Performance Team

name:

perfstats

Collect perfstats for the given site

This test launches browsertime with the perfStats option (will collect low-overhead timings, see Bug 1553254). The test currently runs a short user journey. A selection of popular sites are visited, first as cold pageloads, and then as warm.

perftest_WPT_chrome_init_file.js

owner:

Performance Testing Team

name:

webpagetest-chrome

Run webpagetest performance pageload tests on Chrome against Alexa top 50 websites

This mozperftest gets webpagetest to run pageload tests on Chrome against the 50 most popular websites and provide data. The full list of data returned from webpagetest: firstContentfulPaint, visualComplete90, firstPaint, visualComplete99, visualComplete, SpeedIndex, bytesIn,bytesOut, TTFB, fullyLoadedCPUms, fullyLoadedCPUpct, domElements, domContentLoadedEventStart, domContentLoadedEventEnd, loadEventStart, loadEventEnd

perftest_WPT_firefox_init_file.js

owner:

Performance Testing Team

name:

webpagetest-firefox

Run webpagetest performance pageload tests on Firefox against Alexa top 50 websites

This mozperftest gets webpagetest to run pageload tests on Firefox against the 50 most popular websites and provide data. The full list of data returned from webpagetest: firstContentfulPaint, timeToContentfulPaint, visualComplete90, firstPaint, visualComplete99, visualComplete, SpeedIndex, bytesIn, bytesOut, TTFB, fullyLoadedCPUms, fullyLoadedCPUpct, domElements, domContentLoadedEventStart, domContentLoadedEventEnd, loadEventStart, loadEventEnd

toolkit/components/ml/tests/browser

Performance tests running through Mochitest for ML Models

browser_ml_semantic_history_search_perf.js

owner:

GenAI Team

name:

ML Semantic History Search

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:True, name:memory,unit:MB,shouldAlert:True
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Test for latency for ML Semantic Search History Feature

browser_ml_smart_tab_clustering_perf.js

owner:

GenAI Team

name:

ML Smart Tab Clustering

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:False, name:memory,unit:MiB,shouldAlert:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Testing Smart Tab Clustering

browser_ml_speecht5_tts.js

owner:

GenAI Team

name:

ML Speech T5 TTS

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:False, name:memory,unit:MiB,shouldAlert:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Testing Speech T5 TTS

browser_ml_autofill_perf.js

owner:

GenAI Team

name:

browser_ml_autofill_perf.js

Default options:

--perfherder
--perfherder-metrics name:AUTOFILL-pipeline-ready-latency,unit:ms,shouldAlert:False, name:AUTOFILL-initialization-latency,unit:ms,shouldAlert:False, name:AUTOFILL-model-run-latency,unit:ms,shouldAlert:False, name:AUTOFILL-total-memory-usage,unit:MiB,shouldAlert:False, name:tokenSpeed,unit:tokens/s,shouldAlert:False,lowerIsBetter:False, name:charactersSpeed,unit:chars/s,shouldAlert:False,lowerIsBetter:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Template test for latency for ML Autofill model

browser_ml_engine_multi_perf.js

owner:

GenAI Team

name:

browser_ml_engine_multi_perf.js

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:False, name:memory,unit:MiB,shouldAlert:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Testing model execution concurrently

browser_ml_engine_perf.js

owner:

GenAI Team

name:

browser_ml_engine_perf.js

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:False, name:memory,unit:MiB,shouldAlert:False, name:tokenSpeed,unit:tokens/s,shouldAlert:False,lowerIsBetter:False, name:charactersSpeed,unit:chars/s,shouldAlert:False,lowerIsBetter:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Template test for latency for ml models

browser_ml_llama_summarizer_perf.js

owner:

GenAI Team

name:

browser_ml_llama_summarizer_perf.js

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:False, name:memory,unit:MB,shouldAlert:False, name:tokenSpeed,unit:tokens/s,shouldAlert:False,lowerIsBetter:False, name:charactersSpeed,unit:chars/s,shouldAlert:False,lowerIsBetter:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Template test for latency for Summarizer model using Llama.cpp WASM

browser_ml_smart_tab_perf.js

owner:

GenAI Team

name:

browser_ml_smart_tab_perf.js

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:False, name:memory,unit:MiB,shouldAlert:False, name:tokenSpeed,unit:tokens/s,shouldAlert:False,lowerIsBetter:False, name:charactersSpeed,unit:chars/s,shouldAlert:False,lowerIsBetter:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Testing Smart Tab Models

browser_ml_suggest_feature_perf.js

owner:

GenAI Team

name:

browser_ml_suggest_feature_perf.js

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:True, name:memory,unit:MiB,shouldAlert:True
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Template test for latency for ML suggest Feature

browser_ml_suggest_inference.js

owner:

GenAI Team

name:

browser_ml_suggest_inference.js

Default options:

--perfherder
--perfherder-metrics name:inference-pipeline-ready-latency,unit:ms,shouldAlert:False, name:inference-initialization-latency,unit:ms,shouldAlert:False, name:inference-model-run-latency,unit:ms,shouldAlert:False, name:inference-total-memory-usage,unit:ms,shouldAlert:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Template test for ML suggest Inference Model

browser_ml_summarizer_perf.js

owner:

GenAI Team

name:

browser_ml_summarizer_perf.js

Default options:

--perfherder
--perfherder-metrics name:latency,unit:ms,shouldAlert:True, name:memory,unit:MiB,shouldAlert:True, name:tokenSpeed,unit:tokens/s,shouldAlert:True,lowerIsBetter:False, name:charactersSpeed,unit:chars/s,shouldAlert:True,lowerIsBetter:False
--verbose
--manifest perftest.toml
--manifest-flavor browser-chrome
--try-platform linux, mac, win

Template test for latency for Summarizer model

toolkit/components/url-classifier/tests/performance

Performance tests for the URL Classifier

perftest_exceptionListLookup.js

owner:

Privacy Team

name:

UrlClassifier.ExceptionListLookup

tags:

url-classifier

Default options:

--perfherder
--perfherder-metrics name:UrlClassifier.ExceptionListLookup iterations,unit:iterations, name:UrlClassifier.ExceptionListLookup accumulatedTime,unit:ms, name:UrlClassifier.ExceptionListLookup perCallTime,unit:ms
--verbose

Test the speed of nsIUrlClassifierExceptionList#matches.

If you have any questions, please see this wiki page.