Networking Test Guidelines
This is a high level document to introduce the different test types that are used in necko. The target audience is newcomer of necko team.
Necko Test Types
We only introduce tests under netwerk/test folder in this section.
-
We usually write chrome tests when the code to be tested needs a browser windows to load some particular resources.
Path: netwerk/test/browser
-
Rarely used in necko.
-
Used when the code to be tested can be triggered by WebIDL. e.g., WebSocket and XMLHttpRequest.
Path: netwerk/test/mochitests
-
Mostly used in necko to test objects that can be accessed by JS. e.g.,
nsIHttpChannel
.Path: netwerk/test/unit
-
Useful when the code doesn’t need a http server.
Useful when writing code regarding to parsing strings. e.g., Parsing Server Timing Header
-
Current tests in netwerk/test/perf are all for testing
HTTP/3
code.
There are also web-platform-tests that is related to necko. We don’t usually write new web-platform-tests
. However, we do have lots of web-platform-tests
for XHR, Fetch, and WebSocket.
Running Necko xpcshell-tests
Local:
Run all xpcshell-tests:
./mach xpcshell-test netwerk/test/unit
Note that xpcshell-tests are run in parallel, sometimes we want to run them sequentially for debugging.
./mach xpcshell-test --sequential netwerk/test/unit
Run a single tests:
./mach xpcshell-test netwerk/test/unit/test_http3.js
Run with socket process enabled:
./mach xpcshell-test --setpref="network.http.network_access_on_socket_process.enabled=true" netwerk/test/unit/test_http3.js
We usually debug networking issues with
HTTP Logging
. To enable logging when running tests:MOZ_LOG=nsHttp:5 ./mach xpcshell-test netwerk/test/unit/test_http3.js
Remote:
First of all, we need to know Fuzzy Selector, which is the tool we use to select which test to run on try. If you already know that your code change can be covered by necko xpcshell-tests, you can use the following command to run all tests in
netwerk/test/unit
on try../mach try fuzzy netwerk/test/unit
Run a single test on try:
./mach try fuzzy netwerk/test/unit/test_http3.js
Sometimes we want to debug the failed test on try with logging enabled:
./mach try fuzzy --env "MOZ_LOG=nsHttp:5,nsHostResolver:5" netwerk/tesst/unit/test_http3.js
Note that it’s not usually a good idea to enabling logging when running all tests in a folder on try, since the raw log file can be really huge. The log file might be not available if the size exceeds the limit on try. In the case that your code change is too generic or you are not sure which tests to run, you can use Auto Selector to let it select tests for you.
Debugging Intermittent Test Failures
There are a lot of intermittent failures on try (usually not able to reproduce locally). Debugging these failures can be really annoying and time consuming. Here are some general guidelines to help you debug intermittent failures more efficiently.
Identify whether the failure is caused by your code change.
Try to reproduce the intermittent failure locally. This is the most straightforward way. Adding
--verify
flag is also helpful when debugging locally (see this document for more details).We can check the failure summery on try to see if there is already a bug filed for this test failure. If yes, it’s likely this is not caused by your code change.
Re-trigger the failed test a few times and see if it passed. This can be easily done by clicking the
Push Health
button.Looking for similar failures happening now on other submissions. This can be done by:
click on failing job -> read failure summary -> find similar ones by other authors in similar jobs
To re-run the failed test suite more times, you could add
rebuild
option to./mach try
. For example, the following command allows to run necko xpcshell-tests 20 times on try.
./mach try fuzzy netwerk/test/unit --rebuild 20
In the case that we really need to debug an intermittent test failure, see this document first for some general tips. Unfortunately, there is no easy way to debug this. One can try to isolate the failed test first and enable
HTTP logging
on try to collect the log for further analysis.
Writing Necko XPCShell Tests
The most typical form of necko xpcsehll-test is creating an HTTP server and test your code by letting the server return some specific responses (e.g., 103 Early Hint
). We will only introduce how to write this kind of test in this document.
Code at server side
After bug 1756557, it is possible to create a
nodejs
HTTP server in your test code. This saves us some time for writing code at the server side by reusing the HTTP module provided bynodejs
. This is what it looks like to create a simple HTTP server:let server = new NodeHTTPServer(); await server.start(); registerCleanupFunction(async () => { await server.stop(); }); await server.registerPathHandler("/test", (req, resp) => { resp.writeHead(200); resp.end("done"); });
We can also create a
HTTP/2
server easily by replacingNodeHTTPServer
withNodeHTTP2Server
and adding the server certification.let certdb = Cc["@mozilla.org/security/x509certdb;1"].getService( Ci.nsIX509CertDB ); addCertFromFile(certdb, "http2-ca.pem", "CTu,u,u"); let server = new NodeHTTP2Server();
Code at client side
The recommend way is to create and open an HTTP channel and handle the response with a
Promise
asynchronously. The code would be like:function makeChan(uri) { let chan = NetUtil.newChannel({ uri, loadUsingSystemPrincipal: true, }).QueryInterface(Ci.nsIHttpChannel); chan.loadFlags = Ci.nsIChannel.LOAD_INITIAL_DOCUMENT_URI; return chan; } let chan = makeChan(`http://localhost:${server.port()}/test`); let req = await new Promise(resolve => { chan.asyncOpen(new ChannelListener(resolve, null, CL_ALLOW_UNKNOWN_CL)); });
This is what it looks like to put everything together:
add_task(async function test_http() { let server = new NodeHTTPServer(); await server.start(); registerCleanupFunction(async () => { await server.stop(); }); await server.registerPathHandler("/test", (req, resp) => { resp.writeHead(200); resp.end("done"); }); let chan = makeChan(`http://localhost:${server.port()}/test`); let req = await new Promise(resolve => { chan.asyncOpen(new ChannelListener(resolve, null, CL_ALLOW_UNKNOWN_CL)); }); equal(req.status, Cr.NS_OK); equal(req.QueryInterface(Ci.nsIHttpChannel).responseStatus, 200); equal(req.QueryInterface(Ci.nsIHttpChannel).protocolVersion, "http/1.1"); });