Running the Rooting Hazard AnalysisĀ¶

The js/src/devtools/rootAnalysis directory contains scripts for running Brian Hackettā€™s static GC rooting and thread heap write safety analyses on a JS source directory.

To run the analysis on SpiderMonkey:

  1. Unset your $MOZCONFIG

     unset MOZCONFIG
    
  2. Install prerequisites.

     mach hazards bootstrap
    
  3. Build the shell to run the analysis.

     mach hazards build-shell
    
  4. Compile all the code to gather info.

     mach hazards gather --project=js
    
  5. Analyze the gathered info.

     mach hazards analyze --project=js
    

Output goes to $srctop/haz-js/hazards.txt. This will run the analysis on the js/src tree only; if you wish to analyze the full browser, use

    --project=browser

(or leave it off; --project=browser is the default)

  1. (optional) View the resulting hazards.

     mach hazards view --project=js
    

After running the analysis once, you can reuse the *.xdb database files generated, using modified analysis scripts, by running either the mach hazards analyze command above, or by adding on mach hazards analyze <step> to run a subset of the analysis steps; mach hazards analyze -- --list to see step names.

Also, you can pass -- -v to get exact command lines to cut & paste for running the various stages, which is helpful for running under a debugger.

Incremental AnalysesĀ¶

Once you have an analysis, you can make code changes and rebuild with mach hazards gather. This will add to the existing *.xdb files, which will usually work ok, but sometimes older compilations will have left around information that will get in the way. A typical example is with lambda functions: you may get hazards reported due to lambdas that no longer exist, but the newer compile will not replace them. Although this could be fixed with some amount of effort, youā€™re fighting against something of a fundamental problem where the analysis is depending on certain things NOT happening (eg calls to the GC) and incremental compilation only adds and replaces existing information. It does not remove information unless it is replacing it with something of a matching name (and things like lambdas have autogenerated numbers in their names that vary between compiles.)

In short: for development speed, feel free to use incremental analyses but donā€™t trust them. If the hazard analysis starts claiming the impossible is happening, try mach hazards clobber and do a full rebuild.

Overview of what is going on hereĀ¶

So what does this actually do?

  1. It downloads a GCC compiler and plugin (ā€œsixgillā€) from Mozilla servers.

  2. It runs run_complete, a script that builds the target codebase with the downloaded GCC, generating a few database files containing control flow graphs of the full compile, along with type information etc.

  3. Then it runs analyze.py, a Python script, which runs all the scripts which actually perform the analysis ā€“ the tricky parts. (Those scripts are written in JS.)

The easiest way to get this running is to not try to do the instrumented compilation locally. Instead, grab the relevant files from a try server push and analyze them locally.

Local Analysis of Downloaded Intermediate FilesĀ¶

Another useful path is to let the continuous integration system do the hard work of generating the intermediate files and analyze them locally. This is particularly useful if you are working on the analysis itself.

  • Do a try push with ā€œā€“upload-xdbsā€ appended to the try: ā€¦ā€ line.

      mach try fuzzy -q "'haz" --upload-xdbs
    
  • Create an empty directory to run the analysis.

  • When the try job is complete, download the resulting src_body.xdb.bz2, src_comp.xdb.bz2, and file_source.xdb.bz2 files into your directory.

  • Fetch a compiler and sixgill plugin to use:

      mach hazards bootstrap
    

If you are on osx, these will not be available. Instead, build sixgill manually (these directions are a little stale):

    hg clone https://hg.mozilla.org/users/sfink_mozilla.com/sixgill
    cd sixgill
    CC=$HOME/.mozbuild/hazard-tools/gcc/bin/gcc ./release.sh --build # This will fail horribly.
    make bin/xdb.so CXX=clang++
  • Build an optimized JS shell with ctypes. Note that this does not need to match the source you are analyzing in any way; in fact, you pretty much never need to update this once youā€™ve built it. (Though I reserve the right to use any new JS features implemented in Spidermonkey in the futureā€¦)

      mach hazards build-shell
    

The shell will be placed by default in $topsrcdir/obj-haz-shell.

  • Make a defaults.py file containing the following, with your own paths filled in:

      js = "<objdir>/dist/bin/js"
      sixgill_bin = "<sixgill-dir>/bin"
    
  • For the rooting analysis, run

      python <srcdir>/js/src/devtools/rootAnalysis/analyze.py gcTypes
    
  • For the heap write analysis, run

      python <srcdir>/js/src/devtools/rootAnalysis/analyze.py heapwrites
    

Also, you may wish to run with -v (aka ā€“verbose) to see the exact commands executed that you can cut & paste if needed. (I use them to run under the JS debugger when Iā€™m working on the analysis.)