Trial Inference API Extension Example

This web extension implements an alt text feature on any image in the browser. You can right click on the image and you will get an alt text generated by an image-to-text model.

See the WebExtensions AI API for reference.

Installation

Go into about:debugging > This Nightly > Load Temporary Add-on
Go into the directory containing the extension (toolkit/components/ml/docs/extensions-api-example).
Pick manifest.json and load it.
Once the extension is loaded, go into about:addons and enable the optional permission.

Usage

In a web page with images, right-click on an image then chose “Generate ML description”. The first time you will use it, the model will be downloaded from our servers and you will get a download progress.

Manifest

{
  "manifest_version": 2,
  "name": "test-trial-ml",
  "version": "0.1",
  "description": "test-trial-ml description",
  "optional_permissions": ["trialML"],
  "permissions": ["menus", "activeTab", "scripting"],
  "icons": {
    "64": "icon.png"
  },
  "browser_action": {
    "default_title": "test-trial-ml (browserAction)",
    "default_icon": {
      "64": "icon.png"
    }
  },
  "background": {
    "scripts": ["background.js"]
  },
  "content_scripts": [
    {
      "matches": ["<all_urls>"],
      "js": ["content-script.js"]
    }
  ],
  "options_ui": {
    "page": "settings.html"
  }
}