Back to all articles

How to build an extension using Opera’s Web AI APIs

Hey all,

Henrik here, I’m the Senior AI Product Director at Opera. In this post I’ll walk through how to build a browser extension with Opera’s Web AI APIs, which use local AI models that run on your device and are built into the Opera browser. We’ll build an extension that helps you search through your browser history by meaning.

Your browser history only matches words in a page’s title or URL. If you remember what a page was about but not what it was called, it can’t help you. Searching with the key words or phrases you remember about the page is what I mean by “meaning.”

A cool approach to this is to embed every page you read and search by vector similarity. An embedding model turns a piece of text into a vector, a list of numbers that represents its meaning. Texts about the same subject get vectors that point in a similar direction, even when they share no words.

Searching works the same way. The query is turned into a vector too, and each page is scored by the cosine similarity between its vector and the query’s. Cosine similarity measures the angle between two vectors: 1 means they point the same way, and lower means less related. The pages with the highest scores are the results.

Two things would normally make this extension unattractive to build: a cloud embedding API means sending your whole browsing history to a server, and paying for tokens on every page you index.

Opera’s Web AI APIs solve both problems. Opera runs the models locally, so your browsing history doesn’t leave your machine and each call is free. And because the APIs are built into the browser, there are no API keys to manage and no service in between.

This extension uses two of them:

  • SemanticEmbedder turns page text and search queries into vectors.
  • DecisionModel reads a piece of text and answers questions about it. I use it to score how well each top result matches the query.

Prerequisite: Enable local models and Web AI in Opera

In Opera (latest version), go to Settings → Opera AI and turn on Enable Web AI API. Then open the Web AI panel and download EmbeddingGemma 2 and Gemma 4 E2B. This downloads the local models, which can take a few minutes.

How to build the extension following this blog post

Quick recap: what is a browser extension

A browser extension is a bundle of JS/HTML/CSS described by a manifest.json file (Manifest V3). Extensions are normally packaged for an extension store, but here we’ll load ours unpacked: a plain folder you point Opera at from opera://extensions with developer mode on. You debug it with Opera’s developer tools, inspecting the background service worker from the extensions page and content scripts from the page’s own console.

Prepare the structure

Create a folder for the files discussed below. It should contain these files (with line count listed):

  • manifest.json: 9 lines
  • sw.js: 11 lines, starts the offscreen document and forwards messages
  • content.js: 14 lines, decides if a page is worth indexing
  • offscreen.html: 1 line
  • offscreen.js: 68 lines, models, storage, search and reranking
  • popup.html: 12 lines
  • popup.js: 36 lines, search box and results

For each of the files, copy the code block from this post and paste it into a new file with the matching filename.

To load and test the extension, open opera://extensions, turn on Developer mode, click Load unpacked and pick the folder.

The Opera extensions page with Developer mode turned on and the Load unpacked button highlighted

When you have made updates and need to reload, find the extension in the list and click the reload button.

The Semantic History extension card with the reload button highlighted

A pro-tip is to pin the extension so that it’s shown persistently in Opera’s UI.

The Semantic History extension in the extensions menu with the pin button highlighted

To open developer tools for the popup, right-click in the extension popup and click “Inspect Element”.

The extension popup with the Inspect Element context menu item highlighted

And now we’re ready to actually start building.

Step 1: Manifest and service worker

Manifest V3 kills the service worker after about 30 seconds of inactivity. A SemanticEmbedder or DecisionModel session takes a few seconds to start, so the sessions can’t live there. They live in an offscreen document, which stays alive. The service worker just creates that document and forwards messages to it.

JSON
// manifest.json
{
  "manifest_version": 3,
  "name": "Semantic History",
  "version": "1.0",
  "permissions": ["offscreen", "unlimitedStorage"],
  "background": { "service_worker": "sw.js" },
  "content_scripts": [{ "matches": ["http://*/*", "https://*/*"], "js": ["content.js"] }],
  "action": { "default_popup": "popup.html" }
}
JS
// sw.js
// Service workers die after ~30s idle, so the models live in an offscreen
// document. This file only makes sure it exists and forwards messages to it.
let ready;   // shared, so two messages at once don't both try to create the document
chrome.runtime.onMessage.addListener((msg, _sender, respond) => {
  if (msg.fwd) return;
  ready ??= chrome.offscreen.hasDocument().then((has) => has || chrome.offscreen.createDocument({
    url: 'offscreen.html', reasons: ['WORKERS'], justification: 'Keeps model sessions alive' }));
  ready.then(() => chrome.runtime.sendMessage({ ...msg, fwd: true }))
    .then(respond, (e) => respond({ error: e.message }));
  return true;
});
HTML
<!-- offscreen.html -->
<meta charset="utf-8"><script src="offscreen.js"></script>

The fwd flag: a message from the popup reaches every extension context, including the offscreen document. Without the flag, the offscreen document and the service worker would both answer, and the popup gets whichever reply comes first.

Step 2: Which pages to index

Most pages you open aren’t worth indexing: front pages, search results, logins, tabs you closed after a second. The content script filters them out before anything reaches the model. A page is indexed only if:

  • it’s not on the denylist (mail, banking),
  • it has been the visible tab for 10 seconds,
  • it has at least three paragraphs of 25+ words.
JS
// content.js
// Index a page once it has been in front for 10 seconds and has real prose.
const DENY = /mail|bank|paypal|localhost/;
let seen = 0;
const timer = setInterval(() => {
  if (DENY.test(location.hostname)) return clearInterval(timer);
  if (document.visibilityState !== 'visible' || (seen += 2) < 10) return;
  // Paragraphs of 25+ words tell an article apart from a front page or a product grid.
  const root = document.querySelector('article, main') ?? document.body;
  const paras = [...root.querySelectorAll('p')].map((p) => p.innerText.trim())
    .filter((t) => t.split(/\s+/).length >= 25);
  if (paras.length < 3) return;
  clearInterval(timer);
  chrome.runtime.sendMessage({ type: 'capture', url: location.href.split('#')[0], title: document.title, paras });
}, 2000);

Why paragraphs and not word count: a search results page can have thousands of words, but they’re short snippets and labels. Only articles have several long paragraphs, so this one rule works on most sites.

Step 3: Indexing and search

offscreen.js holds everything else. Storage is an IndexedDB store with one record per page, keyed by URL, with the page’s chunks and their vectors inside.

JS
// offscreen.js — storage and embedding
// The model sessions and the index live here, because this document outlives
// the service worker.
const wait = (r) => new Promise((res, rej) => { r.onsuccess = () => res(r.result); r.onerror = () => rej(r.error); });
const open = indexedDB.open('history', 1);
open.onupgradeneeded = () => open.result.createObjectStore('pages', { keyPath: 'url' });
const db = wait(open);
const pages = async (mode = 'readonly') => (await db).transaction('pages', mode).objectStore('pages');

let embedder;
async function embed(text, taskType) {
  embedder ??= await SemanticEmbedder.create();   // Web AI API
  // A failed call leaves the session dead, so drop it and start fresh next time.
  return embedder.embed(text, { taskType })   // Web AI API
    .then((r) => Float32Array.from(r.embeddings[0].values), (e) => { embedder = null; throw e; });
}

Opera Web AI API used here: SemanticEmbedder (create(), embed())

SemanticEmbedder’s embed() has an input limit at around 2,000 characters. Beyond it, it throws “The model execution session has been destroyed”, and every later call on that session fails too. So the embed() wrapper drops the session on any error, and capture() embeds chunks of at most 1,000 characters, one per call.

JS
// offscreen.js — indexing
let lastError = '';
async function capture({ url, title, paras }) {
  if (await wait((await pages()).count(url))) return;   // already indexed
  // embed() destroys its session on long input, so cap chunks at 1,000
  // characters, embed one per call, and skip any the model still rejects.
  const chunks = [];
  for (const text of paras.flatMap((p) => p.match(/[^]{1,1000}/g))) {
    const vector = await embed(text, 'retrieval-document').catch((e) => { lastError = e.message; return null; });
    if (vector) chunks.push({ text, vector });
  }
  if (chunks.length) await wait((await pages('readwrite')).put({ url, title, chunks }));
}

embed() returns vectors that are already normalized to length 1, so cosine similarity is a plain dot product. Set taskType: the same text embedded as retrieval-query and as retrieval-document gives vectors with a similarity of only 0.88, so the hint changes the output.

JS
// offscreen.js — search
async function search({ query }) {
  const q = await embed(query, 'retrieval-query');
  const score = (v) => v.reduce((sum, x, i) => sum + x * q[i], 0);   // vectors are pre-normalized, so this is the cosine
  const hits = (await wait((await pages()).getAll())).map(({ url, title, chunks }) => {
    const scored = chunks.map((c) => ({ text: c.text, score: score(c.vector) }));
    return { url, title, ...scored.sort((a, b) => b.score - a.score)[0] };   // a page scores as its best chunk
  });
  return hits.sort((a, b) => b.score - a.score).slice(0, 10);
}

A page’s score is the score of its best-matching chunk. Averaging all chunks would bury a long page that matches in only one paragraph.

Comparing the query with every stored vector is fast enough. For example, a search over 32 chunks took 31 ms, and 24 ms of that was embedding the query on my hardware. At 50,000 chunks the dot products take about 64 ms.

Multilingual search comes free: a Norwegian query finds English pages, because the model puts both languages in the same vector space.

Step 4: Similarity isn’t relevance

Cosine search always returns results, even when nothing matches. Here are the top results for a query that none of them matched:

  • 0.688: Ukraine needs money to survive
  • 0.667: NRK front page
  • 0.605: PHP cyber-security flaw
  • 0.562: The animals you can’t outrun

A fixed cutoff doesn’t work either, because 0.6 means something different for a two-word query than for a sentence, or in another language.

So the top five results go through DecisionModel, Opera’s other local model. You give it a schema of questions, and it answers them about a piece of text, with probabilities.

JS
// offscreen.js — reranking and message routing
// Cosine finds similar text, not relevant text, so DecisionModel reads the top
// results and scores each 1-5. Use expectedScore; confidence sits at ~1.0.
async function judge({ query, texts }) {
  const model = await DecisionModel.create({   // Web AI API
    context: 'Rating how closely a web page matches what someone is looking for.',
    expectedInputs: [{ type: 'text', languages: ['en'] }],   // only 'en' is accepted; other languages still work
    questions: [{
      id: 'match', type: 'score',
      prompt: `Someone is looking for: "${query}". How closely does this text match?`,
      options: [
        { label: '1', description: 'Completely different subject' },
        { label: '3', description: 'Related subject' },
        { label: '5', description: 'Exactly what they are looking for' },
      ],
    }],
  });
  const scores = [];   // parallel decide() calls may fail
  for (const text of texts) scores.push(await model.decide(text).then((r) => r.match.expectedScore, () => null));   // Web AI API
  model.destroy?.();   // Web AI API
  return scores;
}

const status = async () => `${await wait((await pages()).count())} pages indexed ${lastError}`;
const routes = { capture, search, judge, status };
chrome.runtime.onMessage.addListener((msg, _sender, respond) => {
  if (!msg.fwd) return;   // ignore the popup's direct message; answer the copy sw.js forwards
  routes[msg.type](msg).then(respond, (e) => respond({ error: e.message }));
  return true;
});

Opera Web AI API used here: DecisionModel (create(), decide(), destroy())

Step 5: The extension popup

This is what the user will see when clicking the extension icon. The popup runs two steps:

  1. search returns the results ranked by cosine similarity, and the popup shows them right away.
  2. judge scores the top five with DecisionModel, which takes a second or two. The popup then moves the best matches to the top and fades any result that scored below 2.5 out of 5.
HTML
<!-- popup.html -->
<meta charset="utf-8">
<style>
  body { width: 400px; font: 13px system-ui; margin: 10px; }
  input { width: 100%; padding: 6px; box-sizing: border-box; }
  a { display: flex; justify-content: space-between; padding: 6px 0; color: inherit; }
  a::after { content: attr(data-score); color: gray; }
  .weak { opacity: 0.4; }
</style>
<form><input id="q" placeholder="What do you half-remember? Press Enter" autofocus></form>
<p id="meta"></p>
<div id="out"></div>
<script src="popup.js"></script>
JS
// popup.js
const $ = (s) => document.querySelector(s);
const say = (text) => ($('#meta').textContent = text);
async function send(type, data) {
  const res = await chrome.runtime.sendMessage({ type, ...data });
  if (res?.error) throw new Error(res.error);
  return res;
}

function show(hits) {
  $('#out').replaceChildren(...hits.map((h) => {
    const a = Object.assign(document.createElement('a'), { href: h.url, target: '_blank', title: h.text, textContent: h.title || h.url });
    a.dataset.score = h.judged?.toFixed(1) ?? h.score.toFixed(2);
    a.classList.toggle('weak', h.judged < 2.5);
    return a;
  }));
}

let seq = 0;
$('form').onsubmit = async (e) => {
  e.preventDefault();
  const query = $('#q').value, mine = ++seq;
  try {
    // Cosine results show right away; the slower judge then re-sorts the top 5.
    const hits = await send('search', { query });
    show(hits);
    const top = hits.slice(0, 5);
    const scores = await send('judge', { query, texts: top.map((h) => h.text) });
    if (mine !== seq) return;   // a newer search has started
    top.forEach((h, i) => (h.judged = scores[i] ?? undefined));
    show([...top.sort((a, b) => (b.judged ?? 0) - (a.judged ?? 0)), ...hits.slice(5)]);
  } catch (err) {
    say(err.message);
  }
};

send('status').then(say, (e) => say(e.message));

And that’s the whole extension.

Step 6: Test the extension

To test the user experience, open and read a few articles for 10 seconds each (scroll a little bit), then open the popup, type what it was about and hit the Enter key. There should now be something like this shown.

The extension popup showing 88 pages indexed and search results for “ai locally”, each with a relevance score

If it doesn’t work

  • Inspect the console log output (as described above).
  • Error mentioning SemanticEmbedder or DecisionModel: turn on Web AI in Settings → Opera AI, or wait for the models to finish downloading.
  • “The model execution session has been destroyed”: a chunk was too long for the model. Reduce the maximum chunk size in capture() from 1,000 characters to 500.
  • “The service is unable to create a new session”: a model is missing. Open the Web AI panel and check that both the Embeddings model and the Decision model show a model.

Wrapping up

That’s search by meaning over your browser history in 151 lines, using two of Opera’s Web AI APIs: SemanticEmbedder finds pages by meaning, and DecisionModel checks which results match. Both run on your machine, so the extension indexes the pages you read without sending them anywhere or paying per page.

This is just an example. To turn it into a real product, you’d still need a few things, like a way to delete pages, a size limit on the index, and re-indexing if the model changes. We look forward to seeing what you build.