Search · scrape · summarise

Search results and pages, turned into a summary you can cite.

A FastAPI service that scrapes search results and web pages, then synthesises an executive summary with any of 54 model providers, from a local Ollama to the hosted ones. API only, no UI.

Python 3.12+ Serves on :8006 Keys stay in keyring MIT licensed
POST /v1/search
{
  "queries": [
    {"query": "widget latency causes", "max_results": 10},
    {"query": "dispatch queue sharding", "site": "reddit.com"}
  ],
  "fetch_pages": true,
  "max_pages": 3,
  "additional_notes": "The reader is an SRE deciding whether to shard."
}

# one bad query never fails the batch: every item has its own status
Per callerProvider keys are resolved per request, for the person asking.
Partial successOne bad URL or blocked query never fails the batch.
Background modeLong work answers 202 with a job to poll, same result shape.
Only what worksThe model list shows what is reachable for you, right now.

How it works

Three operations, one summary step, and no key of its own.

Every third-party credential is stored in keyring and resolved per request. Local model runtimes need no credential and work without keyring at all.

01

Search

A batch of queries, each with its own notes and optional site filter. By default the summary is built from titles and snippets; fetch_pages also reads the top pages.

02

Scrape

A batch of URLs to clean extracted text. render_js is auto, escalating from plain HTTP to a headless browser when the page looks client-rendered, or always, or never.

03

Summarise

Text in, summary out, with the model you name as provider:model. A direct hook for an MCP server.

The API

Seven routes, and interactive docs at /docs.

The reference is docs/api.md; the provider catalogue is docs/providers.md.

POST /v1/search

Search

Batch queries to results, an optional deep fetch, and a summary.

POST /v1/scrape

Scrape

Batch URLs to clean extracted text and a summary.

POST /v1/summarize

Summarise

Text in, summary out.

GET /v1/models

Models

What is reachable right now, with per-model capabilities.

GET /v1/jobs

Jobs

Poll, list and cancel work started in background mode.

GET /ready

Readiness

200 when a browser can launch and a provider is configured; otherwise 503 naming which.

Run it

Three commands.

You need Python 3.12 or newer and uv.

  1. Install.
    make install
  2. Point it at your keyring.
    cp .env.example .env
  3. Serve it; the interactive docs are at /docs.
    make run

Where provider keys live, and how they are resolved per caller, is in docs/keyring.md.

What to know first

What to know before you lean on it.

Where do provider keys live?

Not here. Every third-party credential is stored in keyring and resolved per request for the person the request is made for, so answers are per caller.

What happens when a search engine blocks the request?

That item carries its own error and the batch still answers 200. A page that cannot be fetched simply keeps its snippet: deep fetching is enrichment.

Do background jobs survive a restart?

No. Jobs live in the process that accepted them: they are lost on restart and are not visible to other replicas, so behind a load balancer a client must poll the instance it submitted to.

Is it affiliated with a search engine or a model provider?

No. It is independent software and is not affiliated with Google, or with any model provider it can call.

Docs

Written for the person who has to run it.