# web and search

> tokenade web turns a web page into compact markdown; tokenade search queries several independent engines and returns merged, ranked results.

Source: https://tokenade.net/en/docs/web-search · Updated: 2026-10-07

`tokenade web` fetches a page and returns it as compact markdown, without scripts, menus or markup. `tokenade search` runs one query across several independent search engines and returns merged, deduplicated results. Both take several targets at once and run them concurrently.

## web

```text
tokenade web <url|file>… [--prompt "…"] [--max-tokens N]
```

| Form | Example |
| --- | --- |
| One URL | `tokenade web https://example.com` |
| Several URLs, fetched concurrently | `tokenade web https://a.com https://b.com` |
| Piped HTML | `curl -s https://example.com \| tokenade web -` |
| Saved file | `tokenade web saved.html` |

| Flag | Effect |
| --- | --- |
| `--prompt "q1, q2"` | Return only the passages that answer each question, under its own heading. Also `--question`, `-q` |
| `--max-tokens N` | Bounds the input that gets parsed, not the reply. The page is still converted whole, so the answer is the same size with or without it. To bound the reply, ask a question with `--prompt` |

```text
$ tokenade web https://example.com
# Example Domain

This domain is for use in documentation examples without needing permission. …
```

```text
$ tokenade web https://doc.rust-lang.org/std/keyword.match.html \
    --prompt "can match guards bind variables, what is exhaustiveness checking"
### Q1 · can match guards bind variables
[tokenade: relative links below resolve under https://doc.rust-lang.org]
…
match a_number {
    Some(x) if x <= 5 => println!("0 to 5 num = {x}"),
    Some(x @ 6..=10) => println!("6 to 10 num = {x}"),
…
```

Links in the page are kept usable: relative links are rewritten from the site root, links into the same page become anchors, and links to other sites are written in full.

## search

```text
tokenade search "<query>"… [--answer] [--json]
tokenade search "<q1>" "<q2>"       SEVERAL queries, run concurrently
tokenade search <file>|-            compact a SAVED results page
tokenade search --diagnose          probe every engine
```

| Flag | Effect |
| --- | --- |
| `--answer` | Fetch the top results and return the passages, not links |
| `--json` | Machine-readable output: `engines`, `query`, `refused`, `results[]` with `title`, `url`, `snippet` |
| `--diagnose` | Probe every engine and report which ones answer |
| `--html`, `--serp` | Treat the argument as a saved results page even if it reads like a query |

Each quoted argument is a separate query. Results are merged, deduplicated and ranked by how many engines agree. The output is title, URL and snippet only, which costs a fraction of reading a results page.

```text
$ tokenade search "rust anyhow vs thiserror"
[tokenade:search via yahoo] "rust anyhow vs thiserror":

1. Rust Error Handling: thiserror, anyhow, and When to Use Each
   https://momori.dev/posts/rust-error-handling-thiserror-anyhow/
   Feb 6, 2024 · In this blog post, we'll explore strategies for …
2. Rust Error Handling Compared: anyhow vs thiserror vs snafu
   https://dev.to/leapcell/rust-error-handling-compared-anyhow-vs-thiserror-vs-snafu-2003
   …
```

Engines used: Bing, Brave, DuckDuckGo, Ecosia, Marginalia, Mojeek and Yahoo. `--diagnose` shows which are healthy from your network:

```text
$ tokenade search --diagnose
probing 7 engines with "python asyncio vs threading"

engine bytes hits ms verdict
ecosia 0 0 1705 unreachable
bing 132143 10 223 ok
brave 365044 20 816 ok
duckduckgo 34856 10 696 ok
mojeek 5516 0 132 thin
yahoo 165192 7 842 ok
marginalia 1029 0 188 thin

4/7 engines healthy
```

Follow a result with `tokenade web <url>` to read the page itself.

## Inside your agent

On Claude Code, Tokenade's hooks also handle the agent's own web tools:

- **WebSearch**: the search runs locally across the same engines and replaces the native result. If the engines fail or are too slow, the native tool runs instead.
- **WebFetch**: intercepted only when it pays (PDFs and other binaries, very large pages); everything else goes through untouched.

See [how hooks intercept tool calls](https://tokenade.net/en/docs/hooks) for the full list.

## Turning web features off

| Variable | Effect |
| --- | --- |
| `TOKENADE_NO_WEBSEARCH=1` | Tokenade no longer replaces or compacts the agent's web search, `tokenade search` refuses to run (exit `3`), and the rules Tokenade writes stop mentioning it |
| `TOKENADE_NO_WEBFETCH=1` | Tokenade no longer intercepts the agent's page fetches |

Use these when web queries must leave only through your AI vendor. For Claude Code, set them in the `env` block of `~/.claude/settings.json`, and run `tokenade install` once with the variable set in your shell so the written rules are updated.

## Exit codes and gotchas

- `0` success, `1` the page or file could not be read by `web`, `2` usage error (unknown flag, a path-shaped `search` argument that does not exist), `3` search disabled by `TOKENADE_NO_WEBSEARCH`.
- A mistyped flag is refused, not sent as a query.
- `tokenade search /path/that/does/not/exist.html` is refused instead of being searched, so a local path never leaves your machine.
- Quote each multi-word query: `tokenade search "q1" "q2"` runs two searches.
- A results page larger than the stash threshold is folded; recover details with [`tokenade expand-ref`](https://tokenade.net/en/docs/expand-ref-disclose).

## Related

- [Read documents and media](https://tokenade.net/en/docs/read-documents-media)
- [How hooks intercept tool calls](https://tokenade.net/en/docs/hooks)
- [Get folded output back](https://tokenade.net/en/docs/expand-ref-disclose)
- [Privacy and data](https://tokenade.net/en/docs/privacy-telemetry)
