AI agents for pentesters and SOC analysts: scan output and logs without the token flood

Pentesters and SOC analysts now hand recon output and alert logs to an AI agent. The agent reasons well about a scan; it reads badly through a few megabytes of one. Every host line and every log event becomes tokens, sent again on each turn. Tokenade folds that output on the computer where your agent runs. This page covers authorized testing and defensive work only.

What you deal with every day

Scan output that overflows the context

Recon tools are thorough by design. A guide on local AI-assisted pentesting warns: "A full Nmap scan of a /16 subnet produces megabytes of output." Its advice is to summarize scan output before it reaches the model (alan-west.hashnode.dev, 2026-04-08).

Agent workflows multiply that. A practitioner describes an agent that runs subdomain and DNS tools against "your authorized target", then feeds the live hosts into a vulnerability scanner: "Each tool's output becomes the next decision" (r/cybersecurity, 2026-07-01). Each of those outputs stays in the session.

Logs that make triage slower and dearer

SOC teams report the same thing from the defensive side. A team running a tier-1 triage agent on Wazuh wrote: "Token cost is real and scales with alert volume." And: "Throwing a day of logs at the agent didn't make verdicts better — it made them slower and more expensive, and it gave prompt injection a larger surface" (simplico.net, 2026-05-02).

Their conclusion: aggressive pre-filtering matters more than the model.

Investigations measured in hundreds of thousands of tokens

Vendors who publish numbers confirm the scale. Elastic Security Labs measured approximately 113k to 243k tokens per automated alert investigation with specialized agents, and 532k to 786k with a single general agent, on their own workflows (elastic.co, 2026-07-23). RunReveal puts its own figure at "About $2 per alert" across its customers' investigations (blog.runreveal.com, 2026-05-28).

Client data and credentials in the output

Engagement output is sensitive by nature. The same pentesting guide calls it "a confidentiality nightmare" when workflows "send sensitive engagement data to cloud endpoints". Credentials discovered during a test also show up in tool output, and then in the agent's context.

What Tokenade changes

Tokenade is a local CLI between the AI agent and its tools. tokenade install wires it into Claude Code, Codex, Cursor, Copilot CLI, OpenCode and 15+ other agents (compatibility).

• Pre-filtering, built in. Tokenade folds command output before the agent reads it. For tools without a dedicated compactor, it detects the format (tables, JSON, XML, logs, stack traces) and folds by structure (how compaction works).
• Rules for your scanners. A short TOML file tells Tokenade what to drop from a given tool's output, such as closed ports or informational lines, with regexes and line caps. No code (custom compactors).
• Nothing lost. When output is still large, the agent gets a preview; the full result stays in a local stash for 7 days. tokenade expand-ref <hash> --grep <pattern> returns exact lines, such as one host or one alert ID (details).
• MCP tool results folded. If your toolkit runs as an MCP server, tokenade mcp-wrap folds its large results too (wrap MCP servers).
• Credentials masked. Keys, tokens, private keys and high-entropy strings it recognises are replaced with <redacted> before the agent reads them. Set TOKENADE_REDACT_PII=1 to also mask e-mail addresses, phone numbers, IP and MAC addresses.
• Savings per row. tokenade gain shows what each compaction saved, so you can see which tools cost the most.

On long sessions, Tokenade made sessions 39% cheaper in our public THOL benchmark. Recon and hunting sessions tend to be long.

Confidentiality: what stays on your computer

Folding and masking run on the computer where the agent runs. Scan results, logs and command arguments are never sent to Tokenade's servers. Tokenade sends one usage row per compaction (token counts, compactor, agent, model, program name) to meter your plan (what Tokenade sends and stores).

What it does not change:

• The model provider still sees what the agent reads. Folded output is shorter, not private. Your rules of engagement and your provider's data terms still apply.
• Hostnames, client names and findings pass through. Only credentials, plus contact and network identifiers if you turn that on, are masked.
• It is not a security tool. It does not scan, detect or decide anything. It is a layer that makes your agent's input shorter.

For SOC teams: Tokenade fits analysts who run an agent on their own workstation, for triage, hunting or writing queries. A triage pipeline built directly on a model API is a different setup; Tokenade is not a SIEM integration.

Setting it up

1. Install. Follow Install Tokenade on the machine where your agent runs, for example your testing workstation.
2. Wire your agents. tokenade install adds hooks or a plugin to each agent it finds and wraps its MCP servers.
3. Add rules for your tools. Start with the defaults, check tokenade gain after a session, then write a TOML rule for the scanner that still costs the most.

FAQ

Will folding hide a finding from the agent?

Compactors drop repetition and progress noise, and keep errors and structure. The full output stays in the local stash for 7 days, and the agent can retrieve any line with tokenade expand-ref. Review the results the same way you would without Tokenade.

Does Tokenade send my client's scan data anywhere?

Not to Tokenade. Folding runs on your computer and only usage counts reach tokenade.net. The folded output does go to the model provider your agent uses, as it would without Tokenade.

Does it work with a SOC agent built on the API?

Tokenade is designed for agents that run on your computer: Claude Code, Codex, Cursor and others. A custom pipeline on a model API is outside that scope. Analysts who triage with a local agent get the full coverage.

Can it mask IP addresses?

Yes, if you set TOKENADE_REDACT_PII=1, which also masks e-mail addresses, phone numbers and MAC addresses. It is off by default because these values appear constantly in ordinary output.

Further reading

• Output filtering: trim command logs for agents
• How to reduce tokens on long agent sessions