# Token Analytics & Hard Budgeting

GoClaw records the tokens every request uses and can refuse new requests once a token limit is reached. Because GoClaw agents act autonomously and can spawn long-running SOPs or Subagents, keeping a hard limit on token use is how you keep API costs under control.

## 1. Token Analytics (Usage Tracking)

When a request finishes, GoClaw records the token counts the provider (e.g., OpenAI, Anthropic, Gemini) reported for it. Background work that calls a model, such as session summaries, fact extraction, embeddings and the image and PDF tools, is recorded the same way.

Each record goes into the `token_usage` table of the state database, with:
- **`user_id`**: The person: their linked ID when they are paired, else their ID on the channel the request came from; for a SOP or cron run, its principal (`cron-system`, `sop-system`); `system` for background work.
- **`agent_id`**: The agent that answered.
- **`session_id`**: The conversation or job the request belonged to; for a SOP run with no conversation, its principal (see the SOP note below).
- **`channel`**: Where it came from (Discord, MS Teams, Facebook, Telegram, App API, Web, or `internal:…` for background work).
- **`message`**: The request's text.
- **`tool_calls`**: How many tool calls the request made.
- **`provider`** & **`model`**: The provider and model that answered.
- **`prompt_tokens`** & **`output_tokens`**: The token counts.
- **`created_at`**: When it was recorded (Unix time).

GoClaw records tokens, not money: there is no cost column and no model pricing to configure.

### Viewing Analytics
You can track token use directly in the **GoClaw Web UI**:
1. Open **Usage** in the dashboard's navigation (the **Token Usage & Analytics** view).
2. Filter by agent, channel, provider, model, or date range.
3. Review the daily and hourly charts and the per-agent and per-channel tables to find spikes caused by looping agents or expensive tool-use chains.

## 2. Circuit Breaker (Hard Token Budgeting)

To prevent runaway agents from draining your API credits, GoClaw checks token limits before every model call.

### How it Works
Before every model call, GoClaw adds up the tokens already recorded, plus what the request in progress has used so far, and compares them with the limits that are set, in this order:

1. **Per-agent daily limit**: tokens the agent used today, across all its sessions and the SOP runs that run as it (a SOP run counts for its owner, where `default`, `main` (unless an agent is configured under that ID) and an owner that names no agent are the default agent; for the route's agent when a webhook starts it, and for the conversation's agent when `sop_run` does).
2. **Per-session daily limit**: tokens the request's session used today.
3. **Per-session hourly limit**: tokens the session used in the current hour.
4. **Global daily limit**: tokens all agents used today.

Days and hours are UTC. Which session a SOP run counts against depends on how it started: a run started from a chat (`sop_run`) uses that chat's session; a run started by a webhook or in the background with no request uses `sop-system`. Runs started by their cron schedule use `cron-system` (a job that runs for a person, with `subject_ref`, uses that person), and runs started from the Web UI's **Run** button use `sop-system`.

* **Below the limits**: the request proceeds.
* **A limit reached**: the request is refused before its next model call (a request already running stops there), and the person gets a message such as `⛔ Token budget exceeded for this session. Used … / … tokens today. …`, `⛔ Hourly token limit reached for this session. …`, `⛔ Agent "…" has reached its daily token limit. …` or `⛔ Global daily token budget exceeded. …`.
* **Near a limit**: at the warning threshold (80% by default) a warning is written to the log; the request still runs.

### Configuration
Global limits are set under `ai.token_budget`; an agent can have its own daily limit under `agents.list[].token_budget`. A limit of `0` (the default) means no limit.

```yaml
ai:
  token_budget:
    daily_limit: 5000000          # tokens per session per day
    hourly_limit: 500000          # tokens per session per hour
    global_daily_limit: 50000000  # tokens across all agents per day
    warning_threshold_pct: 80     # log a warning at this percentage of a limit (default 80; 0 = no warnings)

agents:
  list:
    - id: analytics
      token_budget:
        daily_limit: 2000000      # tokens this agent may use per day
        warning_threshold: 0.8    # log a warning at this fraction (default: warning_threshold_pct; 0 = no warnings)
```

A request's tokens are recorded when it ends, so the check before each further call adds the tokens the request has used so far. A request stopped at a limit has what it used recorded too. A request can still go past a limit by its last call: the call is checked before it is made, not cut off halfway.

## 3. SOP Run Limit

Every SOP run has a token limit of its own, on by default: **500,000 tokens**, prompt and output, for every step, attempt and retry of the run together. It is checked before every model call of the run.

- **When a run reaches it**, it stops before its next call. It ends `failed` whatever the step's `failure_mode` says: no retry, no next step, no `on_failure_goto`. The rollback still runs (shell commands use no tokens). The SOP's owner is told through `sop.alerts` how many tokens the run used. A last step that passes as the run reaches the limit completes the run.
- **A cron job whose run stopped at the limit is paused**, with `token_limit` as its last error on the **Cron** page, and the owner is told. Left on, it would spend as much again at every tick. Resume it once the SOP is fixed or the limit raised.
- **A run past 200,000 tokens is marked heavy**: the log says so, and the SOP's run history shows its token count highlighted.

The run history on the **SOPs** page shows what each run, and each step, used.

```yaml
sop:
  max_run_tokens: 500000    # stop a run here; 0 = no limit
  heavy_run_tokens: 200000  # mark a run heavy from here; 0 = never
```

Raise `max_run_tokens` when your SOPs really need more and you are ready to pay for it. `0` turns the limit off, which is not recommended: the cron pause then never happens either.

**How big should a run be?** A rough guide. A model call sends the whole conversation of the step: the system prompt, the tool definitions, the step's messages and the tool results so far. A step that works takes a few calls. Large query results stay out of the conversation (`data_ref`), so a well-made step rarely sends more than 30–40k tokens a call.

| | One model call | One step | One run |
|---|---|---|---|
| Normal | up to 30k | up to 60k | up to 200k |
| Heavy, worth a look | 30k–80k | 60k–150k | 200k–500k |
| Something is wrong | over 80k | over 150k | over 500k |

A run of millions of tokens is almost always a step that loops: it calls tools until the 20-call limit (`tools.max_tool_iterations`), sends large tool results back in every call, and is then tried again. Look at the run's trace on the **Traces** page: how many model calls each step made, and how large each one was.