# Metrics Store Tool

`metrics_store` is a structured metric memory for computed values such as cycle time, lead time, error rate, MTTR, CPU load, or custom SRE metrics. It stores already-computed numbers so agents can query and trend them later without calling the original API again.

## Backends

The tool supports two storage modes:

| Mode | Config required | Notes |
|---|---|---|
| Built-in | No | Uses GoClaw's own state database (`database:`, SQLite by default) |
| External datasource | Yes | Use `datasource=<name>` to keep metrics in a Postgres or ClickHouse database |

For external datasources, define a normal datasource in `config.yaml` and pass its name to the tool. A datasource is read-only unless it sets `allow_write: true`:

- `record`, `annotate` and `purge` need `allow_write: true`, and an administrator or an admin-authored SOP or cron run. A webhook-triggered run, a user in a conversation, and a SOP run a user started are refused even with `allow_write`, and told to use the built-in store (omit `datasource`).
- `query` and `trend` are open to everyone. They create nothing: on a datasource where no metrics were ever recorded, they say so.
- A run acting on behalf of a person (a delegated SOP run) cannot use an external datasource at all, because metrics_store cannot confine what it reads there.
- On ClickHouse, `annotate` is refused: ClickHouse does not update rows in place.

```yaml
datasources:
  metrics_db:
    driver: postgres
    dsn: "postgres://user:pass@localhost/metrics?sslmode=disable"
    allow_write: true   # lets metrics_store (and db_write) write here
```

The external table is created by the first write. Postgres uses `agent_metrics` with JSONB tags; ClickHouse uses a `MergeTree` table.

## Actions

| Action | Purpose | Required fields |
|---|---|---|
| `record` | Store one or more metric values | `source`, `entity_type`, `entity_id`, `metrics` |
| `query` | Read matching metric rows | Optional filters |
| `trend` | Week-over-week aggregation | Optional filters |
| `annotate` | Add notes to existing records | `source`, `entity_id`, `notes` |
| `purge` | Delete old rows | `date_from` |

Date fields accept absolute `YYYY-MM-DD` dates or relative offsets such as `7d`, `4w`, and `3m`.

## Record Metrics

```bash
goclaw tools exec metrics_store action=record source=gitlab entity_type=mr entity_id=myorg/backend#42 metrics='{"cycle_time_h":18.5,"review_time_h":4.2}' tags='{"sprint":"W13","team":"backend"}' notes='Waited on review'
```

Using an external datasource:

```bash
goclaw tools exec metrics_store datasource=metrics_db action=record source=uptrace entity_type=service entity_id=api-service metrics='{"error_rate":0.023,"p95_ms":410}'
```

## Query Metrics

```bash
goclaw tools exec metrics_store action=query filter_source=gitlab filter_entity_type=mr filter_metric=cycle_time_h date_from=4w limit=100
goclaw tools exec metrics_store datasource=metrics_db action=query filter_source=uptrace date_from=2026-06-01 date_to=2026-06-20
```

`query` returns a compact Markdown table with source, entity type, entity ID, timestamp, metric name, value, and tags (notes are stored but not shown). It returns 50 rows unless `limit` says otherwise (at most 500).

## Trend Metrics

```bash
goclaw tools exec metrics_store action=trend filter_source=gitlab filter_metric=cycle_time_h weeks=8
goclaw tools exec metrics_store datasource=metrics_db action=trend filter_source=uptrace filter_metric=error_rate weeks=4
```

`trend` aggregates by week and returns count, average, min, and max for the selected metric.

## Annotate Metrics

```bash
goclaw tools exec metrics_store action=annotate source=gitlab entity_id=myorg/backend#42 notes='Hotfix; merged faster than the normal release path'
```

## Purge Old Metrics

```bash
goclaw tools exec metrics_store action=purge date_from=6m filter_source=gitlab
```

Without `filter_source`, `purge` deletes all records older than the threshold.

## Example Workflows

GitLab cycle time:

```text
1. Use gitlab mr_label_events or issue_label_events.
2. Calculate time between workflow labels.
3. Store cycle_time_h, review_time_h, and lead_time_h with metrics_store record.
4. Use metrics_store trend for weekly reporting.
```

Uptrace error rate:

```text
1. Use uptrace_query to fetch span or service aggregates.
2. Calculate error_rate or p95_ms.
3. Store metrics by service name.
4. Trend them by week or report them through SOP automation.
```

Weekly SOP sketch. A SOP has no schedule of its own: a [cron job](automation/cron-jobs.md) that names its `sop_id` runs it, here with the schedule `0 0 9 * * 1` (Mondays at 09:00; six fields, seconds first). The steps are abridged; see [SOPs](automation/sop.md) for every step field.

```json
{
  "id": "weekly-metrics-report",
  "name": "Weekly metrics report",
  "steps": [
    {"id": "s1", "name": "Last week", "tools": ["metrics_store"],
     "instruction": "Call metrics_store with action=query and date_from=7d."},
    {"id": "s2", "name": "Trend", "tools": ["metrics_store"],
     "instruction": "Call metrics_store with action=trend and weeks=4."},
    {"id": "s3", "name": "Report", "tools": ["generate_pdf"], "uses_outputs": ["s1", "s2"],
     "instruction": "Write a PDF report from the week's metrics and the trend."},
    {"id": "s4", "name": "Send", "tools": ["message"], "uses_outputs": ["s3"],
     "instruction": "Send the report to the team's Discord channel."}
  ]
}
```

## Paired SOP Architectural Pattern (`<topic>_hourly` + `<topic>_daily_summary`)

For continuous monitoring, an established best practice is pairing a frequent collection SOP with a daily roll-up summary SOP via `metrics_store`:

```mermaid
flowchart TD
    HOURLY["<b>Hourly SOP: &lt;topic&gt;_hourly</b> (Runs: 0 0 * * * *)<br/>• Polls raw APIs / logs / spans<br/>• Computes numeric values<br/>• metrics_store action=record"]
    STORE[("<b>metrics_store</b><br/>SQLite, PostgreSQL, or ClickHouse")]
    DAILY["<b>Daily SOP: &lt;topic&gt;_daily_sum</b> (Runs: 0 0 22 * * *)<br/>• metrics_store action=query<br/>• metrics_store action=trend<br/>• Generates executive report<br/>• Sends notification to Teams"]

    HOURLY -->|"saves metrics"| STORE
    STORE -->|"queries 24h data"| DAILY
```

### Why use this pattern?
- **Avoids context-window overflow**: Raw logs and telemetry from an entire day would overwhelm the LLM's prompt window.
- **Resilient & cheap**: If a single hourly run fails or encounters an API glitch, the other 23 hours of data remain intact.
- **No notification fatigue**: Team channels are not spammed every hour; only the daily 22:00 rollup report is posted, while critical threshold alerts can still trigger independently.


## Benchmarking

```bash
goclaw tools exec metrics_store action=query
goclaw tools bench metrics_store action=query filter_source=gitlab --runs=10
```

## Implementation Files

- `internal/agent/tool_metrics.go`
- `internal/store/models.go` (GORM AutoMigrate creates the `agent_metrics` table)
- `config.yaml.example`