# LLM Providers

GoClaw has **5 native providers** (Gemini, OpenAI, Anthropic, Ollama, DeepSeek), embedding providers and an OpenAI-compatible API, where OpenClaw supports 30+ providers. Each agent is served by its own provider and, when that fails, by the providers of its fallback list (see [Provider Fallback & Cooldown System](#provider-fallback--cooldown-system)).

## Architecture

```mermaid
flowchart TD
    subgraph PROV["GoClaw Provider Architecture"]
        P1["<b>Gemini</b><br/>Native SDK (genai)"]
        P2["<b>OpenAI</b><br/>openai-go"]
        P3["<b>Anthropic</b><br/>anthropic-sdk-go"]
        P4["<b>Ollama (local)</b><br/>Ollama api"]
        P5["<b>DeepSeek</b><br/>openai-go, DeepSeek's API"]

        FB["<b>Per-Agent Selection &amp; Failover</b><br/>ai.provider or agents.list[].model, then ai.fallback<br/>(or agent fallback) in order, with cooldowns, until a tool has run"]

        EMB["<b>Embedding Providers (3)</b><br/>OpenAI / Gemini / Ollama"]
        OAI["<b>OpenAI-Compatible API</b><br/>/v1/chat/completions (streaming + sync)"]

        P1 --> FB
        P2 --> FB
        P3 --> FB
        P4 --> FB
        P5 --> FB
        FB --- EMB
        EMB --- OAI
    end
```

## Chat/Agent Providers

| # | Provider | Library | Status | Notes |
|---|----------|---------|--------|-------|
| 2.1 | **Google Gemini (Studio)** | Native `google.golang.org/genai` SDK | Done | Primary provider — Gemini 2.5/3+ support, `thought_signature`, native function calling, and Thinking Configuration. |
| 2.2 | **Google Vertex AI** | Native `google.golang.org/genai` SDK (Vertex AI backend) | Done | Enterprise-grade deployment via Google Cloud (gcloud ADC support). |
| 2.3 | **OpenAI** | Official `github.com/openai/openai-go/v3` SDK | Done | GPT-4o and later, through Chat Completions in the shared agent loop (`openai_stream.go`). `ai.openai.base_url` points it at any OpenAI-compatible server. |
| 2.4 | **Anthropic (Claude)** | Native `github.com/anthropics/anthropic-sdk-go` SDK | Done | The shared agent loop with a native adapter (`claude_stream.go`): thinking blocks sent back whole, all of a round's tool results in one message, prompt caching, `ai.anthropic.effort`, and a fallback model on a refusal where the model takes one. Default model `claude-sonnet-5-5`; set another with `ai.anthropic.model`. |
| 2.5 | **Ollama (local)** | Ollama's official Go client (`github.com/ollama/ollama/api`) | Done | Local models through the native `/api/chat` (`ollama_stream.go`); each call asks for a context window its prompt fits in (`num_ctx`). |
| 2.6 | **Gemini (Legacy)** | `gemini-legacy` | Deprecated | A deprecated name for `gemini`, served by the native genai SDK. The gateway already served `ai.provider: gemini-legacy` this way; `goclaw chat` and `gemini-legacy/<model>` used LangChainGo's Google client, which sent `temperature`, `top_p` and `top_k` on every request. |
| 2.7 | **DeepSeek** | Official `github.com/openai/openai-go/v3` SDK, DeepSeek's base URL | Done | DeepSeek publishes no Go SDK; its documentation calls its API with the OpenAI SDK. `deepseek-flash` (default; reads images) and `deepseek-v4-pro` via `https://api.deepseek.com`, through the OpenAI adapter (`openai_stream.go`) in the shared agent loop, with what DeepSeek's API adds: the reasoning (`reasoning_content`) goes back with the tool calls it came with and is never shown, `ai.deepseek.reasoning_effort` sets thinking (`none`, `low`, `high`, `max`), `ai.deepseek.max_output_tokens` is a call's limit with thinking included (default 16000), background calls do not think, and a response ended with `insufficient_system_resource` is asked again. |

...

## Chat/Agent Features

### Gemini Thinking Configuration
For Gemini 2.5 and later, GoClaw sets how much the model thinks from `ai.gemini.thinking_level` (`off`, `minimal`, `low`, `medium`, `high`) or `ai.gemini.thinking_budget`. Gemini 3 and later are sent `thinking_level` only, Gemini 2.5 `thinking_budget` only; see [Gemini API Implementation](gemini_implementation.md#thinking-configuration).

### Google Search Grounding
GoClaw supports server-side **Google Search Grounding** for Gemini models. When enabled, the model can query Google Search in real-time to provide up-to-date information, with citations automatically formatted and appended to the response.

## Embedding Providers

GoClaw supports 3 embedding providers for memory search (same providers used in chat):

| # | Provider | Model | Dimensions | Status |
|---|----------|-------|:----------:|--------|
| 2.E1 | OpenAI | `text-embedding-3-small` | 1536 | Done |
| 2.E2 | Gemini | `text-embedding-004` | 768 | Done |
| 2.E3 | Ollama | configurable | varies | Done |

## Provider Fallback & Cooldown System

An agent's provider failing before the reply used a tool hands the request to the next provider of
`ai.fallback` (or the agent's `agents.list[].fallback`); see
[Provider Fallback & Cooldown](provider_fallback.md).

`internal/agent/fallback.go` + `internal/agent/provider_cooldown.go`

| # | Feature | OpenClaw | GoClaw | Status |
|---|---------|----------|---------|--------|
| 2.F1 | Multi-provider fallback chain | ✅ Provider rotation | ✅ `ai.fallback` / `agents.list[].fallback` | Done |
| 2.F2 | Error classification (8 types) | Basic retry | ✅ auth, billing, rate_limit, overloaded, timeout, model_missing, format, unknown | **GoClaw only** |
| 2.F3 | Exponential backoff cooldown | ✅ | ✅ `60s × 5^min(n-1, 3)`, max 1 hour | Done |
| 2.F4 | Probe throttling | — | ✅ Primary provider only, 30s minimum interval | **GoClaw only** |
| 2.F5 | Context overflow detection | ✅ | ✅ `isContextOverflow` — stops fallback chain, returns error | Done |
| 2.F6 | Failure window reset | — | ✅ 10 min no-error → counter reset | **GoClaw only** |
| 2.F7 | Probe state TTL + eviction | — | ✅ 24h TTL, max 256 keys | **GoClaw only** |
| 2.F8 | Per-provider usage stats | — | ✅ `ProviderUsageStats` — lastUsed, errorCount, failureCounts, disabled | **GoClaw only** |
| 2.F9 | Disabled vs Cooldown distinction | — | `IsDisabled` checks `DisabledUntil`, which nothing sets; only the cooldown is ever applied | Unused |
| 2.F10 | Provider health dashboard | ✅ | — | **TODO** |

### Cooldown Backoff Formula

```
cooldown_ms = min(60000 × 5^min(errorCount-1, 3), 3600000)

Error 1:  1 min
Error 2:  5 min
Error 3:  25 min
Error 4+: 60 min (max)
```

### Error Classification (`classifyError`)

| Keyword Pattern | Failure Reason |
|----------------|----------------|
| 401, 403, unauthorized | `auth` |
| 402, billing, quota | `billing` |
| 429, rate limit | `rate_limit` |
| 503, overloaded | `overloaded` |
| timeout, deadline | `timeout` |
| 404, model not found | `model_not_found` |
| 400, invalid | `format` |
| (other) | `unknown` |

## OpenAI-Compatible API

`internal/gateway/openai_compat.go`

External tools (Cursor, Continue.dev, etc.) can use GoClaw as OpenAI API:

| # | Feature | OpenClaw | GoClaw | Status |
|---|---------|----------|---------|--------|
| 2.C1 | `/v1/chat/completions` POST | ✅ | ✅ `HandleChatCompletions` | Done |
| 2.C2 | Streaming (SSE) | ✅ | ✅ `text/event-stream` | Done |
| 2.C3 | Non-streaming | ✅ | ✅ JSON response | Done |
| 2.C4 | Token usage estimation | ✅ | ✅ ~4 chars/token approximation | Done |
| 2.C5 | Subagent command routing | — | ✅ `HandleSubagentCommand` pass-through | **GoClaw only** |
| 2.C6 | Session persistence | ✅ | ✅ Per-IP session (`openai-compat:<ip>`) | Done |
| 2.C7 | Model routing | model → provider | Uses default agent | Partial |
| 2.C8 | `/v1/models` list | ✅ | — | **TODO** |
| 2.C9 | Function calling pass-through | ✅ | — | **TODO** |
| 2.C10 | Embeddings API | ✅ | — | **TODO** |

---

## OpenClaw vs GoClaw Provider Comparison

| Aspect | OpenClaw | GoClaw |
|--------|----------|---------|
| **Total providers** | 30+ | 5 (Gemini + OpenAI + Anthropic + DeepSeek + Ollama; `gemini-legacy` is served as Gemini) |
| **Provider framework** | JS plugin system | Each provider's official Go client (`genai`, `anthropic-sdk-go`, `openai-go`, Ollama's `api`); DeepSeek on `openai-go` |
| **Fallback system** | Basic rotation | `ai.fallback` / `agents.list[].fallback`, tried in order before a tool has run |
| **Cooldown tracking** | — | Exponential cooldown per provider, with probes of the agent's own provider |
| **Embedding providers** | Multi-provider | 3 (OpenAI, Gemini, Ollama) |
| **OpenAI-compat API** | ✅ | ✅ + subagent command routing |
| **Local models** | Ollama | Ollama (same) |

**Coverage:** 5 chat providers, 3/3 embedding providers, provider fallback with cooldowns, 7/10 OpenAI-compat features.