# Gemini API Implementation

**Primary files:** `internal/agent/gemini.go`, `internal/agent/gemini_stream.go`
**SDK:** `google.golang.org/genai` v1.67.0
**Default model:** `gemini-2.5-flash`

GoClaw uses the native `google.golang.org/genai` SDK for Gemini. Gemini has its
own streaming tool loop because native `genai.Part`, function calls, grounding
metadata, executable-code parts, and thought signatures do not map cleanly to
LangChainGo's `llms.ToolCallResponse` model.

## Processing Path

`GeminiAgent.ProcessStream()` is the primary execution path. `Process()` only
wraps it with `DrainToResult()` for legacy callers.

```go
eventStream, err := agent.ProcessStream(ctx, content, history)
```

The native loop:

1. Checks token budget before making the request.
2. Builds the RAG query from recent history plus the current user message.
3. Selects skills and tools with `GetStaticSystemPromptFor()` and `GetToolsFor()`.
4. Converts session history to `[]*genai.Content`.
5. Appends the current user message as `genai.Content{Role: "user"}`.
6. Streams `GenerateContentStream` responses into `AgentEvent` values.
7. Runs tool calls through the shared bounded worker pool.
8. Persists trace data and emits final `EventDone`.

The Gemini loop must stay aligned with `RunAgentLoop()` in
`internal/agent/process.go` for shared behavior such as token budget checks,
tool RAG, large-result spilling, loop detection, worker-pool execution, and
context compaction.

## GenerateContentConfig

`GeminiAgent.buildGenConfig()` builds the request configuration:

```go
gCfg := &genai.GenerateContentConfig{
    SystemInstruction: &genai.Content{
        Parts: []*genai.Part{{Text: systemPrompt}},
    },
    ToolConfig:      toolCfg,
    MaxOutputTokens: resolveMaxOutputTokens(cfg, a.model),
}
```

Current behavior:

| Option | Status |
|---|---|
| `SystemInstruction` | Static system prompt plus the session's context, with skill RAG applied per request. What was found for the message (and the time) is a part of the person's message, after its text. |
| `Tools` | RAG-selected custom function declarations plus optional Gemini Google Search. |
| `ToolConfig.FunctionCallingConfig.Mode` | `FunctionCallingConfigModeAuto`. |
| `ToolConfig.IncludeServerSideToolInvocations` | Enabled when Gemini Google Search is enabled. |
| `MaxOutputTokens` | `ai.gemini.max_output_tokens`. Unset, it is `32768` when `thinking_level` or `thinking_budget` is set and the model thinks, otherwise `16384`. A SOP step's call asks for `sop.max_output_tokens` when that is higher. |
| `ThinkingConfig` | Optional: `thinking_level` to Gemini 3 and later, `thinking_budget` to Gemini 2.5 (see below). |
| `Temperature` | Optional via `ai.gemini.temperature`, sent to Gemini 2.5 and older only. |
| `CachedContent` | Intentionally disabled. |

## Thinking Configuration

`ai.gemini.thinking_level` supports `off`, `minimal`, `low`, `medium`, `high`,
and the empty string for the API default. `ai.gemini.thinking_budget` is a
token budget from 0 to 65536.

How much a model thinks is set in one of two ways, and a request carries only
one of them (`internal/geminimodel`, `resolveThinkingConfig`):

| Model | Sent | From `thinking_level` | From `thinking_budget` |
| :--- | :--- | :--- | :--- |
| Gemini 3 and later, and any Gemini name not listed here | `thinking_level` | as is; `off` is the least the model takes (it cannot stop thinking); `minimal` only to models that accept it (3 Flash, 3.1 Flash-Lite, 3.5 Flash, 3.6 Flash), otherwise `low` | 0 → the least level, ≤1024 → `low`, ≤8192 → `medium`, more → `high`; the level wins when both are set |
| Gemini 2.5 | `thinking_budget` | `off` → 0 (128 on 2.5 Pro, which cannot stop thinking), `minimal`/`low` → 1024, `medium` → 8192, `high` → 24576 | as is (0 on 2.5 Pro is sent as 128); the budget wins when both are set |
| Gemini 1.x, 2.0, non-Gemini names | nothing | ignored, with one warning | ignored, with one warning |

Google retired `thinking_budget` for Gemini 3: those models only remapped it
to a level, and newer ones answer it with 400 `INVALID_ARGUMENT`. The API
also rejects `thinking_level` on models before Gemini 3, and both settings
together on any model.

## Sampling Parameters

Gemini 3 and later are never sent `temperature`, `top_p` or `top_k`: since
Gemini 3.6 Flash they are deprecated and set to their defaults whatever is
sent, and newer models reject them. `ai.gemini.temperature` is sent to Gemini
2.5 and older only; for a newer model it is dropped with one warning. The
background calls that ask for a low temperature (metadata extraction, the
skill quality score) do the same (`geminimodel.SamplingParams`).

Provider `gemini-legacy` is a deprecated name for this provider: everything that
names it is served by this SDK. The gateway already served `ai.provider:
gemini-legacy` this way; `goclaw chat` and an agent model written
`gemini-legacy/<model>` used LangChainGo's Google client, which sent
`temperature` 0.5, `top_p` 0.95 and `top_k` 3 on every request and no thinking
settings, until it was removed.

## Multimodal Input

Web chat uploads are saved under `workspace/uploads` and passed through
`gateway.buildContentParts()`.

For image attachments, `buildContentParts()` embeds a marker:

```text
__GCIMG__:<mime_type>:<base64_data>
```

`gemini_stream.go` parses this marker in `parseContentParts()` and converts it
to:

```go
&genai.Part{
    InlineData: &genai.Blob{
        MIMEType: mimeType,
        Data:     imgBytes,
    },
}
```

That keeps the public `ProcessStream(ctx, content string, history)` signature
unchanged while allowing Gemini to analyze image attachments natively. Non-image
attachments are referenced by workspace-relative path so tools such as
`file_read`, `pdf`, `spreadsheet`, or `voice_input` can process them.

## Gemini Built-In Tools

GoClaw supports Gemini Google Search through `ai.gemini.google_search`.

When enabled, Google Search is added as a separate `genai.Tool` entry because the
Gemini API does not allow `FunctionDeclarations` and `GoogleSearch` in the same
tool object.

```go
tools := []*genai.Tool{{FunctionDeclarations: customDecls}}
if cfg.AI.Gemini.GoogleSearch {
    tools = append(tools, &genai.Tool{GoogleSearch: &genai.GoogleSearch{}})
}
```

Grounding citations are appended to the response by
`formatGroundingCitations()`.

Gemini Code Execution is not currently wired into the main agent loop. Use
GoClaw's existing execution and analysis tools instead.

## Tool Schema Conversion

The conversion pipeline is:

```mermaid
flowchart LR
    REG["registered tools"] --> GET["GetToolsFor(ctx, ragContext)"]
    GET --> LLMS["[]llms.Tool"]
    LLMS --> CONV["toGenaiDeclarations()"]
    CONV --> DECL["[]*genai.FunctionDeclaration"]
    DECL --> TOOL["genai.Tool{FunctionDeclarations}"]
```

`convertToSchema()` handles JSON-schema object, string, number, integer,
boolean, and array types.

## Context Caching

Gemini context caching remains disabled. The Gemini API does not allow
`system_instruction`, tools, or tool config to be set on `GenerateContent` when
`CachedContent` is used. Enabling caching would weaken per-query Tool RAG and
Skill RAG, so GoClaw favors dynamic context assembly over cache token savings.

## Maintenance Checklist

When changing the Gemini path, verify both agent loops where applicable:

- `internal/agent/gemini_stream.go`
- `internal/agent/process.go`

Also run:

```bash
go test ./internal/agent -count=1
```