Gemini API Implementation

Primary files: internal/agent/gemini.go, internal/agent/gemini_stream.go SDK: google.golang.org/genai v1.67.0 Default model: gemini-2.5-flash

GoClaw uses the native google.golang.org/genai SDK for Gemini. Gemini has its own streaming tool loop because native genai.Part, function calls, grounding metadata, executable-code parts, and thought signatures do not map cleanly to LangChainGo's llms.ToolCallResponse model.

Processing Path

GeminiAgent.ProcessStream() is the primary execution path. Process() only wraps it with DrainToResult() for legacy callers.

eventStream, err := agent.ProcessStream(ctx, content, history)

The native loop:

1. Checks token budget before making the request.
2. Builds the RAG query from recent history plus the current user message.
3. Selects skills and tools with GetStaticSystemPromptFor() and GetToolsFor().
4. Converts session history to []*genai.Content.
5. Appends the current user message as genai.Content{Role: "user"}.
6. Streams GenerateContentStream responses into AgentEvent values.
7. Runs tool calls through the shared bounded worker pool.
8. Persists trace data and emits final EventDone.

The Gemini loop must stay aligned with RunAgentLoop() in internal/agent/process.go for shared behavior such as token budget checks, tool RAG, large-result spilling, loop detection, worker-pool execution, and context compaction.

GenerateContentConfig

GeminiAgent.buildGenConfig() builds the request configuration:

gCfg := &genai.GenerateContentConfig{
    SystemInstruction: &genai.Content{
        Parts: []*genai.Part{{Text: systemPrompt}},
    },
    ToolConfig:      toolCfg,
    MaxOutputTokens: resolveMaxOutputTokens(cfg, a.model),
}

Current behavior:

Option	Status
SystemInstruction	Static system prompt plus the session's context, with skill RAG applied per request. What was found for the message (and the time) is a part of the person's message, after its text.
Tools	RAG-selected custom function declarations plus optional Gemini Google Search.
ToolConfig.FunctionCallingConfig.Mode	FunctionCallingConfigModeAuto.
ToolConfig.IncludeServerSideToolInvocations	Enabled when Gemini Google Search is enabled.
MaxOutputTokens	ai.gemini.max_output_tokens. Unset, it is 32768 when thinking_level or thinking_budget is set and the model thinks, otherwise 16384. A SOP step's call asks for sop.max_output_tokens when that is higher.
ThinkingConfig	Optional: thinking_level to Gemini 3 and later, thinking_budget to Gemini 2.5 (see below).
Temperature	Optional via ai.gemini.temperature, sent to Gemini 2.5 and older only.
CachedContent	Intentionally disabled.

Thinking Configuration

ai.gemini.thinking_level supports off, minimal, low, medium, high, and the empty string for the API default. ai.gemini.thinking_budget is a token budget from 0 to 65536.

How much a model thinks is set in one of two ways, and a request carries only one of them (internal/geminimodel, resolveThinkingConfig):

Model	Sent	From thinking_level	From thinking_budget
Gemini 3 and later, and any Gemini name not listed here	thinking_level	as is; off is the least the model takes (it cannot stop thinking); minimal only to models that accept it (3 Flash, 3.1 Flash-Lite, 3.5 Flash, 3.6 Flash), otherwise low	0 → the least level, ≤1024 → low, ≤8192 → medium, more → high; the level wins when both are set
Gemini 2.5	thinking_budget	off → 0 (128 on 2.5 Pro, which cannot stop thinking), minimal/low → 1024, medium → 8192, high → 24576	as is (0 on 2.5 Pro is sent as 128); the budget wins when both are set
Gemini 1.x, 2.0, non-Gemini names	nothing	ignored, with one warning	ignored, with one warning

Google retired thinking_budget for Gemini 3: those models only remapped it to a level, and newer ones answer it with 400 INVALID_ARGUMENT. The API also rejects thinking_level on models before Gemini 3, and both settings together on any model.

Sampling Parameters

Gemini 3 and later are never sent temperature, top_p or top_k: since Gemini 3.6 Flash they are deprecated and set to their defaults whatever is sent, and newer models reject them. ai.gemini.temperature is sent to Gemini 2.5 and older only; for a newer model it is dropped with one warning. The background calls that ask for a low temperature (metadata extraction, the skill quality score) do the same (geminimodel.SamplingParams).

Provider gemini-legacy is a deprecated name for this provider: everything that names it is served by this SDK. The gateway already served ai.provider: gemini-legacy this way; goclaw chat and an agent model written gemini-legacy/<model> used LangChainGo's Google client, which sent temperature 0.5, top_p 0.95 and top_k 3 on every request and no thinking settings, until it was removed.

Multimodal Input

Web chat uploads are saved under workspace/uploads and passed through gateway.buildContentParts().

For image attachments, buildContentParts() embeds a marker:

__GCIMG__:<mime_type>:<base64_data>

gemini_stream.go parses this marker in parseContentParts() and converts it to:

&genai.Part{
    InlineData: &genai.Blob{
        MIMEType: mimeType,
        Data:     imgBytes,
    },
}

That keeps the public ProcessStream(ctx, content string, history) signature unchanged while allowing Gemini to analyze image attachments natively. Non-image attachments are referenced by workspace-relative path so tools such as file_read, pdf, spreadsheet, or voice_input can process them.

Gemini Built-In Tools

GoClaw supports Gemini Google Search through ai.gemini.google_search.

When enabled, Google Search is added as a separate genai.Tool entry because the Gemini API does not allow FunctionDeclarations and GoogleSearch in the same tool object.

tools := []*genai.Tool{{FunctionDeclarations: customDecls}}
if cfg.AI.Gemini.GoogleSearch {
    tools = append(tools, &genai.Tool{GoogleSearch: &genai.GoogleSearch{}})
}

Grounding citations are appended to the response by formatGroundingCitations().

Gemini Code Execution is not currently wired into the main agent loop. Use GoClaw's existing execution and analysis tools instead.

Tool Schema Conversion

The conversion pipeline is:

flowchart LR
    REG["registered tools"] --> GET["GetToolsFor(ctx, ragContext)"]
    GET --> LLMS["[]llms.Tool"]
    LLMS --> CONV["toGenaiDeclarations()"]
    CONV --> DECL["[]*genai.FunctionDeclaration"]
    DECL --> TOOL["genai.Tool{FunctionDeclarations}"]

convertToSchema() handles JSON-schema object, string, number, integer, boolean, and array types.

Context Caching

Gemini context caching remains disabled. The Gemini API does not allow system_instruction, tools, or tool config to be set on GenerateContent when CachedContent is used. Enabling caching would weaken per-query Tool RAG and Skill RAG, so GoClaw favors dynamic context assembly over cache token savings.

Maintenance Checklist

When changing the Gemini path, verify both agent loops where applicable:

• internal/agent/gemini_stream.go
• internal/agent/process.go

Also run:

go test ./internal/agent -count=1