AiGpu

Infrastructure·

Google Launches Tool Retrieval for Gemini API to Streamline Large‑Scale Agent Workloads

Google introduces Tool Retrieval in the Gemini Interactions API, letting agents load function schemas on demand to cut token usage and latency.

Illustration of Gemini API tool retrieval workflow

Google has rolled out a preview of Tool Retrieval inside the Gemini Interactions API, a change aimed at developers who run agents with dozens or hundreds of callable functions. By keeping the full tool catalogue on standby and pulling parameter schemas only when a task actually needs them, the feature reduces input‑token counts, lowers latency, and improves the model’s ability to pick the right tool.

How Tool Retrieval Works

  • Add a tool_search entry to the request payload.
  • Mark infrequently used functions with defer_loading: true so their schemas stay out of the initial context.
  • The model still sees each deferred tool’s name and description, enabling it to decide when a tool is required.
  • When the model calls a deferred tool, the API injects the full schema and executes the function within the same request cycle.

In practice, a single API call can now handle a refund request while keeping 50‑plus other functions dormant until they are explicitly invoked. This on‑demand loading pattern is compatible with Gemini 3.6 and later models.

Why it matters for GPU / AI infrastructure: Smaller prompt payloads mean less data movement across the GPU memory hierarchy, translating into lower compute cost per inference and higher throughput for multi‑tenant GPU clusters. For teams running large‑scale agent fleets on shared accelerator pools, the token savings compound quickly, freeing capacity for additional workloads.

  • aigpu
  • ai gpu
  • ai gpu cloud
  • aigpu dubai
  • gemini api
  • tool retrieval
  • function calling
  • llm agents

By AiGpu Editorial · Editorial rewrite based on public reporting (Google AI Blog)

← All articles