# Gemini Agents API Note: This API is in Beta. Endpoints are under `/v1beta/`. Agents are systems that leverage Gemini models, a set of tools, and reasoning capabilities to perform complex, multi-step tasks and achieve specific goals. Unlike a single model call, an agent can plan, execute a series of actions, interact with external systems, and synthesize information to fulfill a user's request. ## Agents ### CreateAgent `POST https://generativelanguage.googleapis.com/v1beta/agents` Creates a new Agent (Typed version for SDK). #### Request Body - **id** (`string`) The unique identifier for the agent. - **base_agent** (`string`) The base agent to extend. - **system_instruction** (`string`) System instruction for the agent. - **description** (`string`) Agent description for developers to quickly read and understand. - **tools** (`array (AgentTool)`) The tools available to the agent. **Possible Types:** (Discriminator: `type`) - **CodeExecution**: A tool that can be used by the model to execute code. - **type** (`object`) *(Required)* Value: `code_execution` - **GoogleSearch**: A tool that can be used by the model to search Google. - **type** (`object`) *(Required)* Value: `google_search` - **search_types** (`array (enum (string))`) The types of search grounding to enable. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. - **UrlContext**: A tool that can be used by the model to fetch URL context. - **type** (`object`) *(Required)* Value: `url_context` - **McpServer**: A MCPServer is a server that can be called by the model to perform actions. - **type** (`object`) *(Required)* Value: `mcp_server` - **name** (`string`) The name of the MCPServer. - **url** (`string`) The full URL for the MCPServer endpoint. Example: "https://api.example.com/mcp" - **headers** (`object`) Optional: Fields for authentication headers, timeouts, etc., if needed. - **allowed_tools** (`array (AllowedTools)`) The allowed tools. - **mode** (`enum (string)`) The mode of the tool choice. Possible values: - `auto`: Auto tool choice. - `any`: Any tool choice. - `none`: No tool choice. - `validated`: Validated tool choice. - **tools** (`array (string)`) The names of the allowed tools. - **base_environment** (`EnvironmentConfig or string`) The environment configuration for the agent. **Possible Types:** (Discriminator: `type`) - **EnvironmentConfig**: Configuration for a custom environment. - **type** (`object`) *(Required)* Value: `remote` - **sources** (`array (Source)`) - **type** (`enum (string)`) Possible values: - `gcs`: A GCS bucket. - `inline`: Inline content. - `repository`: A generic repository. The protocol prefix in the source URL identifies the provider (e.g., github://, gcs://). - `skill_registry`: A skill resource from the Skill Registry Service. Skill: projects/{project}/locations/{location}/skills/{skill} SkillRevision: projects/{project}/locations/{location}/skills/{skill}/revisions/{revision} Support mounting all skills under a project: projects/{project}/locations/{location}/skills. - **source** (`string`) The source of the environment. For GCS, this is the GCS path. For GitHub, this is the GitHub path. - **target** (`string`) Where the source should appear in the environment. - **content** (`string`) The inline content if `type` is `INLINE`. - **encoding** (`string`) Optional encoding for inline content (e.g. `base64`). - **environment_id** (`string`) Optional. The environment ID for the interaction. If specified, the request will update the existing environment instead of creating a new one. - **network** (`EnvironmentNetworkEgressAllowlist or enum (string)`) Network configuration for the environment. Possible values: - `disabled`: Turns all network off. - **string**: #### Response - **id** (`string`) The unique identifier for the agent. - **base_agent** (`string`) The base agent to extend. - **system_instruction** (`string`) System instruction for the agent. - **description** (`string`) Agent description for developers to quickly read and understand. - **tools** (`array (AgentTool)`) The tools available to the agent. **Possible Types:** (Discriminator: `type`) - **CodeExecution**: A tool that can be used by the model to execute code. - **type** (`object`) *(Required)* Value: `code_execution` - **GoogleSearch**: A tool that can be used by the model to search Google. - **type** (`object`) *(Required)* Value: `google_search` - **search_types** (`array (enum (string))`) The types of search grounding to enable. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. - **UrlContext**: A tool that can be used by the model to fetch URL context. - **type** (`object`) *(Required)* Value: `url_context` - **McpServer**: A MCPServer is a server that can be called by the model to perform actions. - **type** (`object`) *(Required)* Value: `mcp_server` - **name** (`string`) The name of the MCPServer. - **url** (`string`) The full URL for the MCPServer endpoint. Example: "https://api.example.com/mcp" - **headers** (`object`) Optional: Fields for authentication headers, timeouts, etc., if needed. - **allowed_tools** (`array (AllowedTools)`) The allowed tools. - **mode** (`enum (string)`) The mode of the tool choice. Possible values: - `auto`: Auto tool choice. - `any`: Any tool choice. - `none`: No tool choice. - `validated`: Validated tool choice. - **tools** (`array (string)`) The names of the allowed tools. - **base_environment** (`EnvironmentConfig or string`) The environment configuration for the agent. **Possible Types:** (Discriminator: `type`) - **EnvironmentConfig**: Configuration for a custom environment. - **type** (`object`) *(Required)* Value: `remote` - **sources** (`array (Source)`) - **type** (`enum (string)`) Possible values: - `gcs`: A GCS bucket. - `inline`: Inline content. - `repository`: A generic repository. The protocol prefix in the source URL identifies the provider (e.g., github://, gcs://). - `skill_registry`: A skill resource from the Skill Registry Service. Skill: projects/{project}/locations/{location}/skills/{skill} SkillRevision: projects/{project}/locations/{location}/skills/{skill}/revisions/{revision} Support mounting all skills under a project: projects/{project}/locations/{location}/skills. - **source** (`string`) The source of the environment. For GCS, this is the GCS path. For GitHub, this is the GitHub path. - **target** (`string`) Where the source should appear in the environment. - **content** (`string`) The inline content if `type` is `INLINE`. - **encoding** (`string`) Optional encoding for inline content (e.g. `base64`). - **environment_id** (`string`) Optional. The environment ID for the interaction. If specified, the request will update the existing environment instead of creating a new one. - **network** (`EnvironmentNetworkEgressAllowlist or enum (string)`) Network configuration for the environment. Possible values: - `disabled`: Turns all network off. - **string**: #### Examples **Create Agent** **REST** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/agents \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "id": "research-assistant-abc123", "base_agent": "antigravity-preview-05-2026", "description": "A helpful research assistant.", "system_instruction": "You are a helpful research assistant.", "base_environment": "remote", "tools": [{"type": "google_search"}] }' ``` **Python** ```python import uuid from google import genai client = genai.Client() agent = client.agents.create( id=f"research-assistant-{uuid.uuid4().hex[:8]}", base_agent="antigravity-preview-05-2026", description="A helpful research assistant.", system_instruction="You are a helpful research assistant.", base_environment="remote", tools=[{"type": "google_search"}], ) print(agent.id) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const agentId = `research-assistant-${crypto.randomUUID().slice(0, 8)}`; const agent = await ai.agents.create({ id: agentId, base_agent: 'antigravity-preview-05-2026', description: 'A helpful research assistant.', system_instruction: 'You are a helpful research assistant.', base_environment: 'remote', tools: [{ type: 'google_search' }], }); if (!agent.id) { throw new Error('Agent creation failed: ID is undefined'); } console.log(agent.id); ``` Response: ```json { "id": "ag_abc123", "display_name": "My Research Agent", "system_instruction": "You are a helpful research assistant.", "tools": [ { "type": "google_search" } ], "object": "agent", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z" } ``` **Agent with Sources** **REST** ```sh curl -X POST https://generativelanguage.googleapis.com/v1beta/agents \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "id": "data-analyst-abc123", "base_agent": "antigravity-preview-05-2026", "system_instruction": "You are a data analyst. Always include visualizations and export results as PDF.", "base_environment": { "type": "remote", "sources": [ { "type": "inline", "target": ".agents/AGENTS.md", "content": "Always use matplotlib for charts. Include a summary table in every report." }, { "type": "repository", "source": "https://github.com/my-org/analysis-templates", "target": "/workspace/templates" } ] } }' ``` **Python** ```python import uuid from google import genai client = genai.Client() agent = client.agents.create( id=f"data-analyst-{uuid.uuid4().hex[:8]}", base_agent="antigravity-preview-05-2026", system_instruction="You are a data analyst. Always include visualizations and export results as PDF.", base_environment={ "type": "remote", "sources": [ { "type": "inline", "target": ".agents/AGENTS.md", "content": "Always use matplotlib for charts. Include a summary table in every report.", }, { "type": "repository", "source": "https://github.com/my-org/analysis-templates", "target": "/workspace/templates", }, ], }, ) print(f"Created agent: {agent.id}") ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const agentId = `data-analyst-${crypto.randomUUID().slice(0, 8)}`; const agent = await ai.agents.create({ id: agentId, base_agent: 'antigravity-preview-05-2026', system_instruction: 'You are a data analyst. Always include visualizations and export results as PDF.', base_environment: { type: 'remote', sources: [ { type: 'inline', target: '.agents/AGENTS.md', content: 'Always use matplotlib for charts. Include a summary table in every report.', }, { type: 'repository', source: 'https://github.com/my-org/analysis-templates', target: '/workspace/templates', }, ], }, }); console.log(`Created agent: ${agent.id}`); ``` Response: ```json { "id": "data-analyst-abc123", "system_instruction": "You are a data analyst. Always include visualizations and export results as PDF.", "object": "agent", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z" } ``` **Agent Forked from Environment** **REST** ```sh # Step 1: Set up the environment interactively RESPONSE=$(curl -s -X POST https://generativelanguage.googleapis.com/v1beta/interactions \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d '{ "agent": "antigravity-preview-05-2026", "input": "Write a basic Hello World template to /workspace/template.py.", "environment": "remote" }') ENV_ID=$(echo $RESPONSE | python3 -c "import sys,json; print(json.load(sys.stdin)['environment_id'])") # Step 2: Fork that environment into a named agent curl -X POST https://generativelanguage.googleapis.com/v1beta/agents \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -H "Api-Revision: 2026-05-20" \ -d "{ \"id\": \"my-data-analyst\", \"base_agent\": \"antigravity-preview-05-2026\", \"system_instruction\": \"You are a data analyst. Use the template at /workspace/template.py for all reports.\", \"base_environment\": \"$ENV_ID\" }" ``` **Python** ```python import uuid from google import genai client = genai.Client() # Step 1: Set up the environment interactively. interaction = client.interactions.create( agent="antigravity-preview-05-2026", input="Write a basic Hello World template to /workspace/template.py.", environment="remote", ) # Step 2: Fork that environment into a named agent. agent_id = f"my-data-analyst-{uuid.uuid4().hex[:8]}" agent = client.agents.create( id=agent_id, base_agent="antigravity-preview-05-2026", system_instruction="You are a data analyst. Use the template at /workspace/template.py for all reports.", base_environment=interaction.environment_id, ) print(f"Forked agent: {agent.id}") ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); // Step 1: Set up the environment interactively. const interaction = await ai.interactions.create({ agent: 'antigravity-preview-05-2026', input: 'Write a basic Hello World template to /workspace/template.py.', environment: 'remote', }); // Step 2: Fork that environment into a named agent. const agentId = `my-data-analyst-${crypto.randomUUID().slice(0, 8)}`; const agent = await ai.agents.create({ id: agentId, base_agent: 'antigravity-preview-05-2026', system_instruction: 'You are a data analyst. Use the template at /workspace/template.py for all reports.', base_environment: interaction.environment_id, }); console.log(`Forked agent: ${agent.id}`); ``` Response: ```json { "id": "my-data-analyst", "system_instruction": "You are a data analyst. Use the template at /workspace/template.py for all reports.", "object": "agent", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z" } ``` --- ### ListAgents `GET https://generativelanguage.googleapis.com/v1beta/agents` Lists all Agents. #### Parameters - **page_size** (`integer`) - **page_token** (`string`) - **parent** (`string`) #### Response - **agents** (`array (Agent)`) - **next_page_token** (`string`) #### Examples **List Agents** **REST** ```sh curl -X GET https://generativelanguage.googleapis.com/v1beta/agents \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Api-Revision: 2026-05-20" ``` **Python** ```python from google import genai client = genai.Client() response = client.agents.list() for agent in response.agents or []: print(agent.id) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); const agents = await ai.agents.list(); for (const agent of (agents.agents ?? [])) { console.log(agent.id); } ``` Response: ```json { "object": "list", "data": [ { "id": "ag_abc123", "display_name": "My Research Agent", "system_instruction": "You are a helpful research assistant.", "object": "agent", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z" } ] } ``` --- ### GetAgent `GET https://generativelanguage.googleapis.com/v1beta/agents/{id}` Gets a specific Agent. #### Parameters - **id** (`string`) *(Required)* #### Response - **id** (`string`) The unique identifier for the agent. - **base_agent** (`string`) The base agent to extend. - **system_instruction** (`string`) System instruction for the agent. - **description** (`string`) Agent description for developers to quickly read and understand. - **tools** (`array (AgentTool)`) The tools available to the agent. **Possible Types:** (Discriminator: `type`) - **CodeExecution**: A tool that can be used by the model to execute code. - **type** (`object`) *(Required)* Value: `code_execution` - **GoogleSearch**: A tool that can be used by the model to search Google. - **type** (`object`) *(Required)* Value: `google_search` - **search_types** (`array (enum (string))`) The types of search grounding to enable. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. - **UrlContext**: A tool that can be used by the model to fetch URL context. - **type** (`object`) *(Required)* Value: `url_context` - **McpServer**: A MCPServer is a server that can be called by the model to perform actions. - **type** (`object`) *(Required)* Value: `mcp_server` - **name** (`string`) The name of the MCPServer. - **url** (`string`) The full URL for the MCPServer endpoint. Example: "https://api.example.com/mcp" - **headers** (`object`) Optional: Fields for authentication headers, timeouts, etc., if needed. - **allowed_tools** (`array (AllowedTools)`) The allowed tools. - **mode** (`enum (string)`) The mode of the tool choice. Possible values: - `auto`: Auto tool choice. - `any`: Any tool choice. - `none`: No tool choice. - `validated`: Validated tool choice. - **tools** (`array (string)`) The names of the allowed tools. - **base_environment** (`EnvironmentConfig or string`) The environment configuration for the agent. **Possible Types:** (Discriminator: `type`) - **EnvironmentConfig**: Configuration for a custom environment. - **type** (`object`) *(Required)* Value: `remote` - **sources** (`array (Source)`) - **type** (`enum (string)`) Possible values: - `gcs`: A GCS bucket. - `inline`: Inline content. - `repository`: A generic repository. The protocol prefix in the source URL identifies the provider (e.g., github://, gcs://). - `skill_registry`: A skill resource from the Skill Registry Service. Skill: projects/{project}/locations/{location}/skills/{skill} SkillRevision: projects/{project}/locations/{location}/skills/{skill}/revisions/{revision} Support mounting all skills under a project: projects/{project}/locations/{location}/skills. - **source** (`string`) The source of the environment. For GCS, this is the GCS path. For GitHub, this is the GitHub path. - **target** (`string`) Where the source should appear in the environment. - **content** (`string`) The inline content if `type` is `INLINE`. - **encoding** (`string`) Optional encoding for inline content (e.g. `base64`). - **environment_id** (`string`) Optional. The environment ID for the interaction. If specified, the request will update the existing environment instead of creating a new one. - **network** (`EnvironmentNetworkEgressAllowlist or enum (string)`) Network configuration for the environment. Possible values: - `disabled`: Turns all network off. - **string**: #### Examples **Get Agent** **REST** ```sh curl -X GET https://generativelanguage.googleapis.com/v1beta/agents/ag_abc123 \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Api-Revision: 2026-05-20" ``` **Python** ```python import uuid from google import genai client = genai.Client() # Create an agent so we have a valid ID to retrieve. agent_id = f"test-agent-{uuid.uuid4().hex[:8]}" client.agents.create( id=agent_id, base_agent="waverunner", description="A test agent.", base_environment="remote", ) agent = client.agents.get(agent_id) print(agent.id) ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); // Create an agent so we have a valid ID to retrieve. const agentId = `test-agent-${crypto.randomUUID().slice(0, 8)}`; await ai.agents.create({ id: agentId, base_agent: 'waverunner', description: 'A test agent.', base_environment: 'remote', }); const agent = await ai.agents.get(agentId); if (!agent.id) { throw new Error('Agent retrieval failed: ID is undefined'); } console.log(agent.id); ``` Response: ```json { "id": "ag_abc123", "display_name": "My Research Agent", "system_instruction": "You are a helpful research assistant.", "tools": [ { "type": "google_search" } ], "object": "agent", "created": "2025-11-26T12:25:15Z", "updated": "2025-11-26T12:25:15Z" } ``` --- ### DeleteAgent `DELETE https://generativelanguage.googleapis.com/v1beta/agents/{id}` Deletes an Agent. #### Parameters - **id** (`string`) *(Required)* #### Response #### Examples **Delete Agent** **REST** ```sh curl -X DELETE https://generativelanguage.googleapis.com/v1beta/agents/ag_abc123 \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Api-Revision: 2026-05-20" ``` **Python** ```python import uuid from google import genai client = genai.Client() # Create an agent so we have a valid ID to delete. agent_id = f"delete-test-{uuid.uuid4().hex[:8]}" client.agents.create( id=agent_id, base_agent="waverunner", description="Temporary agent for deletion.", base_environment="remote", ) client.agents.delete(agent_id) print("Agent deleted successfully.") ``` **JavaScript** ```javascript import {GoogleGenAI} from '@google/genai'; const ai = new GoogleGenAI({}); // Create an agent so we have a valid ID to delete. const agentId = `delete-test-${crypto.randomUUID().slice(0, 8)}`; await ai.agents.create({ id: agentId, base_agent: 'waverunner', description: 'Temporary agent for deletion.', base_environment: 'remote', }); await ai.agents.delete(agentId); console.log('Agent deleted successfully.'); ``` --- ## Resources ### Agent { #Resource:Agent } An agent definition for the CreateAgent API. This message is the target for annotation-parser-based JSON parsing. New format: { "id": "customer-sentinel", "base_agent": "", "system_instruction": "...", "base_environment": { "type": "remote", "sources": [...] }, "tools": [ {"type": "code_execution"} ] } **Properties:** - **id** (`string`) The unique identifier for the agent. - **base_agent** (`string`) The base agent to extend. - **system_instruction** (`string`) System instruction for the agent. - **description** (`string`) Agent description for developers to quickly read and understand. - **tools** (`array (AgentTool)`) The tools available to the agent. **Possible Types:** (Discriminator: `type`) - **CodeExecution**: A tool that can be used by the model to execute code. - **type** (`object`) *(Required)* Value: `code_execution` - **GoogleSearch**: A tool that can be used by the model to search Google. - **type** (`object`) *(Required)* Value: `google_search` - **search_types** (`array (enum (string))`) The types of search grounding to enable. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. - **UrlContext**: A tool that can be used by the model to fetch URL context. - **type** (`object`) *(Required)* Value: `url_context` - **McpServer**: A MCPServer is a server that can be called by the model to perform actions. - **type** (`object`) *(Required)* Value: `mcp_server` - **name** (`string`) The name of the MCPServer. - **url** (`string`) The full URL for the MCPServer endpoint. Example: "https://api.example.com/mcp" - **headers** (`object`) Optional: Fields for authentication headers, timeouts, etc., if needed. - **allowed_tools** (`array (AllowedTools)`) The allowed tools. - **mode** (`enum (string)`) The mode of the tool choice. Possible values: - `auto`: Auto tool choice. - `any`: Any tool choice. - `none`: No tool choice. - `validated`: Validated tool choice. - **tools** (`array (string)`) The names of the allowed tools. - **base_environment** (`EnvironmentConfig or string`) The environment configuration for the agent. **Possible Types:** (Discriminator: `type`) - **EnvironmentConfig**: Configuration for a custom environment. - **type** (`object`) *(Required)* Value: `remote` - **sources** (`array (Source)`) - **type** (`enum (string)`) Possible values: - `gcs`: A GCS bucket. - `inline`: Inline content. - `repository`: A generic repository. The protocol prefix in the source URL identifies the provider (e.g., github://, gcs://). - `skill_registry`: A skill resource from the Skill Registry Service. Skill: projects/{project}/locations/{location}/skills/{skill} SkillRevision: projects/{project}/locations/{location}/skills/{skill}/revisions/{revision} Support mounting all skills under a project: projects/{project}/locations/{location}/skills. - **source** (`string`) The source of the environment. For GCS, this is the GCS path. For GitHub, this is the GitHub path. - **target** (`string`) Where the source should appear in the environment. - **content** (`string`) The inline content if `type` is `INLINE`. - **encoding** (`string`) Optional encoding for inline content (e.g. `base64`). - **environment_id** (`string`) Optional. The environment ID for the interaction. If specified, the request will update the existing environment instead of creating a new one. - **network** (`EnvironmentNetworkEgressAllowlist or enum (string)`) Network configuration for the environment. Possible values: - `disabled`: Turns all network off. - **string**: **JSON Representation:** ```json { "id": "string", "base_agent": "string", "system_instruction": "string", "description": "string", "tools": [ { "type": {} } ], "base_environment": { "type": {}, "sources": [ { "type": "gcs", "source": "string", "target": "string", "content": "string", "encoding": "string" } ], "environment_id": "string", "network": { "allowlist": [ { "domain": "github.com", "transform": [ { "Authorization": "Bearer your-token" } ] }, { "domain": "*.googleapis.com" } ] } } } ``` ## Data Models ### InteractionSseEvent { #Resource:InteractionSseEvent } **Polymorphic Types:** (Discriminator: `event_type`)- **InteractionCreatedEvent** - - **event_type** (`object`) *(Required)* Value: `interaction.created` - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **total_usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. - **interaction** (`InteractionSseEventInteraction`) *(Required)* Partial interaction resource emitted when the stream is created. - **id** (`string`) *(Required)* Required. Output only. A unique identifier for the interaction completion. - **object** (`string`) Output only. The resource type. - **model** (`string`) The model that will complete your prompt. - **agent** (`string`) The agent to interact with. - **status** (`enum (string)`) *(Required)* Required. Output only. The status of the interaction. Possible values: - `in_progress`: The interaction is in progress. - `requires_action`: The interaction requires action/input from the user. - `completed`: The interaction is completed. - `failed`: The interaction failed. - `cancelled`: The interaction was cancelled. - `incomplete`: The interaction is completed, but contains incomplete results (e.g. hitting max_tokens). - **created** (`string`) Output only. The time at which the response was created in ISO 8601 format. - **updated** (`string`) Output only. The time at which the response was last updated in ISO 8601 format. - **service_tier** (`ServiceTier`) The service tier for the interaction. Possible values: - `flex`: Flex service tier. - `standard`: Standard service tier. - `priority`: Priority service tier. - **usage** (`Usage`) Output only. Statistics on the interaction request's token usage. - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. - **steps** (`array (Step)`) Output only. The steps that make up the interaction, if included in this event. **Possible Types:** (Discriminator: `type`) - **UserInputStep**: Input provided by the user. - **content** (`array (Content)`) **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **AudioContent**: An audio content block. - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) The audio content. - **uri** (`string`) The URI of the audio. - **mime_type** (`enum (string)`) The mime type of the audio. Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **channels** (`integer`) The number of audio channels. - **sample_rate** (`integer`) The sample rate of the audio. - **DocumentContent**: A document content block. - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) The document content. - **uri** (`string`) The URI of the document. - **mime_type** (`enum (string)`) The mime type of the document. Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **VideoContent**: A video content block. - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) The video content. - **uri** (`string`) The URI of the video. - **mime_type** (`enum (string)`) The mime type of the video. Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **type** (`object`) *(Required)* Value: `user_input` - **ModelOutputStep**: Output generated by the model. - **type** (`object`) *(Required)* Value: `model_output` - **content** (`array (Content)`) **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **AudioContent**: An audio content block. - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) The audio content. - **uri** (`string`) The URI of the audio. - **mime_type** (`enum (string)`) The mime type of the audio. Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **channels** (`integer`) The number of audio channels. - **sample_rate** (`integer`) The sample rate of the audio. - **DocumentContent**: A document content block. - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) The document content. - **uri** (`string`) The URI of the document. - **mime_type** (`enum (string)`) The mime type of the document. Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **VideoContent**: A video content block. - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) The video content. - **uri** (`string`) The URI of the video. - **mime_type** (`enum (string)`) The mime type of the video. Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **error** (`Status`) The error result of the operation in case of failure or cancellation. - **code** (`integer`) The status code, which should be an enum value of google.rpc.Code. - **message** (`string`) A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. - **details** (`array (object)`) A list of messages that carry the error details. There is a common set of message types for APIs to use. - **ThoughtStep**: A thought step. - **type** (`object`) *(Required)* Value: `thought` - **signature** (`string`) A signature hash for backend validation. - **summary** (`array (ThoughtSummaryContent)`) A summary of the thought. **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **FunctionCallStep**: A function tool call step. - **type** (`object`) *(Required)* Value: `function_call` - **name** (`string`) *(Required)* Required. The name of the tool to call. - **arguments** (`object`) *(Required)* Required. The arguments to pass to the function. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **CodeExecutionCallStep**: Code execution call step. - **type** (`object`) *(Required)* Value: `code_execution_call` - **arguments** (`CodeExecutionCallStepArguments`) *(Required)* Required. The arguments to pass to the code execution. - **language** (`enum (string)`) Programming language of the `code`. Possible values: - `python`: Python >= 3.10, with numpy and simpy available. - **code** (`string`) The code to be executed. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **UrlContextCallStep**: URL context call step. - **type** (`object`) *(Required)* Value: `url_context_call` - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **arguments** (`UrlContextCallArguments`) *(Required)* The arguments to pass to the URL context. - **urls** (`array (string)`) The URLs to fetch. - **McpServerToolCallStep**: MCPServer tool call step. - **type** (`object`) *(Required)* Value: `mcp_server_tool_call` - **name** (`string`) *(Required)* Required. The name of the tool which was called. - **server_name** (`string`) *(Required)* Required. The name of the used MCP server. - **arguments** (`object`) *(Required)* Required. The JSON object of arguments for the function. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **GoogleSearchCallStep**: Google Search call step. - **type** (`object`) *(Required)* Value: `google_search_call` - **arguments** (`GoogleSearchCallStepArguments`) *(Required)* Required. The arguments to pass to Google Search. - **queries** (`array (string)`) Web search queries for the following-up web search. - **search_type** (`enum (string)`) The type of search grounding enabled. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **FileSearchCallStep**: File Search call step. - **type** (`object`) *(Required)* Value: `file_search_call` - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsCallStep**: Google Maps call step. - **type** (`object`) *(Required)* Value: `google_maps_call` - **arguments** (`GoogleMapsCallStepArguments`) The arguments to pass to the Google Maps tool. - **queries** (`array (string)`) The queries to be executed. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **FunctionResultStep**: Result of a function tool call. - **type** (`object`) *(Required)* Value: `function_result` - **name** (`string`) The name of the tool that was called. - **is_error** (`boolean`) Whether the tool call resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (ImageContent or TextContent) or object or string`) *(Required)* The result of the tool call. - **CodeExecutionResultStep**: Code execution result step. - **type** (`object`) *(Required)* Value: `code_execution_result` - **result** (`string`) *(Required)* Required. The output of the code execution. - **is_error** (`boolean`) Whether the code execution resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **UrlContextResultStep**: URL context result step. - **type** (`object`) *(Required)* Value: `url_context_result` - **result** (`array (UrlContextResult)`) *(Required)* Required. The results of the URL context. - **url** (`string`) The URL that was fetched. - **status** (`enum (string)`) The status of the URL retrieval. Possible values: - `success`: Url retrieval is successful. - `error`: Url retrieval is failed due to error. - `paywall`: Url retrieval is failed because the content is behind paywall. - `unsafe`: Url retrieval is failed because the content is unsafe. - **is_error** (`boolean`) Whether the URL context resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **GoogleSearchResultStep**: Google Search result step. - **type** (`object`) *(Required)* Value: `google_search_result` - **result** (`array (GoogleSearchResultItem)`) *(Required)* Required. The results of the Google Search. - **search_suggestions** (`string`) Web content snippet that can be embedded in a web page or an app webview. - **is_error** (`boolean`) Whether the Google Search resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **McpServerToolResultStep**: MCPServer tool result step. - **type** (`object`) *(Required)* Value: `mcp_server_tool_result` - **name** (`string`) Name of the tool which is called for this specific tool call. - **server_name** (`string`) The name of the used MCP server. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (ImageContent or TextContent) or object or string`) *(Required)* The output from the MCP server call. Can be simple text or rich content. - **FileSearchResultStep**: File Search result step. - **type** (`object`) *(Required)* Value: `file_search_result` - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsResultStep**: Google Maps result step. - **type** (`object`) *(Required)* Value: `google_maps_result` - **result** (`array (GoogleMapsResultItem)`) *(Required)* - **places** (`array (GoogleMapsResultPlaces)`) - **place_id** (`string`) - **name** (`string`) - **url** (`string`) - **review_snippets** (`array (ReviewSnippet)`) - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **widget_context_token** (`string`) - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. **Examples** **Interaction Created** ```json { "event_type": "interaction.created", "interaction": { "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3.5-flash", "status": "in_progress", "created": "2025-12-04T15:01:45Z", "updated": "2025-12-04T15:01:45Z" }, "event_id": "evt_123" } ``` **Interaction Created** ```json { "event_type": "interaction.created", "interaction": { "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3-flash-preview", "object": "interaction", "status": "in_progress" }, "event_id": "evt_123" } ``` - **InteractionCompletedEvent** - - **event_type** (`object`) *(Required)* Value: `interaction.completed` - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **total_usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. - **interaction** (`InteractionSseEventInteraction`) *(Required)* Partial completed interaction resource emitted at the end of the stream. - **id** (`string`) *(Required)* Required. Output only. A unique identifier for the interaction completion. - **object** (`string`) Output only. The resource type. - **model** (`string`) The model that will complete your prompt. - **agent** (`string`) The agent to interact with. - **status** (`enum (string)`) *(Required)* Required. Output only. The status of the interaction. Possible values: - `in_progress`: The interaction is in progress. - `requires_action`: The interaction requires action/input from the user. - `completed`: The interaction is completed. - `failed`: The interaction failed. - `cancelled`: The interaction was cancelled. - `incomplete`: The interaction is completed, but contains incomplete results (e.g. hitting max_tokens). - **created** (`string`) Output only. The time at which the response was created in ISO 8601 format. - **updated** (`string`) Output only. The time at which the response was last updated in ISO 8601 format. - **service_tier** (`ServiceTier`) The service tier for the interaction. Possible values: - `flex`: Flex service tier. - `standard`: Standard service tier. - `priority`: Priority service tier. - **usage** (`Usage`) Output only. Statistics on the interaction request's token usage. - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. - **steps** (`array (Step)`) Output only. The steps that make up the interaction, if included in this event. **Possible Types:** (Discriminator: `type`) - **UserInputStep**: Input provided by the user. - **content** (`array (Content)`) **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **AudioContent**: An audio content block. - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) The audio content. - **uri** (`string`) The URI of the audio. - **mime_type** (`enum (string)`) The mime type of the audio. Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **channels** (`integer`) The number of audio channels. - **sample_rate** (`integer`) The sample rate of the audio. - **DocumentContent**: A document content block. - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) The document content. - **uri** (`string`) The URI of the document. - **mime_type** (`enum (string)`) The mime type of the document. Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **VideoContent**: A video content block. - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) The video content. - **uri** (`string`) The URI of the video. - **mime_type** (`enum (string)`) The mime type of the video. Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **type** (`object`) *(Required)* Value: `user_input` - **ModelOutputStep**: Output generated by the model. - **type** (`object`) *(Required)* Value: `model_output` - **content** (`array (Content)`) **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **AudioContent**: An audio content block. - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) The audio content. - **uri** (`string`) The URI of the audio. - **mime_type** (`enum (string)`) The mime type of the audio. Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **channels** (`integer`) The number of audio channels. - **sample_rate** (`integer`) The sample rate of the audio. - **DocumentContent**: A document content block. - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) The document content. - **uri** (`string`) The URI of the document. - **mime_type** (`enum (string)`) The mime type of the document. Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **VideoContent**: A video content block. - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) The video content. - **uri** (`string`) The URI of the video. - **mime_type** (`enum (string)`) The mime type of the video. Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **error** (`Status`) The error result of the operation in case of failure or cancellation. - **code** (`integer`) The status code, which should be an enum value of google.rpc.Code. - **message** (`string`) A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. - **details** (`array (object)`) A list of messages that carry the error details. There is a common set of message types for APIs to use. - **ThoughtStep**: A thought step. - **type** (`object`) *(Required)* Value: `thought` - **signature** (`string`) A signature hash for backend validation. - **summary** (`array (ThoughtSummaryContent)`) A summary of the thought. **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **FunctionCallStep**: A function tool call step. - **type** (`object`) *(Required)* Value: `function_call` - **name** (`string`) *(Required)* Required. The name of the tool to call. - **arguments** (`object`) *(Required)* Required. The arguments to pass to the function. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **CodeExecutionCallStep**: Code execution call step. - **type** (`object`) *(Required)* Value: `code_execution_call` - **arguments** (`CodeExecutionCallStepArguments`) *(Required)* Required. The arguments to pass to the code execution. - **language** (`enum (string)`) Programming language of the `code`. Possible values: - `python`: Python >= 3.10, with numpy and simpy available. - **code** (`string`) The code to be executed. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **UrlContextCallStep**: URL context call step. - **type** (`object`) *(Required)* Value: `url_context_call` - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **arguments** (`UrlContextCallArguments`) *(Required)* The arguments to pass to the URL context. - **urls** (`array (string)`) The URLs to fetch. - **McpServerToolCallStep**: MCPServer tool call step. - **type** (`object`) *(Required)* Value: `mcp_server_tool_call` - **name** (`string`) *(Required)* Required. The name of the tool which was called. - **server_name** (`string`) *(Required)* Required. The name of the used MCP server. - **arguments** (`object`) *(Required)* Required. The JSON object of arguments for the function. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **GoogleSearchCallStep**: Google Search call step. - **type** (`object`) *(Required)* Value: `google_search_call` - **arguments** (`GoogleSearchCallStepArguments`) *(Required)* Required. The arguments to pass to Google Search. - **queries** (`array (string)`) Web search queries for the following-up web search. - **search_type** (`enum (string)`) The type of search grounding enabled. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **FileSearchCallStep**: File Search call step. - **type** (`object`) *(Required)* Value: `file_search_call` - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsCallStep**: Google Maps call step. - **type** (`object`) *(Required)* Value: `google_maps_call` - **arguments** (`GoogleMapsCallStepArguments`) The arguments to pass to the Google Maps tool. - **queries** (`array (string)`) The queries to be executed. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **FunctionResultStep**: Result of a function tool call. - **type** (`object`) *(Required)* Value: `function_result` - **name** (`string`) The name of the tool that was called. - **is_error** (`boolean`) Whether the tool call resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (ImageContent or TextContent) or object or string`) *(Required)* The result of the tool call. - **CodeExecutionResultStep**: Code execution result step. - **type** (`object`) *(Required)* Value: `code_execution_result` - **result** (`string`) *(Required)* Required. The output of the code execution. - **is_error** (`boolean`) Whether the code execution resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **UrlContextResultStep**: URL context result step. - **type** (`object`) *(Required)* Value: `url_context_result` - **result** (`array (UrlContextResult)`) *(Required)* Required. The results of the URL context. - **url** (`string`) The URL that was fetched. - **status** (`enum (string)`) The status of the URL retrieval. Possible values: - `success`: Url retrieval is successful. - `error`: Url retrieval is failed due to error. - `paywall`: Url retrieval is failed because the content is behind paywall. - `unsafe`: Url retrieval is failed because the content is unsafe. - **is_error** (`boolean`) Whether the URL context resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **GoogleSearchResultStep**: Google Search result step. - **type** (`object`) *(Required)* Value: `google_search_result` - **result** (`array (GoogleSearchResultItem)`) *(Required)* Required. The results of the Google Search. - **search_suggestions** (`string`) Web content snippet that can be embedded in a web page or an app webview. - **is_error** (`boolean`) Whether the Google Search resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **McpServerToolResultStep**: MCPServer tool result step. - **type** (`object`) *(Required)* Value: `mcp_server_tool_result` - **name** (`string`) Name of the tool which is called for this specific tool call. - **server_name** (`string`) The name of the used MCP server. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (ImageContent or TextContent) or object or string`) *(Required)* The output from the MCP server call. Can be simple text or rich content. - **FileSearchResultStep**: File Search result step. - **type** (`object`) *(Required)* Value: `file_search_result` - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsResultStep**: Google Maps result step. - **type** (`object`) *(Required)* Value: `google_maps_result` - **result** (`array (GoogleMapsResultItem)`) *(Required)* - **places** (`array (GoogleMapsResultPlaces)`) - **place_id** (`string`) - **name** (`string`) - **url** (`string`) - **review_snippets** (`array (ReviewSnippet)`) - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **widget_context_token** (`string`) - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. **Examples** **Interaction Completed** ```json { "event_type": "interaction.completed", "interaction": { "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3.5-flash", "status": "completed", "created": "2025-12-04T15:01:45Z", "updated": "2025-12-04T15:01:45Z" }, "event_id": "evt_123" } ``` **Interaction Completed** ```json { "event_type": "interaction.completed", "interaction": { "id": "v1_ChdXS0l4YWZXTk9xbk0xZThQczhEcmlROBIXV0tJeGFmV05PcW5NMWU4UHM4RHJpUTg", "model": "gemini-3-flash-preview", "object": "interaction", "status": "completed", "created": "2025-12-04T15:01:45Z", "updated": "2025-12-04T15:01:45Z" }, "event_id": "evt_123" } ``` - **InteractionStatusUpdate** - - **event_type** (`object`) *(Required)* Value: `interaction.status_update` - **interaction_id** (`string`) *(Required)* - **status** (`enum (string)`) *(Required)* Possible values: - `in_progress`: The interaction is in progress. - `requires_action`: The interaction requires action/input from the user. - `completed`: The interaction is completed. - `failed`: The interaction failed. - `cancelled`: The interaction was cancelled. - `incomplete`: The interaction is completed, but contains incomplete results (e.g. hitting max_tokens). - `budget_exceeded`: The interaction was halted because the token budget was exceeded. - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **total_usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Interaction Status Update** ```json { "event_type": "interaction.status_update", "interaction_id": "v1_ChdTMjQ0YWJ5TUF1TzcxZThQdjRpcnFRcxIXUzI0NGFieU1BdU83MWU4UHY0aXJxUXM", "status": "in_progress" } ``` - **ErrorEvent** - - **event_type** (`object`) *(Required)* Value: `error` - **error** (`Error`) - **code** (`string`) A URI that identifies the error type. - **message** (`string`) A human-readable error message. - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **total_usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Error Event** ```json { "event_type": "error", "error": { "message": "Failed to get completed interaction: Result not found.", "code": "not_found" } } ``` - **StepStart** - - **event_type** (`object`) *(Required)* Value: `step.start` - **index** (`integer`) *(Required)* - **step** (`Step`) *(Required)* **Possible Types:** (Discriminator: `type`) - **UserInputStep**: Input provided by the user. - **content** (`array (Content)`) **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **AudioContent**: An audio content block. - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) The audio content. - **uri** (`string`) The URI of the audio. - **mime_type** (`enum (string)`) The mime type of the audio. Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **channels** (`integer`) The number of audio channels. - **sample_rate** (`integer`) The sample rate of the audio. - **DocumentContent**: A document content block. - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) The document content. - **uri** (`string`) The URI of the document. - **mime_type** (`enum (string)`) The mime type of the document. Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **VideoContent**: A video content block. - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) The video content. - **uri** (`string`) The URI of the video. - **mime_type** (`enum (string)`) The mime type of the video. Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **type** (`object`) *(Required)* Value: `user_input` - **ModelOutputStep**: Output generated by the model. - **type** (`object`) *(Required)* Value: `model_output` - **content** (`array (Content)`) **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **AudioContent**: An audio content block. - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) The audio content. - **uri** (`string`) The URI of the audio. - **mime_type** (`enum (string)`) The mime type of the audio. Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **channels** (`integer`) The number of audio channels. - **sample_rate** (`integer`) The sample rate of the audio. - **DocumentContent**: A document content block. - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) The document content. - **uri** (`string`) The URI of the document. - **mime_type** (`enum (string)`) The mime type of the document. Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **VideoContent**: A video content block. - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) The video content. - **uri** (`string`) The URI of the video. - **mime_type** (`enum (string)`) The mime type of the video. Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **error** (`Status`) The error result of the operation in case of failure or cancellation. - **code** (`integer`) The status code, which should be an enum value of google.rpc.Code. - **message** (`string`) A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. - **details** (`array (object)`) A list of messages that carry the error details. There is a common set of message types for APIs to use. - **ThoughtStep**: A thought step. - **type** (`object`) *(Required)* Value: `thought` - **signature** (`string`) A signature hash for backend validation. - **summary** (`array (ThoughtSummaryContent)`) A summary of the thought. **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **FunctionCallStep**: A function tool call step. - **type** (`object`) *(Required)* Value: `function_call` - **name** (`string`) *(Required)* Required. The name of the tool to call. - **arguments** (`object`) *(Required)* Required. The arguments to pass to the function. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **CodeExecutionCallStep**: Code execution call step. - **type** (`object`) *(Required)* Value: `code_execution_call` - **arguments** (`CodeExecutionCallStepArguments`) *(Required)* Required. The arguments to pass to the code execution. - **language** (`enum (string)`) Programming language of the `code`. Possible values: - `python`: Python >= 3.10, with numpy and simpy available. - **code** (`string`) The code to be executed. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **UrlContextCallStep**: URL context call step. - **type** (`object`) *(Required)* Value: `url_context_call` - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **arguments** (`UrlContextCallArguments`) *(Required)* The arguments to pass to the URL context. - **urls** (`array (string)`) The URLs to fetch. - **McpServerToolCallStep**: MCPServer tool call step. - **type** (`object`) *(Required)* Value: `mcp_server_tool_call` - **name** (`string`) *(Required)* Required. The name of the tool which was called. - **server_name** (`string`) *(Required)* Required. The name of the used MCP server. - **arguments** (`object`) *(Required)* Required. The JSON object of arguments for the function. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **GoogleSearchCallStep**: Google Search call step. - **type** (`object`) *(Required)* Value: `google_search_call` - **arguments** (`GoogleSearchCallStepArguments`) *(Required)* Required. The arguments to pass to Google Search. - **queries** (`array (string)`) Web search queries for the following-up web search. - **search_type** (`enum (string)`) The type of search grounding enabled. Possible values: - `web_search`: Setting this field enables web search. Only text results are returned. - `image_search`: Setting this field enables image search. Image bytes are returned. - `enterprise_web_search`: Setting this field enables enterprise web search. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **FileSearchCallStep**: File Search call step. - **type** (`object`) *(Required)* Value: `file_search_call` - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsCallStep**: Google Maps call step. - **type** (`object`) *(Required)* Value: `google_maps_call` - **arguments** (`GoogleMapsCallStepArguments`) The arguments to pass to the Google Maps tool. - **queries** (`array (string)`) The queries to be executed. - **id** (`string`) *(Required)* Required. A unique ID for this specific tool call. - **signature** (`string`) A signature hash for backend validation. - **FunctionResultStep**: Result of a function tool call. - **type** (`object`) *(Required)* Value: `function_result` - **name** (`string`) The name of the tool that was called. - **is_error** (`boolean`) Whether the tool call resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (ImageContent or TextContent) or object or string`) *(Required)* The result of the tool call. - **CodeExecutionResultStep**: Code execution result step. - **type** (`object`) *(Required)* Value: `code_execution_result` - **result** (`string`) *(Required)* Required. The output of the code execution. - **is_error** (`boolean`) Whether the code execution resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **UrlContextResultStep**: URL context result step. - **type** (`object`) *(Required)* Value: `url_context_result` - **result** (`array (UrlContextResult)`) *(Required)* Required. The results of the URL context. - **url** (`string`) The URL that was fetched. - **status** (`enum (string)`) The status of the URL retrieval. Possible values: - `success`: Url retrieval is successful. - `error`: Url retrieval is failed due to error. - `paywall`: Url retrieval is failed because the content is behind paywall. - `unsafe`: Url retrieval is failed because the content is unsafe. - **is_error** (`boolean`) Whether the URL context resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **GoogleSearchResultStep**: Google Search result step. - **type** (`object`) *(Required)* Value: `google_search_result` - **result** (`array (GoogleSearchResultItem)`) *(Required)* Required. The results of the Google Search. - **search_suggestions** (`string`) Web content snippet that can be embedded in a web page or an app webview. - **is_error** (`boolean`) Whether the Google Search resulted in an error. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **McpServerToolResultStep**: MCPServer tool result step. - **type** (`object`) *(Required)* Value: `mcp_server_tool_result` - **name** (`string`) Name of the tool which is called for this specific tool call. - **server_name** (`string`) The name of the used MCP server. - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (ImageContent or TextContent) or object or string`) *(Required)* The output from the MCP server call. Can be simple text or rich content. - **FileSearchResultStep**: File Search result step. - **type** (`object`) *(Required)* Value: `file_search_result` - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsResultStep**: Google Maps result step. - **type** (`object`) *(Required)* Value: `google_maps_result` - **result** (`array (GoogleMapsResultItem)`) *(Required)* - **places** (`array (GoogleMapsResultPlaces)`) - **place_id** (`string`) - **name** (`string`) - **url** (`string`) - **review_snippets** (`array (ReviewSnippet)`) - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **widget_context_token** (`string`) - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **signature** (`string`) A signature hash for backend validation. - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **total_usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Step Start** ```json { "event_type": "step.start", "index": 0, "step": { "type": "model_output" } } ``` - **StepDelta** - - **event_type** (`object`) *(Required)* Value: `step.delta` - **index** (`integer`) *(Required)* - **delta** (`StepDeltaData`) *(Required)* **Possible Types:** (Discriminator: `type`) - **TextDelta**: - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* - **ImageDelta**: - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) - **uri** (`string`) - **mime_type** (`enum (string)`) Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **AudioDelta**: - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) - **uri** (`string`) - **mime_type** (`enum (string)`) Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **sample_rate** (`integer`) The sample rate of the audio. - **channels** (`integer`) The number of audio channels. - **DocumentDelta**: - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) - **uri** (`string`) - **mime_type** (`enum (string)`) Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **VideoDelta**: - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) - **uri** (`string`) - **mime_type** (`enum (string)`) Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **ThoughtSummaryDelta**: - **type** (`object`) *(Required)* Value: `thought_summary` - **content** (`Content`) A new summary item to be added to the thought. **Possible Types:** (Discriminator: `type`) - **TextContent**: A text content block. - **type** (`object`) *(Required)* Value: `text` - **text** (`string`) *(Required)* Required. The text content. - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ImageContent**: An image content block. - **type** (`object`) *(Required)* Value: `image` - **data** (`string`) The image content. - **uri** (`string`) The URI of the image. - **mime_type** (`enum (string)`) The mime type of the image. Possible values: - `image/png`: PNG image format - `image/jpeg`: JPEG image format - `image/webp`: WebP image format - `image/heic`: HEIC image format - `image/heif`: HEIF image format - `image/gif`: GIF image format - `image/bmp`: BMP image format - `image/tiff`: TIFF image format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **AudioContent**: An audio content block. - **type** (`object`) *(Required)* Value: `audio` - **data** (`string`) The audio content. - **uri** (`string`) The URI of the audio. - **mime_type** (`enum (string)`) The mime type of the audio. Possible values: - `audio/wav`: WAV audio format - `audio/mp3`: MP3 audio format - `audio/aiff`: AIFF audio format - `audio/aac`: AAC audio format - `audio/ogg`: OGG audio format - `audio/flac`: FLAC audio format - `audio/mpeg`: MPEG audio format - `audio/m4a`: M4A audio format - `audio/l16`: L16 audio format - `audio/opus`: OPUS audio format - `audio/alaw`: ALAW audio format - `audio/mulaw`: MULAW audio format - **channels** (`integer`) The number of audio channels. - **sample_rate** (`integer`) The sample rate of the audio. - **DocumentContent**: A document content block. - **type** (`object`) *(Required)* Value: `document` - **data** (`string`) The document content. - **uri** (`string`) The URI of the document. - **mime_type** (`enum (string)`) The mime type of the document. Possible values: - `application/pdf`: PDF document format - `text/csv`: CSV document format - **VideoContent**: A video content block. - **type** (`object`) *(Required)* Value: `video` - **data** (`string`) The video content. - **uri** (`string`) The URI of the video. - **mime_type** (`enum (string)`) The mime type of the video. Possible values: - `video/mp4`: MP4 video format - `video/mpeg`: MPEG video format - `video/mpg`: MPG video format - `video/mov`: MOV video format - `video/avi`: AVI video format - `video/x-flv`: FLV video format - `video/webm`: WebM video format - `video/wmv`: WMV video format - `video/3gpp`: 3GPP video format - **resolution** (`MediaResolution`) The resolution of the media. Possible values: - `low`: Low resolution. - `medium`: Medium resolution. - `high`: High resolution. - `ultra_high`: Ultra high resolution. - **ThoughtSignatureDelta**: - **type** (`object`) *(Required)* Value: `thought_signature` - **signature** (`string`) Signature to match the backend source to be part of the generation. - **TextAnnotationDelta**: - **type** (`object`) *(Required)* Value: `text_annotation_delta` - **annotations** (`array (Annotation)`) Citation information for model-generated content. **Possible Types:** (Discriminator: `type`) - **UrlCitation**: A URL citation annotation. - **type** (`object`) *(Required)* Value: `url_citation` - **url** (`string`) The URL. - **title** (`string`) The title of the URL. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **FileCitation**: A file citation annotation. - **type** (`object`) *(Required)* Value: `file_citation` - **document_uri** (`string`) The URI of the file. - **file_name** (`string`) The name of the file. - **source** (`string`) Source attributed for a portion of the text. - **custom_metadata** (`object`) User provided metadata about the retrieved context. - **page_number** (`integer`) Page number of the cited document, if applicable. - **media_id** (`string`) Media ID in-case of image citations, if applicable. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **PlaceCitation**: A place citation annotation. - **type** (`object`) *(Required)* Value: `place_citation` - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **start_index** (`integer`) Start of segment of the response that is attributed to this source. Index indicates the start of the segment, measured in bytes. - **end_index** (`integer`) End of the attributed segment, exclusive. - **ArgumentsDelta**: - **type** (`object`) *(Required)* Value: `arguments_delta` - **arguments** (`string`) - **CodeExecutionCallDelta**: - **type** (`object`) *(Required)* Value: `code_execution_call` - **arguments** (`CodeExecutionCallArguments`) *(Required)* - **language** (`enum (string)`) Programming language of the `code`. Possible values: - `python`: Python >= 3.10, with numpy and simpy available. - **code** (`string`) The code to be executed. - **signature** (`string`) A signature hash for backend validation. - **UrlContextCallDelta**: - **type** (`object`) *(Required)* Value: `url_context_call` - **arguments** (`UrlContextCallArguments`) *(Required)* - **urls** (`array (string)`) The URLs to fetch. - **signature** (`string`) A signature hash for backend validation. - **GoogleSearchCallDelta**: - **type** (`object`) *(Required)* Value: `google_search_call` - **arguments** (`GoogleSearchCallArguments`) *(Required)* - **queries** (`array (string)`) Web search queries for the following-up web search. - **signature** (`string`) A signature hash for backend validation. - **McpServerToolCallDelta**: - **type** (`object`) *(Required)* Value: `mcp_server_tool_call` - **name** (`string`) *(Required)* - **server_name** (`string`) *(Required)* - **arguments** (`object`) *(Required)* - **FileSearchCallDelta**: - **type** (`object`) *(Required)* Value: `file_search_call` - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsCallDelta**: - **type** (`object`) *(Required)* Value: `google_maps_call` - **arguments** (`GoogleMapsCallArguments`) The arguments to pass to the Google Maps tool. - **queries** (`array (string)`) The queries to be executed. - **signature** (`string`) A signature hash for backend validation. - **CodeExecutionResultDelta**: - **type** (`object`) *(Required)* Value: `code_execution_result` - **result** (`string`) *(Required)* - **is_error** (`boolean`) - **signature** (`string`) A signature hash for backend validation. - **UrlContextResultDelta**: - **type** (`object`) *(Required)* Value: `url_context_result` - **result** (`array (UrlContextResult)`) *(Required)* - **url** (`string`) The URL that was fetched. - **status** (`enum (string)`) The status of the URL retrieval. Possible values: - `success`: Url retrieval is successful. - `error`: Url retrieval is failed due to error. - `paywall`: Url retrieval is failed because the content is behind paywall. - `unsafe`: Url retrieval is failed because the content is unsafe. - **is_error** (`boolean`) - **signature** (`string`) A signature hash for backend validation. - **GoogleSearchResultDelta**: - **type** (`object`) *(Required)* Value: `google_search_result` - **result** (`array (GoogleSearchResult)`) *(Required)* - **search_suggestions** (`string`) Web content snippet that can be embedded in a web page or an app webview. - **is_error** (`boolean`) - **signature** (`string`) A signature hash for backend validation. - **McpServerToolResultDelta**: - **type** (`object`) *(Required)* Value: `mcp_server_tool_result` - **name** (`string`) - **server_name** (`string`) - **result** (`array (ImageContent or TextContent) or object or string`) *(Required)* - **FileSearchResultDelta**: - **type** (`object`) *(Required)* Value: `file_search_result` - **result** (`array (FileSearchResult)`) *(Required)* - **signature** (`string`) A signature hash for backend validation. - **GoogleMapsResultDelta**: - **type** (`object`) *(Required)* Value: `google_maps_result` - **result** (`array (GoogleMapsResult)`) The results of the Google Maps. - **places** (`array (Places)`) The places that were found. - **place_id** (`string`) The ID of the place, in `places/{place_id}` format. - **name** (`string`) Title of the place. - **url** (`string`) URI reference of the place. - **review_snippets** (`array (ReviewSnippet)`) Snippets of reviews that are used to generate answers about the features of a given place in Google Maps. - **title** (`string`) Title of the review. - **url** (`string`) A link that corresponds to the user review on Google Maps. - **review_id** (`string`) The ID of the review snippet. - **widget_context_token** (`string`) Resource name of the Google Maps widget context token. - **signature** (`string`) A signature hash for backend validation. - **FunctionResultDelta**: - **type** (`object`) *(Required)* Value: `function_result` - **name** (`string`) - **is_error** (`boolean`) - **call_id** (`string`) *(Required)* Required. ID to match the ID from the function call block. - **result** (`array (ImageContent or TextContent) or object or string`) *(Required)* - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StepDeltaMetadata`) Optional metadata accompanying ANY streamed event. - **total_usage** (`Usage`) Statistics on the interaction request's token usage. - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Step Delta** ```json { "event_type": "step.delta", "index": 0, "delta": { "type": "text", "text": "Hello" } } ``` - **StepStop** - - **event_type** (`object`) *(Required)* Value: `step.stop` - **index** (`integer`) *(Required)* - **usage** (`Usage`) Cumulative model usage stats from the start of the session. - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. - **step_usage** (`Usage`) Model usage stats for this specific step. - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. - **event_id** (`string`) The event_id token to be used to resume the interaction stream, from this event. - **metadata** (`StreamMetadata`) Optional metadata accompanying ANY streamed event. - **total_usage** (`Usage`) - **total_input_tokens** (`integer`) Number of tokens in the prompt (context). - **input_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of input token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_cached_tokens** (`integer`) Number of tokens in the cached part of the prompt (the cached content). - **cached_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of cached token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_output_tokens** (`integer`) Total number of tokens across all the generated responses. - **output_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of output token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_tool_use_tokens** (`integer`) Number of tokens present in tool-use prompt(s). - **tool_use_tokens_by_modality** (`array (ModalityTokens)`) A breakdown of tool-use token usage by modality. - **modality** (`ResponseModality`) The modality associated with the token count. Possible values: - `text`: Indicates the model should return text. - `image`: Indicates the model should return images. - `audio`: Indicates the model should return audio. - `video`: Indicates the model should return video. - `document`: Indicates the model should return documents. - **tokens** (`integer`) Number of tokens for the modality. - **total_thought_tokens** (`integer`) Number of tokens of thoughts for thinking models. - **total_tokens** (`integer`) Total token count for the interaction request (prompt + responses + other internal tokens). - **grounding_tool_count** (`array (GroundingToolCount)`) Grounding tool count. - **type** (`enum (string)`) The grounding tool type associated with the count. Possible values: - `google_search`: Grounding with Google Web Search and Image Search, & Web Grounding for Enterprise. - `google_maps`: Grounding with Google Maps. - `retrieval`: Grounding with customer's data, for example, VertexAISearch. - **count** (`integer`) The number of grounding tool counts. **Examples** **Step Stop** ```json { "event_type": "step.stop", "index": 0 } ``` **JSON Representation:** ```json { "event_type": {}, "event_id": "string", "metadata": { "total_usage": { "total_input_tokens": 0, "input_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_cached_tokens": 0, "cached_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_output_tokens": 0, "output_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_tool_use_tokens": 0, "tool_use_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_thought_tokens": 0, "total_tokens": 0, "grounding_tool_count": [ { "type": "google_search", "count": 0 } ] } }, "interaction": { "id": "string", "object": "string", "model": "string", "agent": "string", "status": "in_progress", "created": "string", "updated": "string", "service_tier": "flex", "usage": { "total_input_tokens": 0, "input_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_cached_tokens": 0, "cached_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_output_tokens": 0, "output_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_tool_use_tokens": 0, "tool_use_tokens_by_modality": [ { "modality": "text", "tokens": 0 } ], "total_thought_tokens": 0, "total_tokens": 0, "grounding_tool_count": [ { "type": "google_search", "count": 0 } ] }, "steps": [ { "content": [ "..." ], "type": {} } ] } } ```