# Model Configuration
The AG2 framework provides an explicit, predictable, and type-safe way to configure Large Language Models (LLMs) for your agents. The configuration API is designed to provide a consistent developer experience across different model providers while maintaining strong typing support.
## Supported Providers
AG2 supports multiple LLM providers through dedicated configuration classes. Each provider requires its respective optional dependencies to be installed.
| Provider | Configuration Class | Installation Command |
| --- | --- | --- |
| **[OpenAI Responses](https://developers.openai.com/api/reference/responses/overview)** | `OpenAIResponsesConfig` | `pip install "ag2[openai]"` |
| **[OpenAI](https://developers.openai.com/api/reference/overview)** | `OpenAIConfig` | `pip install "ag2[openai]"` |
| **[Anthropic](https://platform.claude.com/docs/en/build-with-claude/overview)** | `AnthropicConfig` | `pip install "ag2[anthropic]"` |
| **[Gemini](https://ai.google.dev/gemini-api/docs)** | `GeminiConfig` | `pip install "ag2[gemini]"` |
| **[Gemini on Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs)** | `VertexAIConfig` | `pip install "ag2[gemini]"` |
| **[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html)** | `BedrockConfig` | `pip install "ag2[bedrock]"` |
| **[Ollama](https://docs.ollama.com/api/introduction)** | `OllamaConfig` | `pip install "ag2[ollama]"` |
| **[DashScope](https://www.alibabacloud.com/help/en/model-studio/first-api-call-to-qwen)** | `DashScopeConfig` | `pip install "ag2[dashscope]"` |
| **[xAI](https://docs.x.ai/docs/overview)** | `XAIConfig` | `pip install "ag2[xai]"` |
| **[Z.AI](https://docs.z.ai/)** | `ZAIConfig` | `pip install "ag2[zai]"` |
*(Note: `OpenAIConfig` is also available for OpenAI-compatible endpoints).*
## How to Configure a Model
### Basic Configuration
To configure a model, import the specific provider's configuration class and initialize it with your desired parameters. The most common parameters are `model`, `api_key`, and `base_url`.
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
``` | ```
from ag2.config import OpenAIResponsesConfig
# Configure an OpenAI Responses API model
config = OpenAIResponsesConfig(
model="gpt-4.1-nano",
api_key="sk-...",
streaming=True
)
``` |
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
9
``` | ```
from ag2.config import OpenAIConfig
# Configure an OpenAI model
config = OpenAIConfig(
model="gpt-4o-mini",
api_key="sk-...",
temperature=0.2,
streaming=True
)
``` |
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
``` | ```
from ag2.config import AnthropicConfig
# Configure an Anthropic model
config = AnthropicConfig(
model="claude-haiku-4-5-20251001",
api_key="sk-ant-...",
streaming=True
)
``` |
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
``` | ```
from ag2.config import GeminiConfig
# Configure a Gemini model
config = GeminiConfig(
model="gemini-3-flash-preview",
api_key="...",
streaming=True
)
``` |
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
``` | ```
from ag2.config import BedrockConfig
# Configure an Amazon Bedrock model (Converse API)
config = BedrockConfig(
model="anthropic.claude-sonnet-4-5-20250929-v1:0",
region_name="us-east-1",
streaming=True
)
``` |
Credentials follow the standard AWS resolution chain: explicit `aws_access_key_id` / `aws_secret_access_key`, a named `profile_name`, environment variables, shared config files, or instance roles. `model` accepts a Bedrock model id or an inference-profile ARN. See [Amazon Bedrock authentication](https://docs.ag2.ai/docs/user-guide/model_configuration/#amazon-bedrock-authentication) for the API-key (bearer token) alternative.
### Using Environment Variables
For security and convenience, you don't need to hardcode your API keys. If `api_key` is not explicitly provided, the configuration will automatically attempt to load it from your environment variables.
The system looks for provider-specific keys (e.g., `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, `XAI_API_KEY`, `ZAI_API_KEY`).
| | |
| --- | --- |
| ```
1
2
3
4
``` | ```
from ag2.config import OpenAIConfig
# Automatically falls back to OPENAI_API_KEY from the environment
config = OpenAIConfig(model="gpt-5")
``` |
### Amazon Bedrock Authentication
`BedrockConfig` authenticates in either of two ways, both resolved by the underlying AWS SDK:
**1. AWS credentials (SigV4)** — explicit keys, a `profile_name`, or the standard environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`). Recommended for production; credentials refresh automatically through botocore.
**2. Bedrock API keys (bearer token)** — set the [Amazon Bedrock API key](https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html) as an environment variable and boto3 uses bearer-token auth for Bedrock calls automatically (no other credentials needed):
```
export AWS_BEARER_TOKEN_BEDROCK=
export AWS_DEFAULT_REGION=us-east-1
```
| | |
| --- | --- |
| ```
1
2
3
4
``` | ```
from ag2.config import BedrockConfig
# Auth from AWS_BEARER_TOKEN_BEDROCK, region from AWS_DEFAULT_REGION
config = BedrockConfig(model="anthropic.claude-sonnet-4-5-20250929-v1:0")
``` |
A region is always required — pass `region_name=` or set `AWS_DEFAULT_REGION`. Notes on API keys:
- **Short-term keys** expire with the console session that minted them (max 12 hours) and are region-bound — generate the key in the same region you call.
- **Long-term keys** are backed by an auto-created IAM user; AWS recommends them for exploration only.
- API keys work only for Bedrock / Bedrock Runtime actions. If both a bearer token and AWS credentials are present, the bearer token wins for Bedrock calls.
### Google Vertex AI (Gemini)
For Gemini on **Vertex AI** (Google Cloud), use the dedicated `VertexAIConfig` class. `GeminiConfig` covers the public Developer API (`api_key`); `VertexAIConfig` covers the Vertex path (GCP `project`, `location`, and Google-issued credentials).
Authentication accepts any of the following:
[Service account key file](https://docs.ag2.ai/docs/user-guide/model_configuration/#__tabbed_2_1)[Application Default Credentials](https://docs.ag2.ai/docs/user-guide/model_configuration/#__tabbed_2_2)[Pre-built Credentials object](https://docs.ag2.ai/docs/user-guide/model_configuration/#__tabbed_2_3)
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
9
10
``` | ```
from ag2.config import VertexAIConfig
config = VertexAIConfig(
model="gemini-3-flash-preview",
project="my-gcp-project",
location="us-central1",
credentials="/path/to/service-account-key.json",
# Path to a service-account JSON key file downloaded from
# GCP Console -> IAM & Admin -> Service Accounts -> Keys.
)
``` |
The service account needs the **Vertex AI User** (`roles/aiplatform.user`) IAM role on the project.
### Environment variables
Instead of passing parameters explicitly, the underlying `google-genai` SDK resolves any field left unset from the following environment variables:
| Environment variable | Used by | Equivalent parameter | Notes |
| --- | --- | --- | --- |
| `GOOGLE_API_KEY` | `GeminiConfig` | `api_key` | Takes precedence over `GEMINI_API_KEY` if both are set. |
| `GEMINI_API_KEY` | `GeminiConfig` | `api_key` | Developer API key. |
| `GOOGLE_CLOUD_PROJECT` | `VertexAIConfig` | `project` | GCP project ID. |
| `GOOGLE_CLOUD_LOCATION` | `VertexAIConfig` | `location` | GCP region (or `global`). |
| `GOOGLE_APPLICATION_CREDENTIALS` | `VertexAIConfig` | `credentials` | Path to a service-account JSON key file, read via ADC. |
With the three Vertex variables set in the environment, configuration collapses to just the model name:
```
export GOOGLE_CLOUD_PROJECT=my-gcp-project
export GOOGLE_CLOUD_LOCATION=us-central1
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account-key.json
```
| | |
| --- | --- |
| ```
1
2
3
4
``` | ```
from ag2.config import VertexAIConfig
# All Vertex auth parameters resolved from the environment.
config = VertexAIConfig(model="gemini-3-flash-preview")
``` |
### Controlling Gemini Thinking
Gemini 3 Pro models default to **dynamic / unbounded** thinking, which can cause individual calls to spend a large internal token budget before responding. Both `GeminiConfig` and `VertexAIConfig` accept thinking controls that map directly to [Google's Thinking API](https://ai.google.dev/gemini-api/docs/thinking).
Use `thinking_level` for **Gemini 3** models, or `thinking_budget` for **Gemini 2.5** models:
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
``` | ```
from ag2.config import GeminiConfig, VertexAIConfig
# Gemini 3 — bound thinking with a level
gemini3 = GeminiConfig(
model="gemini-3-flash-preview",
thinking_level="low", # "low" | "medium" | "high"
)
# Gemini 2.5 — bound thinking with an explicit token budget
gemini25 = VertexAIConfig(
model="gemini-2.5-pro",
project="my-gcp-project",
location="us-central1",
thinking_budget=512, # 0 disables thinking entirely
)
``` |
### Z.AI (GLM) Configuration
`ZAIConfig` connects to Z.AI's [GLM models](https://docs.z.ai/) through the official `zai-sdk`. Install the optional dependency with `pip install "ag2[zai]"`.
If `api_key` and `base_url` are not passed explicitly, the SDK resolves them from the `ZAI_API_KEY` and `ZAI_BASE_URL` environment variables; `base_url` otherwise defaults to the international endpoint (`https://api.z.ai/api/paas/v4`).
| | |
| --- | --- |
| ```
1
2
3
4
``` | ```
from ag2.config import ZAIConfig
# Auth and endpoint resolved from ZAI_API_KEY / ZAI_BASE_URL when omitted
config = ZAIConfig(model="glm-4.6", streaming=True)
``` |
### Self-Hosted and OpenAI-Compatible Models (vLLM, LM Studio, etc.)
If you are using a self-hosted model or an API that is compatible with the OpenAI format (such as **vLLM**, **LM Studio**, **FastChat**, or **Together AI**), you can use the `OpenAIConfig` class and specify a custom `base_url`.
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
9
``` | ```
from ag2.config import OpenAIConfig
# Configure a vLLM or other OpenAI-compatible endpoint
config = OpenAIConfig(
model="qwen-3",
base_url="http://localhost:8000/v1",
# Some endpoints don't require an API key, but the client expects a non-empty string
api_key="NotRequired",
``` |
### Extra Body Parameters
Some OpenAI API-compatible providers require additional, provider-specific parameters in the request body. Use the `extra_body` parameter on `OpenAIConfig` to pass these through directly to the API call.
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
``` | ```
from ag2.config import OpenAIConfig
# NVIDIA NIM
nemotron = OpenAIConfig(
model="nvidia/nemotron-3-super-120b-a12b",
base_url="https://integrate.api.nvidia.com/v1",
extra_body={"chat_template_kwargs": {"thinking": True}},
)
``` |
## Reusing and Overriding Configurations
Model configurations are **immutable**. If you need to reuse a configuration for multiple agents with slight variations (e.g., changing the model version or adjusting the temperature), use the `.copy()` method. This creates a new updated instance without mutating the original configuration.
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
``` | ```
from ag2.agent import Agent
from ag2.config import OpenAIConfig
base_config = OpenAIConfig(model="gpt-5")
agent1 = Agent(
"Assistant",
# Create a new configuration with updated temperature
config=base_config.copy(temperature=0.2),
)
agent2 = Agent(
"AnotherAssistant",
# Create a new configuration with updated model and temperature
config=base_config.copy(model="gpt-5-mini", temperature=0.8),
)
``` |
## Delaying Model Configuration
In many use cases, you may want to separate the logic of defining your agent (tools, system messages, instructions) from configuring the specific model it uses. This allows you to construct an agent once and dynamically provide the model configuration later during execution.
You can accomplish this by passing the configuration to the `.ask()` method when interacting with the agent. This is especially useful for applications like web servers where the user might bring their own API key or choose a different model on the fly.
| | |
| --- | --- |
| ```
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
``` | ```
from ag2.agent import Agent
from ag2.config import OpenAIConfig
# Define an agent without an initial model config,
# or with a default one you plan to override later
agent = Agent(
"Assistant",
prompt="You are a helpful assistant.",
# other tools and settings...
)
# Ask the agent, passing the explicit model configuration
response = await agent.ask(
"Hello!",
config=OpenAIConfig(
model="gpt-5",
api_key="sk-user-specific-key"
)
)
``` |