ModelClient

``autogen.ModelClient #

Bases: Protocol

A client class must implement the following methods:

• create must return a response object that implements the ModelClientResponseProtocol
• cost must return the cost of the response
• get_usage must return a dict with the following keys:
  • prompt_tokens
  • completion_tokens
  • total_tokens
  • cost
  • model

This class is used to create a client that can be used by OpenAIWrapper. The response returned from create must adhere to the ModelClientResponseProtocol but can be extended however needed. The message_retrieval method must be implemented to return a list of str or a list of messages from the response.

RESPONSE_USAGE_KEYS`class-attributeinstance-attribute`#

RESPONSE_USAGE_KEYS = ['prompt_tokens', 'completion_tokens', 'total_tokens', 'cost', 'model']

``ModelClientResponseProtocol #

Bases: Protocol

``choicesinstance-attribute#

choices

``modelinstance-attribute#

model

``Choice #

Bases: Protocol

``messageinstance-attribute#

message

``Message #

Bases: Protocol

``contentinstance-attribute#

content

``create #

create(params)

Source code in autogen/llm_config/client.py

``message_retrieval #

message_retrieval(response)

Retrieve and return a list of strings or a list of Choice.Message from the response.

NOTE: if a list of Choice.Message is returned, it currently needs to contain the fields of OpenAI's ChatCompletion Message object, since that is expected for function or tool calling in the rest of the codebase at the moment, unless a custom agent is being used.

Source code in autogen/llm_config/client.py

``cost #

cost(response)

Source code in autogen/llm_config/client.py

``get_usagestaticmethod#

get_usage(response)

Return usage summary of the response using RESPONSE_USAGE_KEYS.

Source code in autogen/llm_config/client.py