Execute Agent

post https://api.egp.scale.com/v4/agents/execute

Description

Executes one Agent inference step. Given a list of messages and a list of tools to ask for help from, the Agent will either respond with a final answer directly or ask the user to execute a tool to provide more information.

Details

An Agent is a component that utilizes a Language Model (LLM) as an interpreter and decision maker. Unlike asking an LLM for a direct response, communicating with an agent consists of a running dialogue where an agent can optionally ask the user to execute specialized tools for specific tasks, such as calculations, web searches, or accessing custom data from private knowledge bases.

An agent is designed to be stateless, emitting outputs one step at a time. This means that client-side applications are responsible for managing message history, tool execution, and responses. This grants users greater flexibility to write and execute custom tools and maintain explicit control over their message history.

Message Types

User Message: A message from the user to the agent.
System Message: An informational text message from the system to guide the agent. It is not a user message or agent message because it did not come from either entity.
Agent Message: A message from the agent to the client. It will contain either a final answer as content or a request for the user to execute a tool as a tool_request.
Tool Message: A message from the user to the agent that contains the output of a tool execution. The tool message will be processed by the agent and the agent will respond with either a final answer or another tool request.

Agent Instructions

Instructions are used to guide the agent's decision making process and output generation.

Good prompt engineering is crucial to getting performant results from the agent. If you are having trouble getting the agent to perform well, try writing more specific instructions before trying more expensive techniques such as swapping in other models or finetuning the underlying LLM.

For example, the default instructions we set for the agent are the following:

You are an AI assistant that helps users with their questions. You can answer questions directly or acquire information from any of the attached tools to assist you. Always answer the user's most recent query to the best of your knowledge.

When asked about what tools are available, you must list each attached tool's name and description. When asked about what you can do, mention that in addition to your normal capabilities, you can also use the attached tools by listing their names and descriptions. You cannot use any other tools other than the ones provided to you explicitly.

Restrictions and Limits

Message Limits:

The message list is not limited by length, but by the context limit of the underlying language model. If you are getting an error regarding the underlying model's context limit, try using a memory strategy to condense the input messages.

Model Restrictions:

Currently, only closed source models like GPT and Claude are supported due to the limitations of open source models when it comes to tool selection, generating tool arguments in valid JSON, and planning out multi-step tool execution. Specialized fine-tuning will likely be required to make open source models compatible with agents.

Body Params

model

string

required

The ID of the model to use for the agent. We only support the models listed here so far.

memory_strategy

object

The memory strategy to use for the agent. A memory strategy is a way to prevent the underlying LLM's context limit from being exceeded. Each memory strategy uses a different technique to condense the input message list into a smaller payload for the underlying LLM.

We only support the Last K memory strategy right now, but will be adding new strategies soon.

tools

array of objects

required

The list of specs of tools that the agent can use. Each spec must contain a name key set to the name of the tool, a description key set to the description of the tool, and an arguments key set to a JSON Schema compliant object describing the tool arguments.

The name and description of each tool is used by the agent to decide when to use certain tools. Because some queries are complex and may require multiple tools to complete, it is important to make these descriptions as informative as possible. If a tool is not being chosen when it should, it is common practice to tune the description of the tool to make it more apparent to the agent when the tool can be used effectively.

Tools*

messages

array

required

The list of messages in the conversation.

Expand each message type to see how it works and when to use it. Most conversations should begin with a single user message.

Messages*

model_parameters

object

Configuration parameters for the agent model, such as temperature, max_tokens, and stop_sequences.

If not specified, the default value are:

temperature: 0.2
max_tokens: None (limited by the model's max tokens)
stop_sequences: None

instructions

string

Defaults to You are an AI assistant that helps users with their questions. You can answer questions directly or acquire information from any of the attached tools to assist you. Always answer the user's most recent query to the best of your knowledge. When asked about what tools are available, you must list each attached tool's name and description. When asked about what you can do, mention that in addition to your normal capabilities, you can also use the attached tools by listing their names and descriptions. You cannot use any other tools other than the ones provided to you explicitly.

The initial instructions to provide to the agent.

Use this to guide the agent to act in more specific ways. For example, if you have specific rules you want to restrict the agent to follow you can specify them here. For example, if I want the agent to always use certain tools before others, I can write that rule in these instructions.

Good prompt engineering is crucial to getting performant results from the agent. If you are having trouble getting the agent to perform well, try writing more specific instructions here before trying more expensive techniques such as swapping in other models or finetuning the underlying LLM.

Responses

Language

Credentials

Header

Request


xxxxxxxxxx
16
1
curl --request POST \
2
     --url https://api.egp.scale.com/v4/agents/execute \
3
     --header 'accept: application/json' \
4
     --header 'content-type: application/json' \
5
     --data @- <<EOF
6
{
7
  "model": "gpt-4",
8
  "memory_strategy": {
9
    "name": "last_k"
10
  },
11
  "model_parameters": {
12
    "temperature": 0.2
13
  },
14
  "instructions": "You are an AI assistant that helps users with their questions. You can answer questions directly or acquire information from any of the attached tools to assist you. Always answer the user's most recent query to the best of your knowledge.\n\nWhen asked about what tools are available, you must list each attached tool's name and description. When asked about what you can do, mention that in addition to your normal capabilities, you can also use the attached tools by listing their names and descriptions. You cannot use any other tools other than the ones provided to you explicitly."
15
}
16
EOF

Click Try It! to start a request and see the response here! Or choose an example:

application/json

200Successful Response

422Validation Error