Skip to main content

Agent as Tool

LLMs have the capability to call external tools. Agent as Tool pattern allows an agent to be used as a tool for an LLM to use. This means that an LLM can choose to call an agent as a tool when it thinks necessary. This also means that we don't need to always have fixed processes to finish a task, but ask an LLM to act on its own to finish a task.

We can mix agents and tools to handle various types of tasks.

  • If a task has a predefined flow, patterns like Evaluator-Optimizer, Parallelization Workflow, Routing Workflow and Chain Workflow can be used to build an agent to finish this task. This agent leverages other agents to finish the task.
  • If a task doesn't have a predefined flow or requires creative thinking, we can provide a list of agents as tools to an LLM. The LLM uses these tools to finish the task.

Implementation​

To register an agent as a tool for an LLM to use, we can leverage the capability provided by different frameworks. For Spring AI, we can define a Task Execution agent as a Java Function, then register this Function as a Spring bean. These tools can be enabled when using a ChatClient to send the requests to an LLM.

Example​

The example is to generate Python code to test a REST API. This REST API accepts the User model created in the example of Task Execution pattern. The agent to generate test users is configured as a tool for an LLM to use.

Below is the prompt for an LLM to generate code.

Prompt to generate REST API test
Goal: Write Python code to test a REST API.
        
Requirements:
- The API is responsible for creating a new user.
- Use a POST request with JSON content to create a new user.
- When creation is successful, the API returns a 201 code.
- The API url is http://localhost:8080/api/v1/user.
- User data has a defined structure, don't generate it yourself, use the tool to generate it.

From the generated code, we can see that the LLM uses the user generation agent to generate a test user.

Generated Python code
import requests

# Define the API URL
api_url = "http://localhost:8080/api/v1/user"

# Define the user data
user_data = {
    "id": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
    "name": "John Doe",
    "email": "john.doe@example.com",
    "mobilePhone": "+1234567890",
    "addresses": [
        {
            "id": "c9bf9e57-1685-4c89-bafb-ff5af830be8a",
            "addressType": "HOME",
            "countryOrRegion": "US",
            "provinceOrState": "NY",
            "city": "New York",
            "addressLine": "123 Fake Street",
            "zipCode": "10001"
        }
    ]
}

# Make the POST request to create a new user
response = requests.post(api_url, json=user_data)

# Check if the user was created successfully
if response.status_code == 201:
    print("User created successfully.")
else:
    print(f"Failed to create user. Status code: {response.status_code}")
Last updated on