Tools

Spring AI Book

This page comes from my Spring AI book.

This article shows how to use tools in Spring AI.

Basic Tool Usage

Create Tools

The first step is to create tools. The easiest way to create tools is using POJO classes and @Tool annotation.

CalculatorTool shown below is a POJO class with two methods add and subtract annotated with @Tool. By adding the @Tool annotation to a method, this method becomes a tool that can be called by a LLM.

CalculatorTool

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;

public class CalculatorTool {
  @Tool(name = "add", description = "Add two numbers")
  public int add(
      @ToolParam(description = "first number") int a,
      @ToolParam(description = "second number") int b) {
    return a + b;
  }

  @Tool(name = "subtract", description = "Subtract b from a")
  public int subtract(
      @ToolParam(description = "first number") int a,
      @ToolParam(description = "second number") int b) {
    return a - b;
  }
}

The table below shows attributes of @Tool.

Attribute	Type	Default value	Description
name	String	''	The name of the tool. If not provided, the method name will be used.
description	String	''	The description of the tool. If not provided, the method name will be used.
returnDirect	boolean	false	Whether the tool result should be returned directly or passed back to the model.
resultConverter	Class<? extends ToolCallResultConverter>	DefaultToolCallResultConverter.class	The class to use to convert the tool call result to a String.

For a method annotated with @Tool, the @ToolParam annotation can be used to provide metadata about an argument. It's recommended to always provide a description of an argument.

The table below shows attributes of @ToolParam.

Attribute	Type	Default value	Description
name	String	''	The description of the tool argument.
required	boolean	true	Whether the tool argument is required.

Use Tools

After creating tools, we can now use them when interacting with an LLM. This is done by using the tools method of ChatClientRequestSpec.

In the ToolCallController shown below, when calling the tools method, a new instance of CalculatorTool is provided. This makes tools in CalculatorTool available to the LLM.

ToolCallController

@RestController
public class ToolCallController {

  private final ChatClient chatClient;

  public ToolCallController(ChatClient.Builder builder,
                            LoggingAdvisor loggingAdvisor) {
    this.chatClient = builder.defaultAdvisors(loggingAdvisor).build();
  }

  @PostMapping("/toolcall")
  public String chat(@RequestBody ChatRequest request) {
    return chatClient.prompt().user(request.input())
        .tools(new CalculatorTool())
        .call().content();
  }
}

Here we only need to provide name and description of a tool and its arguments, the actual input schema required by the LLM is constructed by Spring AI.

ToolCallback

POJO classes and the usage of @Tool annotation are convenient ways for developers to easily create tools. Behind the scenes, org.springframework.ai.tool.ToolCallback is the actual interface to represent a tool whose execution can be triggered by an AI model.

ToolCallback is a simple interface with a few methods.

• The getToolDefinition method returns the ToolDefinition used by the AI model to determine when and how to call the tool.

• The call method executes the tool with the given input and return the result to send back to the AI model.

• The getToolMetadata method returns the ToolMetadata providing additional information on how to handle the tool.

ToolCallback

public interface ToolCallback {
  ToolDefinition getToolDefinition();

  String call(String toolInput);

  default ToolMetadata getToolMetadata() {
    return ToolMetadata.builder().build();
  }
}

ToolDefinition

ToolDefinition provides the definition of a tool. It provides methods to get name, description and input schema of a tool.

ToolCallback

public interface ToolDefinition {
  String name();

  String description();

  String inputSchema();
}

The builder method of ToolDefinition returns a builder to create new ToolDefinition objects.

Most of the time, we don't need to implement ToolCallback ourselves, but use built-in implementations.

MethodToolCallback

MethodToolCallback creates tools from Java Method objects. Actually, methods annotated with @Tool annotation are converted to MethodToolCallback objects.

FunctionToolCallback

FunctionToolCallback creates tools from BiFunction objects. A BiFunction object takes two input parameters, the input object and a ToolContext, and returns a result object.

FunctionToolCallback objects are usually created from builders returned by calling the builder method.

GetWeatherTool shown below is a Java Function. It accepts a GetWeatherRequest object and returns a GetWeatherResponse object.

GetWeatherTool

import java.util.function.Function;

public class GetWeatherTool implements
    Function<GetWeatherTool.GetWeatherRequest,
        GetWeatherTool.GetWeatherResponse> {

  @Override
  public GetWeatherResponse apply(GetWeatherRequest getWeatherRequest) {
    return new GetWeatherResponse("Sunny");
  }

  public record GetWeatherRequest(String location) {
  }

  public record GetWeatherResponse(String condition) {
  }
}

This GetWeatherTool function can be used to create a FunctionToolCallback.

GetWeatherToolConfig

@Configuration
public class GetWeatherToolConfig {
  @Bean
  @Qualifier("getWeather")
  public FunctionToolCallback<?, ?> getWeatherTool() {
    return FunctionToolCallback.builder("getWeather", new GetWeatherTool())
        .description("Get weather")
        .inputType(GetWeatherTool.GetWeatherRequest.class)
        .build();
  }
}

Below is the updated ToolCallController with the new tool.

ToolCallController

@RestController
public class ToolCallController {

  private final ChatClient chatClient;
  private final FunctionToolCallback<?, ?> getWeatherTool;

  public ToolCallController(
      ChatClient.Builder builder,
      LoggingAdvisor loggingAdvisor,
      @Qualifier("getWeather") FunctionToolCallback<?, ?> getWeatherTool) {
    this.getWeatherTool = getWeatherTool;
    this.chatClient = builder.defaultAdvisors(loggingAdvisor).build();
  }

  @PostMapping("/toolcall")
  public String chat(@RequestBody ChatRequest request) {
    return chatClient.prompt().user(request.input())
        .tools(new CalculatorTool(), this.getWeatherTool)
        .call().content();
  }
}

ToolCallbackProvider

Except from using ToolCallbacks directly, we can also use ToolCallbackProviders to provide ToolCallbacks. ToolCallbackProvider is a simple interface with only one method getToolCallbacks to return an array of ToolCallbacks.

ToolCallbackProvider

public interface ToolCallbackProvider {

  ToolCallback[] getToolCallbacks();
}    

StaticToolCallbackProvider is an implementation of ToolCallbackProvider that maintains a static array of ToolCallback objects.