Tool
Tools play an important role in building AI applications, especially for agents. After an LLM is trained, its parametric knowledge is frozen. By using techniques like Retrieval Augmented Generation (RAG), we can provide extra contextual information when interacting with LLMs. Tools, on the other hand, allow us to extend LLM’s capabilities to interact with the real world.
Just like human beings, using tools is a major step for an AI application to become more powerful. This means that the AI application not only generates content, but also starts to impact the real world and help users solve real problems.
A typical tool that an LLM can use is to get the weather. Apparently, an LLM itself doesn’t know what’s the weather will look like tomorrow. However, this information is useful for the LLM to generate meaningful content. For example, if you are using an LLM to plan a journey, knowing the weather forecast will be very helpful. If it’s going to be cold and rainy, then the LLM should plan for more indoor activities, and suggest you to bring more clothes.
A tool has a name, a description, and a schema for the input parameters. The name and description is for the LLM to infer whether it should use this tool. The schema of input parameters instructs the LLM about how to invoke this tool.
The key point here is that an LLM only infers the parameters to invoke a tool. Actual invocations of a tool are done by the AI application. Invocation results will be passed back to the LLM for further generations.
Let’s use the weather tool as an example. This tool accepts a single string parameter city, which means the location to get the current weather.
Below is the JSON description of this tool. The schema of parameters is described using JSON Schema.
{
"name": "getWeather",
"description": "Get the current weather by location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City name"
}
}
}
}
When sending a request to the LLM, the tool's name and description can be included in the request. If the LLM infers that it should invoke the tool to get the result, it infers the parameters to invoke the function, then returns it to the application.
{
"function": "getWeather",
"parameters": {
"location": "New York"
}
}
After receiving the response from the LLM, the application finds the getWeather tool from its internal registry, then invokes this tool with provided parameters. The invocation result is then passed back to LLM.
{
"function": "getWeather",
"result": {
"weather": "Cold and rainy"
}
}
LLM uses the tool invocation result to generate content.