Introduction to Advisor
This page comes from my Spring AI book.
When using ChatClient to send requests to AI models and receive responses, we may want to apply common processing logics to requests and responses. The Advisor interface in Spring AI represents advisors called before and after the ChatModel.call(Prompt) and ChatModel.stream(Prompt) methods calls. A ChatClient maintains a chain of advisors with shared execution context.
Advisor
Advisor interface extends from Spring Ordered. The order determines the position of an Advisor in the advisor chain.
public interface Advisor extends Ordered {
int DEFAULT_CHAT_MEMORY_PRECEDENCE_ORDER =
Ordered.HIGHEST_PRECEDENCE + 1000;
String getName();
}
There are two sub-interfaces of Advisor, CallAdvisor and StreamAdvisor.
CallAdvisor
The CallAdvisor interface is used for non-streaming calls. It has only one method adviseCall. This method advises the requests sent to AI models.
public interface CallAdvisor extends Advisor {
ChatClientResponse adviseCall(ChatClientRequest chatClientRequest,
CallAdvisorChain callAdvisorChain);
}
ChatClientRequest is a record type which wraps a Prompt and a context map. The Prompt in ChatClientRequest will be sent to AI models. The context map is a generic container for anything that may be used by advisors.
public record ChatClientRequest(Prompt prompt,
Map<String, Object> context) {
}
The result of adviseCall is an ChatClientResponse representing advised responses. ChatClientResponse is a record type which wraps a ChatResponse and a context map.
public record ChatClientResponse(ChatResponse chatResponse,
Map<String, Object> context) {
}
The callAdvisorChain parameter of type CallAdvisorChain represents the advisors chain. The nextCall method of CallAdvisorChain invokes the next advisor in the chain with a ChatClientRequest and receives a ChatClientResponse.
An advisor in the chain prepares the ChatClientRequest for the next advisor in the chain, and receives the ChatClientResponse from the next advisor. The current advisor can also update the received ChatClientResponse when necessary, then return it to the previous advisor.
public interface CallAdvisorChain {
ChatClientResponse nextCall(ChatClientRequest chatClientRequest);
List<CallAdvisor> getCallAdvisors();
CallAdvisorChain copy(CallAdvisor after);
}
The last advisor in the chain uses ChatModel to send requests to AI models. This last advisor is added by ChatClient internally.
StreamAdvisor
The StreamAdvisor interface is used for streaming calls. The adviseStream method returns a Flux<ChatClientResponse> and uses StreamAdvisorChain.
public interface StreamAdvisor extends Advisor {
Flux<ChatClientResponse> adviseStream(
ChatClientRequest chatClientRequest,
StreamAdvisorChain streamAdvisorChain);
}
StreamAdvisorChain has the nextStream method to call next StreamAdvisor in the chain. It has the same function as nextCall in CallAdvisor.
public interface StreamAdvisorChain extends AdvisorChain {
Flux<ChatClientResponse> nextStream(ChatClientRequest chatClientRequest);
List<StreamAdvisor> getStreamAdvisors();
}
BaseAdvisor
If an Advisor needs to support both non-streaming and streaming modes, it can implement BaseAdvisor. BaseAdvisor extends from both CallAdvisor and StreamAdvisor. It provides before and after methods for implementations to implement the advising logic. BaseAdvisor provides default implementation for adviseCall and adviseStream methods using before and after.
public interface BaseAdvisor extends CallAdvisor, StreamAdvisor {
ChatClientRequest before(ChatClientRequest chatClientRequest,
AdvisorChain advisorChain);
ChatClientResponse after(ChatClientResponse chatClientResponse,
AdvisorChain advisorChain);
}