Evaluator-Optimizer Agent

To use this agent, add the Maven dependency to your project, replacing ${agentic-patterns-version} with the latest version.

Latest version:

<dependency>
    <groupId>com.javaaidev.agenticpatterns</groupId>
    <artifactId>evaluator-optimizer</artifactId>
    <version>${agentic-patterns-version}</version>
</dependency>

Usage

EvaluatorOptimizerAgent

To create a Evaluator-Optimizer agent, you can create a subclass of EvaluatorOptimizerAgent. When constructing a EvaluatorOptimizerAgent, parameters in the table below can be provided.

You can use the same ChatClient for generation and evaluation.

Parameter	Required?	Type	Description
generationChatClient	yes	ChatClient	Spring AI ChatClient for generating results
evaluationChatClient	yes	ChatClient	Spring AI ChatClient for evaluating results
responseType	no	Type	Type of response, only required when auto-detection fails
observationRegistry	no	ObservationRegistry	Micrometer ObservationRegistry for tracing

EvaluatorOptimizerAgent uses three other TaskExecutionAgents for execute a task. These agents must be created by subclasses.

Agent	Request	Response	Method	Description
initialResultAgent	Request	Response	buildInitialResultAgent	Agent to generate the initial result
evaluationAgent	Request	Evaluation	buildEvaluationAgent	Agent to evaluate a result
optimizationAgent	OptimizationInput<Response>	Response	buildOptimizationAgent	Agent to optimize a result based on feedback

The subclass can override getMaxIterations method to update the maximum number of evaluation iterations. The default value is 3.

PromptBasedEvaluatorOptimizerAgent

Most of agents used by an EvaluatorOptimizerAgent use an LLM to generate results. In this case, you can extend from PromptBasedEvaluatorOptimizerAgent instead. When extending from PromptBasedEvaluatorOptimizerAgent, you only need to provide prompt templates and the logic to prepare values for template variables.

Method	Description
getInitialResultPromptTemplate	Prompt template for the agent to generate initial result
buildInitialResultPromptContext	Prepare for the values of variables in the prompt template to generate initial result
getEvaluationPromptTemplate	Prompt template for the agent to evaluate a result
buildEvaluationPromptContext	Prepare for the values of variables in the prompt template to evaluate a result
getOptimizationPromptTemplate	Prompt template for the agent to optimize a result
buildOptimizationPromptContext	Prepare for the values of variables in the prompt template to optimize a result

Example

The example is an agent to generate code with evaluation feedbacks.

In the implementation of CodeGenerationAgent, we only need to implement methods to get the prompt templates and prepare the context for templates.

CodeGenerationAgent

package com.javaaidev.agenticpatterns.examples.evaluatoroptimizer;

import com.javaaidev.agenticpatterns.core.AgentUtils;
import com.javaaidev.agenticpatterns.evaluatoroptimizer.PromptBasedEvaluatorOptimizerAgent;
import com.javaaidev.agenticpatterns.examples.evaluatoroptimizer.CodeGenerationAgent.CodeGenerationRequest;
import com.javaaidev.agenticpatterns.examples.evaluatoroptimizer.CodeGenerationAgent.CodeGenerationResponse;
import io.micrometer.observation.ObservationRegistry;
import java.util.Map;
import java.util.Optional;
import org.jspecify.annotations.Nullable;
import org.springframework.ai.chat.client.ChatClient;

/**
 * An agent to generate code with evaluation feedbacks
 */
public class CodeGenerationAgent extends
    PromptBasedEvaluatorOptimizerAgent<CodeGenerationRequest, CodeGenerationResponse> {

  public CodeGenerationAgent(ChatClient chatClient,
      @Nullable ObservationRegistry observationRegistry) {
    super(chatClient, chatClient, CodeGenerationResponse.class, observationRegistry);
  }

  @Override
  protected String getInitialResultPromptTemplate() {
    return AgentUtils.loadPromptTemplateFromClasspath(
        "prompt_template/code-generator/initial-result.st");
  }

  @Override
  protected @Nullable Map<String, Object> buildInitialResultPromptContext(
      @Nullable CodeGenerationRequest request) {
    return AgentUtils.objectToMap(request);
  }

  @Override
  protected String getEvaluationPromptTemplate() {
    return AgentUtils.loadPromptTemplateFromClasspath(
        "prompt_template/code-generator/evaluation.st");
  }

  @Override
  protected @Nullable Map<String, Object> buildEvaluationPromptContext(
      @Nullable CodeGenerationResponse codeGenerationResponse) {
    return AgentUtils.objectToMap(codeGenerationResponse);
  }

  @Override
  protected String getOptimizationPromptTemplate() {
    return AgentUtils.loadPromptTemplateFromClasspath(
        "prompt_template/code-generator/optimization.st");
  }

  @Override
  protected @Nullable Map<String, Object> buildOptimizationPromptContext(
      @Nullable OptimizationInput<CodeGenerationResponse> optimizationInput) {
    var optionalInput = Optional.ofNullable(optimizationInput);
    return Map.of(
        "code", optionalInput.map(input -> input.response().code()),
        "feedback", optionalInput.map(input -> input.evaluation().feedback())
    );
  }

  public record CodeGenerationRequest(String input) {

  }

  public record CodeGenerationResponse(String code) {

  }
}

We can also use a REST API to access the agent.

REST controller of agent

package com.javaaidev.agenticpatterns.examples.evaluatoroptimizer;

import com.javaaidev.agenticpatterns.examples.evaluatoroptimizer.CodeGenerationAgent.CodeGenerationRequest;
import com.javaaidev.agenticpatterns.examples.evaluatoroptimizer.CodeGenerationAgent.CodeGenerationResponse;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/code_generation")
public class CodeGenerationController {

  private final CodeGenerationAgent codeGenerationAgent;

  public CodeGenerationController(CodeGenerationAgent codeGenerationAgent) {
    this.codeGenerationAgent = codeGenerationAgent;
  }

  @PostMapping
  public CodeGenerationResponse generateCode(@RequestBody CodeGenerationRequest request) {
    return codeGenerationAgent.call(request);
  }
}