Routing Workflow

Routing Workflow patterns routes a task input to another Task Execution agent by selecting the most suitable target from a list of candidate agents. Each route has a name and a description. The logic of target route selection is usually done by using an LLM.

Implementation

This pattern consists of a routing task and a number of target tasks. Only one subtask is selected to execute the input. The main task and all target tasks are implemented using Task Execution pattern.

The flow chart below shows the basic steps.

Route Selection

When using an LLM to select the target route, we need to include name and description of each target route. Information of all target routes and original input are combined together for the LLM to make selection. The LLM returns the name of target route. The task input is then passed to the target route agent for execution, and the result is returned as the task output.

We can include more context information for LLM to select the best route.

Example

An example usage of Routing Workflow pattern is selecting the best sub-system to answer customer support queries. For a large e-commerce website, there may be different departments to handle various customer support queries. Each department has its own customer support agent. These agents may use different fine-tuned models, or connect to different RAG systems. For a customer support query, the correct sub-system is selected first, then the query is forwarded to the selected sub-system for handling.

For demo purpose, we have three target agents to handle customer queries:

• payment: Handle queries about payment and refund
• shipping: Handle queries about shipping
• general: Handle general queries

Given the customer query below:

customer query

I paid my order. why it's still unpaid in the app!

This input is first handled by the routing agent. The prompt sent to an LLM looks like below. The output of LLM is JSON data with two fields:

• name is the name of the selected target route.
• reason is the reason for the selection.

Prompt to select target route

Goal: Select the best target to handle the input

Choices:
- payment: Handle queries about payment and refund
- shipping: Handle queries about shipping
- general: Handle general queries

Input:
I paid my order. why it's still unpaid in the app!

Your response should be in JSON format.
Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
Do not include markdown code blocks in your response.
Remove the ```json markdown from the output.
Here is the JSON Schema instance your output must adhere to:
```{
  "$schema" : "https://json-schema.org/draft/2020-12/schema",
  "type" : "object",
  "properties" : {
    "name" : {
      "type" : "string"
    },
    "reason" : {
      "type" : "string"
    }
  },
  "additionalProperties" : false
}```

The output from LLM is shown as below. The payment route is selected to handle the query.

Route selection result

{
    "name": "payment",
    "reason": "The query is about a payment issue where the user has paid but the app shows the order as unpaid."
}

The query is then forwarded to the payment agent for handling. Below is the output from payment agent. This is the output of the main task.

Output of agent

{
  "answer": "I'm sorry to hear about the issue. Please ensure that the payment was processed successfully and check for any confirmation emails. If the problem persists, try refreshing the app or contact our support team for further assistance."
}

Reference Implementation

See this page for reference implementation and examples.