Structured Output
Structured Output plays an important role in integrating AI functionalities into applications. People are familiar with the chat UI provided by ChatGPT, where the AI service mostly returns text and images. When integrating AI services in an application, we want to deal with structured outputs, which makes the output very easy to be consumed by code. Most LLMs support JSON as the output format.
Structured Output in AI Assistant
Enabling structured output is very straightforward. We just tell the LLM to output JSON content in the prompt.
In the prompt below, the LLM is instructed to output two test users in JSON format.
Generate 2 users. Each user has id, name, email and phone number. Output JSON content.
Below is the output of ChatGPT. We can copy the JSON output and use it.
To make sure the JSON output is in correct format, we may also need to include the JSON schema of the JSON output, so LLM will generate JSON data satisfying the schema.
Spring AI Support
Spring AI further simplifies the usage of structured output. In Spring AI, we don't need to use JSON schemas, we only use Java types. After using the call method of ChatClient to send a Prompt to LLM, instead of using the content method to get the text output, we can use the entity method to specify a Java type, then the return value is an object of that type.
Under the hood, Spring AI generates the JSON schema from the Java type using jsonschema-generator, then includes the JSON schema in the prompt sent to LLM. After receiving the response from LLM, JSON content is extracted from the response and deserialized into a Java object. When using Spring AI, we don't need to care about the implementation details.
The flow chart below shows how structured output is implemented in Spring AI.
Example
Let's see an example of using structured output in Spring AI. User record type shown below represents users in the system.
public record User(String id, String name,
String email, String address, String mobilePhone) {
}
Now we want to use LLM to generate some sample users. The code is very simple. The prompt simply asks the LLM to generate 5 users. When invoking the entity method, new ParameterizedTypeReference<List<User>>() {} means the entity type is List<User>. It can also be simple Java classes for non-generic types.
public List<User> generateUsers() {
return chatClient.prompt().user("Generate 5 users")
.call()
.entity(new ParameterizedTypeReference<List<User>>() {});
}
The actual prompt sent to LLM is shown as below. Spring AI includes some instructions and the JSON schema of List<User> in the prompt.
Generate 5 users
Your response should be in JSON format.
Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
Do not include markdown code blocks in your response.
Remove the ```json markdown from the output.
Here is the JSON Schema instance your output must adhere to:
```{
"$schema" : "https://json-schema.org/draft/2020-12/schema",
"type" : "array",
"items" : {
"type" : "object",
"properties" : {
"address" : {
"type" : "string"
},
"email" : {
"type" : "string"
},
"id" : {
"type" : "string"
},
"mobilePhone" : {
"type" : "string"
},
"name" : {
"type" : "string"
}
},
"additionalProperties" : false
}
}```
Returned JSON content is shown as below. In the code, we can directly use the returned List<User> object.
[
{
"id": "1234567890",
"name": "John Doe",
"email": "user@example.com",
"address": "123 Main St, Anytown, USA",
"mobilePhone": "(555) 123-4567"
},
{
"id": "9876543210",
"name": "Jane Smith",
"email": "user2@example.com",
"address": "456 Elm St, Anytown, USA",
"mobilePhone": "(555) 987-6543"
},
{
"id": "1029876543",
"name": "Bob Johnson",
"email": "user3@example.com",
"address": "789 Oak St, Anytown, USA",
"mobilePhone": "(555) 102-987-6543"
},
{
"id": "3210987654",
"name": "Sara Lee",
"email": "user4@example.com",
"address": "321 Pine St, Anytown, USA",
"mobilePhone": "(555) 321-098-7654"
},
{
"id": "4567890123",
"name": "Mike Brown",
"email": "user5@example.com",
"address": "654 Maple St, Anytown, USA",
"mobilePhone": "(555) 456-789-0123"
}
]
We can control generated data by updating the prompt.