Building a Production-Ready Agent with Spring AI

An agent that works in a demo can still fail in production. A tool may time out, the model may return an invalid argument, or a retry may repeat an operation that has already changed data. Production readiness comes from making these boundaries explicit.

This article builds on the agent concepts and task execution pattern already covered on this site. The goal is a bounded assistant that can look up an order and, when allowed, request a cancellation. The same principles apply to other Spring AI agents.

Start with a narrow job and explicit tools

Give the agent a small set of capabilities with clear descriptions. A tool description is part of the model's decision-making context, but it is not an authorization mechanism. The application must still check whether the current user can act on the requested resource.

For example, keep lookup separate from a state-changing request:

@Component
class OrderTools {
    private final OrderService orders;
    private final CurrentUser currentUser;

    OrderTools(OrderService orders, CurrentUser currentUser) {
        this.orders = orders;
        this.currentUser = currentUser;
    }

    @Tool(description = "Look up an order that belongs to the signed-in user")
    OrderSummary getOrder(String orderId) {
        return orders.findForUser(orderId, currentUser.id());
    }

    @Tool(description = "Request cancellation of an eligible order after user confirmation")
    CancellationRequest requestCancellation(String orderId, String requestKey) {
        return orders.requestCancellation(
            orderId, currentUser.id(), requestKey);
    }
}

The exact @Tool package and imports depend on the Spring AI version in use. In Spring AI 2.0, use the current Spring AI tool-calling APIs; see the 2.0 migration notes if upgrading from 1.x.

Notice what the tools do not accept: a user ID supplied by the model. Identity comes from the authenticated request context. The service checks ownership and eligibility, and the cancellation request carries an idempotency key so a repeated call does not create duplicate work.

Put confirmation and authorization in the application

Do not let a model's interpretation of a prompt authorize a sensitive operation. For a consequential action, use a separate confirmation step in the product flow:

1. The agent explains what it proposes to do.
2. The user confirms through an application-controlled UI or API action.
3. The backend rechecks identity, ownership, and current order state.
4. The backend records an idempotent cancellation request.

The confirmation should be tied to the exact order and action. If the order state changes between proposal and confirmation, reject the stale request and ask the user to review the new state.

This is a practical form of human-in-the-loop control. The model can help interpret intent, while application code decides whether an operation is permitted.

Bound every model and tool call

Agent loops can multiply latency and cost. Set limits at each boundary:

• A request deadline for the whole interaction.
• A timeout for each downstream tool call.
• A maximum number of tool calls or agent steps.
• Input size and output token limits.
• A bounded retry policy for transient failures only.

Retries need care. Retrying a read is often safe. Retrying a state-changing request is safe only when the operation is idempotent or the service can determine whether the earlier attempt succeeded. Do not retry validation failures, permission denials, or malformed arguments as if they were transient network errors.

At the API boundary, return a controlled response when a budget is exhausted. Avoid returning raw provider errors, stack traces, or internal tool output to the user.

Validate tool inputs and outputs

Structured arguments reduce ambiguity, but they do not replace validation. Check identifiers, lengths, enum values, numeric ranges, and business rules inside the tool implementation. Validate tool results before feeding them back into a prompt, especially if results contain user-generated text or content from external systems.

Treat retrieved text and tool output as data, not instructions. A customer note that says “ignore previous instructions” must not change the tool policy. Keep system instructions separate from untrusted content, and limit the fields exposed to the model to what it needs for the task.

Handle failures as normal outcomes

Tools should distinguish expected failures from infrastructure failures. For instance, “order is not eligible for cancellation” is a business result; a database timeout is an operational failure. The agent should receive a concise, typed outcome when it can safely continue. Infrastructure details belong in logs and traces, not the prompt or user response.

Choose a fallback for each failure:

Failure	Application response
Invalid tool arguments	Reject the call and return a validation result
User lacks access	Deny the action without disclosing another user's data
Business rule prevents action	Explain the rule and offer a safe next step
Temporary dependency failure	Apply a bounded retry or return a retryable error
Model or provider unavailable	Return a graceful unavailable response
Step or time budget reached	Stop the loop and summarize what remains undone

The application, rather than the model, should decide whether another attempt is allowed.

Add observability without logging secrets

Record enough to reconstruct an interaction: request or trace ID, selected model, latency, token usage when available, tool name, tool duration, outcome category, retry count, and final status. Avoid recording API keys, credentials, full personal data, or unrestricted prompts and tool payloads by default. Apply the same retention and access rules as the rest of the application.

Use traces to answer operational questions:

• Which tool is responsible for most of the latency?
• How often does the model select an invalid or unavailable action?
• How many interactions stop at the step limit?
• Which failure categories are increasing after a deployment?

The site's Langfuse and RustFS article is a useful starting point for tracing. Keep metrics low-cardinality: use tool names and outcome categories, not user IDs or raw prompts as metric labels.

Test the boundaries

Test the agent as an application workflow, not only as a prompt. Cover at least:

• The happy path for looking up an order.
• A user asking for another user's order.
• A cancellation request without confirmation.
• Confirmation after the order becomes ineligible.
• A timeout, malformed tool arguments, and provider failure.
• Duplicate cancellation requests with the same idempotency key.
• Prompt-injection text in an order note or tool result.

Use deterministic service tests for authorization and business rules. For model behavior, maintain a small evaluation set of representative inputs and expected tool choices or response properties. Model outputs can vary, so assert important behaviors and safety properties rather than exact wording.

A practical readiness checklist

Before exposing an agent to real users, verify that:

• Its job and available tools are narrowly scoped.
• Authorization is enforced by backend services for every resource and action.
• Sensitive operations require application-controlled confirmation.
• Calls, retries, and total execution time have limits.
• State-changing operations are idempotent or otherwise protected from duplicate execution.
• Tool inputs and outputs are validated, and external content is treated as untrusted.
• Failures have typed, user-safe outcomes.
• Traces and metrics support diagnosis without collecting unnecessary sensitive data.
• Tests cover denied access, stale state, timeouts, and duplicate requests.

An agent becomes production-ready through these ordinary engineering controls around the model. Start with one useful workflow, make every external action explicit, and expand its capabilities only when the boundaries are observable and tested.