Advertisement

Spring AI: Integrating LLMs into Java Applications

•20 min read•Spring Boot, AI, LLM

The Java ecosystem has officially entered the AI era with the release of the Spring AI project. Gone are the days of hacking together HTTP clients to talk to OpenAI or LangChain wrappers. Spring AI brings the familiar dependency injection, configuration, and abstraction patterns of Spring to the world of Large Language Models.

What is Spring AI?

Spring AI is an application framework for AI engineering. Its goal is to apply Spring ecosystem design principles, such as portability and modular design, to the AI domain and promote using POJOs as the building blocks of an AI application across the AI providers.

Key features include:

  • Portable API: Support for chat, text-to-image, and embedding models across providers (OpenAI, Microsoft, Amazon, Google, Huggingface).
  • Prompt Templates: Powerful template engine for managing prompts.
  • Output Parsers: Map AI responses to POJOs automatically.
  • RAG Support: Built-in document loaders, splitters, and vector store integrations.
  • Function Calling: Delegate tool execution to Spring beans.

Getting Started

To start using Spring AI, you need to add the Spring AI milestone repository and the OpenAI starter (or any other provider you prefer).

<!-- pom.xml -->
<repositories>
    <repository>
        <id>spring-milestones</id>
        <name>Spring Milestones</name>
        <url>https://repo.spring.io/milestone</url>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
</repositories>

<dependencies>
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
        <version>1.0.0-SNAPSHOT</version>
    </dependency>
</dependencies>

Configure your API key in `application.properties`:

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4-turbo

The ChatClient Interface

The `ChatClient` is the core interface. It abstracts the interaction with the LLM. Here is how you can inject and use it in a controller.

@RestController
@RequestMapping("/ai")
public class AssistantController {

    private final ChatClient chatClient;

    public AssistantController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/chat")
    public String chat(@RequestParam String message) {
        return chatClient.prompt()
                .user(message)
                .call()
                .content();
    }

    @GetMapping("/generate-json")
    public MovieRecommendation recommendMovie(@RequestParam String genre) {
         return chatClient.prompt()
                .user(u -> u.text("Recommend a movie in the {genre} genre").param("genre", genre))
                .call()
                .entity(MovieRecommendation.class);
    }
}

record MovieRecommendation(String title, String year, String summary) {}

Notice how we can ask for a POJO response directly using `.entity()`. Spring AI handles the prompt engineering (requesting JSON) and the deserialization for us.

Implementing Retrieval Augmented Generation (RAG)

RAG allows you to ground your AI responses in your own data. Spring AI provides a consistent API for Vector Stores.

@Service
public class DocumentService {

    private final VectorStore vectorStore;

    public DocumentService(VectorStore vectorStore) {
        this.vectorStore = vectorStore;
    }

    public void ingestDocs() {
        // Load documents
        List<Document> documents = new TextReader(new ClassPathResource("policy.txt")).get();

        // Split into chunks
        TokenTextSplitter splitter = new TokenTextSplitter();
        List<Document> chunks = splitter.apply(documents);

        // Save to Vector DB (e.g., PGVector)
        vectorStore.add(chunks);
    }

    public String chatWithDocs(String query) {
        // Retrieve similar documents
        List<Document> similarDocs = vectorStore.similaritySearch(
            SearchRequest.query(query).withTopK(3)
        );

        // Construct prompt with context
        String context = similarDocs.stream()
            .map(Document::getContent)
            .collect(Collectors.joining("\n"));

        // Call ChatClient (implementation omitted for brevity)
        return callChatModel(query, context);
    }
}

Security Considerations

Integrating LLMs into enterprise Java applications introduces new attack vectors. The nondeterministic nature of AI responses combined with the ability to execute actions via function calling requires a defense-in-depth approach.

Spring AI Security Checklist

  • Prompt Injection Defense: Never trust user input directly in prompts. Use delimiters and system instructions to separate instructions from data.
  • Output Sanitization: If the AI generates HTML or SQL, validate and sanitize it before rendering or executing.
  • PII Redaction: Use Spring AOP to intercept requests and redact sensitive info (SSN, emails) before sending to external LLM providers.
  • Function Calling Controls: Apply strict RBAC (Role-Based Access Control) to methods exposed as AI functions. An AI agent should not have admin privileges.
  • Rate Limiting: Implement Resilience4j or Spring Cloud Gateway rate limiters to prevent DoS attacks that target expensive AI endpoints.
  • Audit Logging: Log all prompts and completions (redacted) for compliance and forensic analysis.

Example: PII Redaction Interceptor
You can create a custom interceptor to scan for sensitive data patterns.

@Component
public class PiiRedactionInterceptor implements ClientHttpRequestInterceptor {

    @Override
    public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
        String content = new String(body, StandardCharsets.UTF_8);
        if (containsPii(content)) {
            throw new SecurityException("PII detected in outbound AI request");
        }
        return execution.execute(request, body);
    }

    private boolean containsPii(String content) {
        // Regex checks for credit cards, SSNs, etc.
        return false;
    }
}

Conclusion

Spring AI bridges the gap between traditional enterprise Java development and the cutting-edge world of Generative AI. By providing a consistent abstraction layer, it allows developers to swap models, implement RAG, and build agents without rewriting their entire codebase.

As you adopt these tools, remember that AI is not just another API—it requires careful thought regarding ethics, security, and user experience. Start small, secure your inputs, and leverage the power of the Spring ecosystem to build robust, scalable AI applications.

Ready to Build AI Apps?

Check out our other guides on building RAG systems and securing AI agents.

View All Guides

Written by the DevMetrix Team • Published December 8, 2025

Advertisement