Jonathan Lalou's Blog

Posts Tagged ‘SpringIO2024’

At Spring I/O 2024, Marcos Maia and Lars Werkman from Bitvavo, Europe’s leading cryptocurrency exchange, unveiled the architectural intricacies of their high-performance trading platform. Based in the Netherlands, Bitvavo processes thousands of transactions per second with sub-millisecond latency. Marcos and Lars detailed how they integrate ultra-low-latency systems with Spring Boot applications, offering a deep dive into their strategies for scalability and performance. Their talk, rich with technical insights, challenged conventional software practices, urging developers to rethink performance optimization.

Architecting for Ultra-Low Latency

Marcos opened by highlighting Bitvavo’s mission to enable seamless crypto trading for nearly two million customers. The exchange’s hot path, where orders are processed, demands microsecond response times. To achieve this, Bitvavo employs the Aeron framework, an open-source tool designed for high-performance messaging. By using memory-mapped files, UDP-based communication, and lock-free algorithms, the platform minimizes latency. Marcos explained how they bypass traditional databases, opting for in-memory processing with eventual disk synchronization, ensuring deterministic outcomes critical for trading fairness.

Optimizing the Hot Path

The hot path’s design is uncompromising, as Marcos elaborated. Bitvavo avoids garbage collection by preallocating and reusing objects, ensuring predictable memory usage. Single-threaded processing, counterintuitive to many, leverages CPU caches for nanosecond-level performance. The platform uses distributed state machines, guaranteeing consistent outputs across executions. Lars complemented this by discussing inter-process communication via shared memory and DPDK for kernel-bypassing network operations. These techniques, rooted in decades of trading system expertise, enable Bitvavo to handle peak loads of 30,000 transactions per second.

Bridging with Spring Boot

Integrating high-performance systems with the broader organization poses significant challenges. Marcos detailed the “cold sink,” a Spring Boot application that consumes data from the hot path’s Aeron archive, feeding it into Kafka and MySQL for downstream processing. By batching requests and using object pools, the cold sink minimizes garbage collection, maintaining performance under heavy loads. Fine-tuning batch sizes and applying backpressure ensure the system keeps pace with the hot path’s output, preventing data lags in Bitvavo’s 24/7 operations.

Enhancing JWT Signing Performance

Lars concluded with a case study on optimizing JWT token signing, a “warm path” process targeting sub-millisecond latency. Initially, their RSA-based signing took 8.8 milliseconds, far from the goal. By switching to symmetric HMAC signing and adopting Azul Prime’s JVM, they achieved a 30x performance boost, reaching 260-280 microsecond response times. Lars emphasized the importance of benchmarking with JMH and leveraging Azul’s features like Falcon JIT compiler for stable throughput. This optimization underscores Bitvavo’s commitment to performance across all system layers.

Links:

• Bitvavo company website
• Bitvavo on LinkedIn

At Spring I/O 2024 in Barcelona, Jatin Makhija, an engineering leader at Deutsche Telekom Digital Labs, delivered an insightful presentation on leveraging Spring Boot Admin to enhance application monitoring and management. With a rich background in startups like Exigo and VWO, Jatin shared practical use cases and live demonstrations, illustrating how Spring Boot Admin empowers developers to streamline operations in complex, distributed systems. This talk, filled with actionable insights, highlighted the tool’s versatility in addressing real-world challenges, from log management to feature flag automation.

Empowering Log Management

Jatin began by addressing a universal pain point for developers: debugging production issues. He emphasized the critical role of logs in resolving incidents, noting that Spring Boot Admin allows engineers to dynamically adjust log levels—from info to trace—in seconds without redeploying applications. Through a live demo, Jatin showcased how to filter logs at the class level, enabling precise debugging. However, he cautioned about the costs of excessive logging, both in infrastructure and compliance with GDPR. By masking personally identifiable information (PII) and reverting log levels promptly, teams can maintain security and efficiency. This capability ensures rapid issue resolution while keeping customers satisfied, as Jatin illustrated with real-time log adjustments.

Streamlining Feature Flags

Feature flags are indispensable in modern applications, particularly in multi-tenant environments. Jatin explored how Spring Boot Admin simplifies their management, allowing teams to toggle features without redeploying. He presented two compelling use cases: a legacy discount system and a mobile exchange program. In the latter, Jatin demonstrated dynamically switching locales (e.g., from German to English) to adapt third-party integrations, ensuring seamless user experiences across regions. By refreshing application contexts on the fly, Spring Boot Admin reduces downtime and enhances testing coverage. Jatin’s approach empowers product owners to experiment confidently, minimizing technical debt and ensuring robust feature validation.

Automating Operations

Automation is a cornerstone of efficient development, and Jatin showcased how Spring Boot Admin’s REST APIs can be harnessed to automate testing workflows. By integrating with CI/CD pipelines like Jenkins and test frameworks such as Selenium, teams can dynamically patch configurations and validate multi-tenant setups. A recorded demo illustrated an automated test toggling a mobile exchange feature, highlighting increased test coverage and early defect detection. Jatin emphasized that this automation reduces manual effort, boosts regression testing accuracy, and enables scalable deployments, allowing teams to ship with confidence.

Scaling Monitoring and Diagnostics

Monitoring distributed systems is complex, but Spring Boot Admin simplifies it with built-in metrics and diagnostics. Jatin demonstrated accessing health statuses, thread dumps, and heap dumps through the tool’s intuitive interface. He shared a story of debugging a Kubernetes pod misconfiguration, where Spring Boot Admin revealed discrepancies in CPU allocation, preventing application instability. By integrating the Git Commit Plugin, teams can track deployment details like commit IDs and timestamps, enhancing traceability in microservices. Jatin also addressed scalability, showcasing a deployment managing 374 instances across 24 applications, proving Spring Boot Admin’s robustness in large-scale environments.

Links:

• Deutsche Telekom Digital Labs on LinkedIn

In a dynamic session at Spring I/O 2024 in Barcelona, Dan Erez, lead architect at ZIM, a global shipping company, championed Spring Cloud Gateway as a versatile and powerful API gateway solution. Drawing on his extensive experience with Java and Spring, Dan demonstrated how Spring Cloud Gateway addresses the complexities of microservices architectures, offering features like dynamic routing, rate limiting, and circuit breakers. His talk underscored the gateway’s customizability and integration with Spring, making it an ideal choice for organizations seeking to streamline API management.

The Role of API Gateways in Microservices

API gateways simplify microservices architectures by acting as a unified entry point, hiding the complexity of multiple services. Dan traced the evolution from monolithic applications to microservices, noting that while microservices enhance modularity, they introduce challenges like service discovery and load balancing. An API gateway mitigates these by routing requests, authenticating users, and managing traffic. Spring Cloud Gateway excels in these areas, supporting functionalities like caching, rate limiting, and authentication with JWT tokens. Dan highlighted its ability to integrate with service discovery tools like Eureka, ensuring seamless communication across services.

Configuring and Customizing Routes

Spring Cloud Gateway’s flexibility shines in its routing capabilities. Dan showcased two configuration approaches: declarative YAML files and code-based Java configurations. Routes can be defined based on predicates like paths, headers, or query parameters, allowing fine-grained control. For example, a route for “/orders” can direct to an internal service, while “/invoices” targets a third-party API. Dan demonstrated a simple route adding a request header, executed via WebFlux for high concurrency. This customizability, paired with Spring’s familiarity, empowers developers to tailor the gateway to specific needs without relying on external DevOps teams.

Enhancing Resilience with Circuit Breakers and Caching

Resilience is critical in distributed systems, and Spring Cloud Gateway offers robust tools to handle failures. Dan illustrated circuit breakers, which redirect requests to fallback endpoints when a service is unavailable, preventing timeouts. In a demo, a delayed service triggered a fallback response, showcasing how developers can return cached data or custom errors. Caching, implemented with libraries like Caffeine, reduces database load by storing frequent responses. Dan’s example showed a cached HTTP response remaining consistent across refreshes, highlighting performance gains. These features ensure reliable and efficient API interactions.

Dynamic Rate Limiting and Cost Savings

Dan emphasized Spring Cloud Gateway’s ability to implement dynamic rate limiting, a feature that adapts to server load. In his demo, a custom filter adjusted the maximum request size based on simulated load, allowing larger requests during low traffic and restricting them during peaks. This approach maintains service availability without costly infrastructure scaling. Dan also shared a real-world use case at ZIM, where Spring Cloud Gateway enabled developers to run local services within a shared environment, routing requests dynamically to avoid conflicts. This solution saved significant costs by reducing the need for individual cloud environments.

Links:

• Slides on Spring I/O Website
• Spring Cloud Gateway Documentation

At Spring I/O 2024 in Barcelona, Victor Martin, a product manager for Oracle Database, delivered a compelling session on Text-to-SQL, a transformative approach to querying databases using natural language, powered by generative AI. Stepping in for his colleague Corrado De Bari, who was unable to attend, Victor explored how Large Language Models (LLMs) and Oracle’s innovative tools, including Spring AI and Select AI, enable business users with no SQL expertise to interact seamlessly with databases. The talk highlighted practical implementations, security considerations, and emerging technologies like AI Vector Search, offering a glimpse into the future of database interaction.

The Promise of Text-to-SQL

Text-to-SQL leverages LLMs to translate natural language queries into executable SQL, democratizing data access for non-technical users. Victor began by posing a challenge: how long would it take to build a REST endpoint for a business user to query a database using plain text? Traditionally, this task required manual SQL construction, schema validation, and error handling. With modern frameworks like Spring Boot and Oracle’s Select AI, this process is streamlined. Select AI, integrated into Oracle Database 19c and enhanced in 23 AI, supports features like RUN_SQL to execute generated queries, NARRATE to return results as human-readable text, and EXPLAIN_SQL to detail query reasoning. Victor emphasized that these tools reduce development time, enabling rapid deployment of user-friendly database interfaces.

Configuring Oracle Database for Text-to-SQL

Implementing Text-to-SQL requires minimal configuration within Oracle Database. Victor outlined the steps: first, set up an Access Control List (ACL) to allow external LLM calls, specifying the host and port. Next, create credentials for the LLM service (e.g., Oracle Cloud Infrastructure Generative AI, Open AI, or Azure Open AI) using the DBMS_CLOUD_AI package. Finally, define a profile linking the schema, tables, and chosen LLM. This profile is selected per session to ensure queries use the correct context. Victor demonstrated this with a Spring Boot application, where the profile is set before invoking Select AI. The simplicity of this setup, combined with Spring AI’s abstraction, makes it accessible even for developers new to AI-driven database interactions.

Enhancing Queries with Schema Annotations

A key challenge in Text-to-SQL is ensuring LLMs interpret ambiguous schemas correctly. Victor highlighted that table and column names like “C1” or “Table1” can confuse models. To address this, Oracle Database supports annotations—comments on tables and columns that provide business context. For example, annotating a column as “process status” with possible values clarifies its purpose, aiding the LLM in generating accurate joins and filters. These annotations, which don’t affect production queries, are created collaboratively by DBAs and business stakeholders. Victor shared a real-world example from Oracle’s telecom applications, where annotated schemas improved query precision, enabling complex queries without manual intervention.

AI Vector Search: Querying Unstructured Data

Victor introduced AI Vector Search, a cutting-edge feature in Oracle Database 23 AI, which extends Text-to-SQL to unstructured data. Unlike traditional SQL, which queries structured data, vector search encodes text, images, or audio into high-dimensional vectors representing semantic meaning. These vectors, stored as a new VECTOR data type, enable similarity-based queries. For instance, a job search query for “software engineer positions in New York” can combine structured filters (e.g., location) with vector-based matching of job descriptions and resumes. Victor explained how embedding models, deployed via Oracle’s DBMS_DATA_MINING package, generate these vectors, with metrics like cosine similarity determining relevance. This capability opens new use cases, from document search to personalized recommendations.

Links:

• Oracle Select AI Documentation
• Spider Text-to-SQL Benchmark
• Spring AI Documentation
• Oracle Backend for Spring Applications

Serverless computing has transformed application development by abstracting infrastructure management, offering fine-grained scaling, and billing only for execution time. At Spring I/O 2024 in Barcelona, Maximilian Schellhorn and Dennis Kieselhorst, AWS Solutions Architects, shared their expertise on building serverless Java applications with Spring. Their session explored running existing Spring Boot applications in serverless environments and developing event-driven applications using Spring Cloud Function, with a focus on performance optimizations and practical tooling.

The Serverless Paradigm

Maximilian began by contrasting traditional containerized applications with serverless architectures. Containers, while resource-efficient, require developers to manage orchestration, networking, and scaling. Serverless computing, exemplified by AWS Lambda, eliminates these responsibilities, allowing developers to focus on code. Maximilian highlighted four key promises: reduced operational overhead, automatic granular scaling, pay-per-use billing, and high availability. Unlike containers, which remain active and incur costs even when idle, serverless functions scale to zero, executing only in response to events like API requests or queue messages, optimizing cost and resource utilization.

Spring Cloud Function for Event-Driven Development

Spring Cloud Function simplifies serverless development by enabling developers to write event-driven applications as Java functions. Maximilian demonstrated how it leverages Spring Boot’s familiar features—autoconfiguration, dependency injection, and testing—while abstracting cloud-specific details. Functions receive event payloads (e.g., JSON from API Gateway or Kafka) and can convert them into POJOs, streamlining business logic implementation. The framework’s generic invoker supports function routing, allowing multiple functions within a single codebase, and enables local testing via HTTP endpoints. This portability ensures applications can target various serverless platforms without vendor lock-in, enhancing flexibility.

Adapting Existing Spring Applications

For teams with existing Spring Boot applications, Dennis introduced the AWS Serverless Java Container, an open-source library acting as an adapter to translate serverless events into Java Servlet requests. This allows REST controllers to function unchanged in a serverless environment. Version 2.0.2, released during the conference, supports Spring Boot 3 and integrates with Spring Cloud Function. Dennis emphasized its ease of use: add the library, configure a handler, and deploy. While this approach incurs some overhead compared to native functions, it enables rapid migration of legacy applications, preserving existing investments without requiring extensive rewrites.

Optimizing Performance with SnapStart and GraalVM

Performance, particularly cold start times, is a critical concern in serverless Java applications. Dennis addressed this by detailing AWS Lambda SnapStart, which snapshots the initialized JVM and micro-VM, reducing startup times by up to 80% without additional costs. SnapStart, integrated with Spring Boot 3.2’s CRaC (Coordinated Restore at Checkpoint) support, manages initialization hooks to handle resources like database connections. For further optimization, Maximilian discussed GraalVM native images, which compile Java code into binaries for faster startups and lower memory usage. However, GraalVM’s complexity and framework limitations make SnapStart the preferred starting point, with GraalVM reserved for extreme performance needs.

Practical Considerations and Tooling

Maximilian and Dennis stressed practical considerations, such as database connection management and observability. Serverless scaling can overwhelm traditional databases, necessitating connection pooling adjustments or proxies like AWS RDS Proxy. Observability in Lambda relies on a push model, integrating with tools like CloudWatch, X-Ray, or OpenTelemetry, though additional layers may impact performance. To aid adoption, they offered a Lambda Workshop and a Serverless Java Replatforming Guide, providing hands-on learning and written guidance. These resources, accessible via AWS accounts, empower developers to experiment and apply serverless principles effectively.

Links:

• Slides on SpeakerDeck
• AWS Serverless Java Container
• Lambda Workshop
• Serverless Java Replatforming Guide

At Spring I/O 2024 in Barcelona, Balkrishna Rawool, a software engineer at ING Bank, captivated attendees with an in-depth exploration of continuations, the underlying mechanism powering Java’s virtual threads. Introduced as a final feature in Java 21 under Project Loom, virtual threads promise unprecedented scalability for Java applications. Balkrishna’s session demystified how continuations enable this scalability by allowing programs to pause and resume execution, offering a deep dive into their mechanics and practical applications.

Understanding Virtual Threads

Virtual threads, a cornerstone of Project Loom, are lightweight user threads designed to enhance scalability in Java applications. Unlike platform threads, which map directly to operating system threads and are resource-intensive, virtual threads require minimal memory, enabling developers to create millions without significant overhead. Balkrishna illustrated this by comparing platform threads, often pooled due to their cost, to virtual threads, which are created and discarded as needed, avoiding pooling anti-patterns. He emphasized that virtual threads rely on platform threads—termed carrier threads—for execution, with a scheduler mounting and unmounting them dynamically. This mechanism ensures efficient CPU utilization, particularly in I/O-bound applications where threads spend considerable time waiting, thus boosting scalability.

The Power of Continuations

Continuations, the core focus of Balkrishna’s talk, are objects that represent a program’s current state or the “rest” of its computation. They allow developers to pause a program’s execution and resume it later, a capability critical to virtual threads’ efficiency. Using Java’s Continuation API, Balkrishna demonstrated how continuations pause execution via the yield method, transferring control back to the caller, and resume via the run method. He showcased this with a simple example where a continuation printed values, paused at specific points, and resumed, highlighting the manipulation of the call stack to achieve this control transfer. Although the Continuation API is not intended for direct application use, understanding it provides insight into virtual threads’ behavior and scalability.

Building a Generator with Continuations

To illustrate continuations’ versatility, Balkrishna implemented a generator—a data structure yielding values lazily—using only the Continuation API, eschewing Java’s streams or iterators. Generators are ideal for resource-intensive computations, producing values only when needed. In his demo, Balkrishna created a generator yielding strings (“a,” “b,” “c”) by defining a Source object to handle value yields and pauses via continuations. The generator paused after each yield, allowing consumers to iterate over values in a loop, demonstrating how continuations enable flexible control flow beyond virtual threads, applicable to constructs like coroutines or exception handling.

Crafting a Simple Virtual Thread

In the session’s climax, Balkrishna guided attendees through implementing a simplified virtual thread class using continuations. The custom virtual thread paused execution during blocking operations, freeing platform threads, and supported a many-to-many relationship with carrier threads. He introduced a scheduler to manage virtual threads on a fixed pool of platform threads, using a queue for first-in-first-out scheduling. A demo with thousands of virtual threads, each simulating blocking calls, outperformed an equivalent platform-thread implementation, underscoring virtual threads’ scalability. By leveraging scoped values and timers, Balkrishna ensured accurate thread identification and resumption, providing a clear, hands-on understanding of virtual threads’ mechanics.

Links:

• Balkrishna Rawool’s GitHub Repository
• Slides on SpeakerDeck