Posts Tagged ‘VirtualThreads’
[DevoxxBE2025] Virtual Threads, Structured Concurrency, and Scoped Values: Putting It All Together
Lecturer
Balkrishna Rawool leads IT chapters at ING Bank, focusing on scalable software solutions and Java concurrency. He actively shares insights on Project Loom through conferences and writings, drawing from practical implementations in financial systems.
Abstract
This review dissects Project Loom’s enhancements to Java’s concurrency: virtual threads for efficient multitasking, structured concurrency for task orchestration, and scoped values for secure data sharing. Placed in web development contexts, it explains their interfaces and combined usage via a Spring Boot loan processing app. The evaluation covers integration techniques, traditional threading issues, and effects on legibility, expandability, and upkeep in parallel code.
Project Loom Foundations and Virtual Threads
Project Loom overhauls Java concurrency with lightweight alternatives to OS-bound threads, which limit scale due to overheads. Virtual threads, managed by the JVM, enable vast concurrency on few carriers, ideal for IO-heavy web services.
In the loan app—computing offers via credit, account, and loan calls—virtual threads parallelize without resource strain. Configuring Tomcat to use them boosts TPS from hundreds to thousands, as non-blocking calls unmount threads.
The interface mirrors traditional: Thread.ofVirtual().start(task). Internals use continuations for suspension, allowing carrier reuse. Consequences: lower memory, natural exception flow.
Care needed for pinning: synchronized blocks block carriers; ReentrantLocks avoid this, sustaining performance.
Structured Concurrency for Unified Task Control
Structured concurrency organizes subtasks as cohesive units, addressing executors’ scattering. StructuredTaskScope scopes forks, ensuring completion before progression.
In the app, scoping credit/account/loan forks with ShutdownOnFailure cancels on errors, avoiding leaks. Example:
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
var credit = scope.fork(() -> getCredit(request));
var account = scope.fork(() -> getAccount(request));
var loan = scope.fork(() -> calculateLoan(request));
scope.join();
// Aggregate
} catch (Exception e) {
// Manage
}
This ensures orderly shutdowns, contrasting unstructured daemons. Effects: simpler debugging, no dangling tasks.
Scoped Values for Immutable Inheritance
Scoped values supplant ThreadLocals for virtual threads, binding data immutably in scopes. ThreadLocals mutate, risking inconsistencies; scoped values inherit safely.
For request IDs in logs: ScopedValue.where(ID, uuid).run(() -> tasks); IDs propagate to forks via scopes.
Example:
ScopedValue.where(REQ_ID, UUID.randomUUID()).run(() -> {
// Forks access ID
});
This solves ThreadLocal inefficiencies in Loom. Effects: secure sharing in hierarchies.
Combined Usage and Prospects
Synergies yield maintainable concurrency: virtual threads scale, scopes structure, values share. The app processes concurrently yet organized, IDs tracing.
Effects: higher IO throughput, easier upkeep. Prospects: framework integrations reshaping concurrency.
In overview, Loom’s features enable efficient, readable parallel systems.
Links:
- Lecture video: https://www.youtube.com/watch?v=iO79VR0zAhQ
- Balkrishna Rawool on LinkedIn: https://nl.linkedin.com/in/balkrishnarawool
- Balkrishna Rawool on Twitter/X: https://twitter.com/BalaRawool
- ING Bank website: https://www.ing.com/
[SpringIO2025] Spring I/O 2025 Keynote
Lecturer
The keynote features Spring leadership: Juergen Hoeller (Framework Lead), Rossen Stoyanchev (Web), Ana Maria Mihalceanu (AI), Moritz Halbritter (Boot), Mark Paluch (Data), Josh Long (Advocate), Mark Pollack (Messaging). Collectively, they steer the Spring portfolio’s technical direction and community engagement.
Abstract
The keynote unveils Spring Framework 7.0 and Boot 4.0, establishing JDK 21 and Jakarta EE 11 as baselines while advancing AOT compilation, virtual threads, structured concurrency, and AI integration. Live demonstrations and roadmap disclosures illustrate how these enhancements—combined with refined observability, web capabilities, and data access—position Spring as the preeminent platform for cloud-native Java development.
Baseline Evolution: JDK 21 and Jakarta EE 11
Spring Framework 7.0 mandates JDK 21, embracing virtual threads for lightweight concurrency and records for immutable data carriers. Jakarta EE 11 introduces the Core Profile and CDI Lite, trimming enterprise bloat. The demonstration showcases a virtual thread-per-request web handler processing 100,000 concurrent connections with minimal heap, contrasting traditional thread pools. This baseline shift enables native image compilation via Spring AOT, reducing startup to milliseconds and memory footprint by 90%.
AOT and Native Image Optimization
Spring Boot 4.0 refines AOT processing through Project Leyden integration, pre-computing bean definitions and proxy classes at build time. Native executables startup in under 50ms, suitable for serverless platforms. The live demo compiles a Kafka Streams application to GraalVM native image, achieving sub-second cold starts and 15MB RSS—transforming deployment economics for event-driven microservices.
AI Integration and Modern Web Capabilities
Spring AI matures with function calling, tool integration, and vector database support. A live-coded agent retrieves beans from a running context to answer natural language queries about application metrics. WebFlux enhances structured concurrency with Schedulers.boundedElastic() replacement via virtual threads, simplifying reactive code. The demonstration contrasts traditional Mono/Flux composition with straightforward sequential logic executing on virtual threads, preserving backpressure while improving readability.
Data, Messaging, and Observability Advancements
Spring Data advances R2DBC connection pooling and Redis Cluster native support. Spring for Apache Kafka 4.0 introduces configurable retry templates and Micrometer metrics out-of-the-box. Unified observability aggregates metrics, traces, and logs: Prometheus exposes 200+ Kafka client metrics, OpenTelemetry correlates spans across HTTP and Kafka, and structured logging propagates MDC context. A Grafana dashboard visualizes end-to-end latency from REST ingress to database commit, enabling proactive incident response.
Community and Future Trajectory
The keynote celebrates Spring’s global community, highlighting contributions to null-safety (JSpecify), virtual thread testing, and AOT hint generation. Planned enhancements include JDK 23 support, Project Panama integration for native memory access, and AI-driven configuration validation. The vision positions Spring as the substrate for the next decade of Java innovation, balancing cutting-edge capabilities with backward compatibility.
Links:
[DevoxxBE2024] The Next Phase of Project Loom and Virtual Threads by Alan Bateman
At Devoxx Belgium 2024, Alan Bateman delivered a comprehensive session on the advancements in Project Loom, focusing on virtual threads and their impact on Java concurrency. As a key contributor to OpenJDK, Alan explored how virtual threads enable high-scale server applications with a thread-per-task model, addressing challenges like pinning, enhancing serviceability, and introducing structured concurrency. His talk provided practical insights into leveraging virtual threads for simpler, more scalable code, while detailing ongoing improvements in JDK 24 and beyond.
Understanding Virtual Threads and Project Loom
Project Loom, a transformative initiative in OpenJDK, aims to enhance concurrency in Java by introducing virtual threads—lightweight, user-mode threads that support a thread-per-task model. Unlike traditional platform threads, which are resource-intensive and often pooled, virtual threads are cheap, allowing millions to run within a single JVM. Alan emphasized that virtual threads enable developers to write simple, synchronous, blocking code that is easy to read and debug, avoiding the complexity of reactive or asynchronous models. Finalized in JDK 21 after two preview releases, virtual threads have been widely adopted by frameworks like Spring and Quarkus, with performance and reliability proving robust, though challenges like pinning remain.
The Pinning Problem and Its Resolution
A significant pain point with virtual threads is “pinning,” where a virtual thread cannot unmount from its carrier thread during blocking operations within synchronized methods or blocks, hindering scalability. Alan detailed three scenarios causing pinning: blocking inside synchronized methods, contention on synchronized methods, and object wait/notify operations. These can lead to scalability issues or even deadlocks if all carrier threads are pinned. JEP 444 acknowledged this as a quality-of-implementation issue, not a flaw in the synchronized keyword itself. JEP 491, currently in Early Access for JDK 24, addresses this by allowing carrier threads to be released during such operations, eliminating the need to rewrite code to use java.util.concurrent.locks.ReentrantLock. Alan urged developers to test these Early Access builds to validate reliability and performance, noting successful feedback from initial adopters.
Enhancing Serviceability for Virtual Threads
With millions of virtual threads in production, diagnosing issues is critical. Alan highlighted improvements in serviceability tools, such as thread dumps that now distinguish carrier threads and include stack traces for mounted virtual threads in JDK 24. A new JSON-based thread dump format, introduced with virtual threads, supports parsing for visualization and preserves thread groupings, aiding debugging of complex applications. For pinning, JFR (Java Flight Recorder) events now capture stack traces when blocking occurs in synchronized methods, with expanded support for FFM and JNI in JDK 24. Heap dumps in JDK 23 include unmounted virtual thread stacks, and new JMX-based monitoring interfaces allow dynamic inspection of the virtual thread scheduler, enabling fine-tuned control over parallelism.
Structured Concurrency: Simplifying Concurrent Programming
Structured concurrency, a preview feature in JDK 21–23, addresses the complexity of managing concurrent tasks. Alan presented a motivating example of aggregating data from a web service and a database, comparing sequential and concurrent approaches using thread pools. Traditional thread pools with Future.get() can lead to leaks or wasted cycles if tasks fail, requiring complex cancellation logic. The StructuredTaskScope API simplifies this by ensuring all subtasks complete before the main task proceeds, using a single join method to wait for results. If a subtask fails, others are canceled, preventing leaks and preserving task relationships in a tree-like structure. An improved API in Loom Early Access builds, planned for JDK 24 preview, introduces static factory methods and streamlined exception handling, making structured concurrency a powerful complement to virtual threads.
Future Directions and Community Engagement
Alan outlined Project Loom’s roadmap, focusing on JEP 491 for pinning resolution, enhanced diagnostics, and structured concurrency’s evolution. He emphasized that virtual threads are not a performance boost for individual methods but excel in scalability through sheer numbers. Misconceptions, like replacing all platform threads with virtual threads or pooling them, were debunked, urging developers to focus on task migration. Structured concurrency’s simplicity aligns with virtual threads’ lightweight nature, promising easier debugging and maintenance. Alan encouraged feedback on Early Access builds for JEP 491 and structured concurrency (JEP 480), highlighting their importance for production reliability. Links to JEP 444, JEP 491, and JEP 480 provide further details for developers eager to explore.
Links:
[DevoxxGR2024] Butcher Virtual Threads Like a Pro at Devoxx Greece 2024 by Piotr Przybyl
Piotr Przybyl, a Java Champion and developer advocate at Elastic, captivated audiences at Devoxx Greece 2024 with a dynamic exploration of Java 21’s virtual threads. Through vivid analogies, practical demos, and a touch of humor, Piotr demystified virtual threads, highlighting their potential and pitfalls. His talk, rich with real-world insights, offered developers a guide to leveraging this transformative feature while avoiding common missteps. As a seasoned advocate for technologies like Elasticsearch and Testcontainers, Piotr’s presentation was a masterclass in navigating modern Java concurrency.
Understanding Virtual Threads
Piotr began by contextualizing virtual threads within Java’s concurrency evolution. Introduced in Java 21 under Project Loom, virtual threads address the limitations of traditional platform threads, which are costly to create and limited in number. Unlike platform threads, virtual threads are lightweight, managed by a scheduler that mounts and unmounts them from carrier threads during I/O operations. This enables a thread-per-request model, scaling applications to handle millions of concurrent tasks. Piotr likened virtual threads to taxis in a busy city like Athens, efficiently transporting passengers (tasks) without occupying resources during idle periods.
However, virtual threads are not a universal solution. Piotr emphasized that they do not inherently speed up individual requests but improve scalability by handling more concurrent tasks. Their API remains familiar, aligning with existing thread practices, making adoption seamless for developers accustomed to Java’s threading model.
Common Pitfalls and Pinning
A central theme of Piotr’s talk was “pinning,” a performance issue where virtual threads remain tied to carrier threads, negating benefits. Pinning occurs during I/O or native calls within synchronized blocks, akin to keeping a taxi running during a lunch break. Piotr demonstrated this with a legacy Elasticsearch client, using Testcontainers and Toxiproxy to simulate slow network calls. By enabling tracing with flags like -J-DTracePinnThreads, He identified and resolved pinning issues, replacing synchronized methods with modern, non-blocking clients.
Piotr cautioned against misuses like thread pooling or reusing virtual threads, which disrupt their lightweight design. He advocated for careful monitoring using JFR events to ensure threads remain unpinned, ensuring optimal performance in production environments.
Structured Concurrency and Scope Values
Piotr explored structured concurrency, a preview feature in Java 21, designed to eliminate thread leaks and cancellation delays. By creating scopes that manage forks, developers can ensure tasks complete or fail together, simplifying error handling. He demonstrated a shutdown-on-failure scope, where a single task failure cancels all others, contrasting this with the complexity of managing interdependent futures.
Scope Values, another preview feature, offer immutable, one-way thread locals to prevent bugs like data leakage in thread pools. Piotr illustrated their use in maintaining request context, warning against mutability to preserve reliability. These features, he argued, complement virtual threads, fostering robust, maintainable concurrent applications.
Practical Debugging and Best Practices
Through live coding, Piotr showcased how debugging with logging can inadvertently introduce I/O, unmounting virtual threads and degrading performance. He compared this to a concert where logging scatters tasks, reducing completion rates. To mitigate this, he recommended avoiding I/O in critical paths and using structured concurrency for monitoring.
Piotr’s best practices included using framework-specific annotations (e.g., Quarkus, Spring) to enable virtual threads and ensuring tasks are interruptible. He urged developers to test thoroughly, leveraging tools like Testcontainers to simulate real-world conditions. His blog post on testing unpinned threads provides further guidance for practitioners.
Conclusion
Piotr’s presentation was a clarion call to embrace virtual threads with enthusiasm and caution. By understanding their mechanics, avoiding pitfalls like pinning, and leveraging structured concurrency, developers can unlock unprecedented scalability. His engaging analogies and practical demos made complex concepts accessible, empowering attendees to modernize Java applications responsibly. As Java evolves, Piotr’s insights ensure developers remain equipped to navigate its concurrency landscape.
Links:
[SpringIO2024] Continuations: The Magic Behind Virtual Threads in Java by Balkrishna Rawool @ Spring I/O 2024
At Spring I/O 2024 in Barcelona, Balkrishna Rawool, a software engineer at ING Bank, captivated attendees with an in-depth exploration of continuations, the underlying mechanism powering Java’s virtual threads. Introduced as a final feature in Java 21 under Project Loom, virtual threads promise unprecedented scalability for Java applications. Balkrishna’s session demystified how continuations enable this scalability by allowing programs to pause and resume execution, offering a deep dive into their mechanics and practical applications.
Understanding Virtual Threads
Virtual threads, a cornerstone of Project Loom, are lightweight user threads designed to enhance scalability in Java applications. Unlike platform threads, which map directly to operating system threads and are resource-intensive, virtual threads require minimal memory, enabling developers to create millions without significant overhead. Balkrishna illustrated this by comparing platform threads, often pooled due to their cost, to virtual threads, which are created and discarded as needed, avoiding pooling anti-patterns. He emphasized that virtual threads rely on platform threads—termed carrier threads—for execution, with a scheduler mounting and unmounting them dynamically. This mechanism ensures efficient CPU utilization, particularly in I/O-bound applications where threads spend considerable time waiting, thus boosting scalability.
The Power of Continuations
Continuations, the core focus of Balkrishna’s talk, are objects that represent a program’s current state or the “rest” of its computation. They allow developers to pause a program’s execution and resume it later, a capability critical to virtual threads’ efficiency. Using Java’s Continuation API, Balkrishna demonstrated how continuations pause execution via the yield method, transferring control back to the caller, and resume via the run method. He showcased this with a simple example where a continuation printed values, paused at specific points, and resumed, highlighting the manipulation of the call stack to achieve this control transfer. Although the Continuation API is not intended for direct application use, understanding it provides insight into virtual threads’ behavior and scalability.
Building a Generator with Continuations
To illustrate continuations’ versatility, Balkrishna implemented a generator—a data structure yielding values lazily—using only the Continuation API, eschewing Java’s streams or iterators. Generators are ideal for resource-intensive computations, producing values only when needed. In his demo, Balkrishna created a generator yielding strings (“a,” “b,” “c”) by defining a Source object to handle value yields and pauses via continuations. The generator paused after each yield, allowing consumers to iterate over values in a loop, demonstrating how continuations enable flexible control flow beyond virtual threads, applicable to constructs like coroutines or exception handling.
Crafting a Simple Virtual Thread
In the session’s climax, Balkrishna guided attendees through implementing a simplified virtual thread class using continuations. The custom virtual thread paused execution during blocking operations, freeing platform threads, and supported a many-to-many relationship with carrier threads. He introduced a scheduler to manage virtual threads on a fixed pool of platform threads, using a queue for first-in-first-out scheduling. A demo with thousands of virtual threads, each simulating blocking calls, outperformed an equivalent platform-thread implementation, underscoring virtual threads’ scalability. By leveraging scoped values and timers, Balkrishna ensured accurate thread identification and resumption, providing a clear, hands-on understanding of virtual threads’ mechanics.
Links:
[SpringIO2023] Spring Framework 6.1: Infrastructure Revisited by Juergen Hoeller
At Spring I/O 2023 in Barcelona, Juergen Hoeller, a pivotal figure in the Spring Framework, delivered an insightful session on the upcoming Spring Framework 6.1, focusing on its alignment with cutting-edge JVM innovations. Building on themes introduced in the conference keynote, Juergen explored how Spring Framework 6.1 integrates with OpenJDK’s Project Loom (virtual threads) and CRaC (Checkpoint/Restore), empowering developers to leverage these advancements for scalable, efficient applications. With Spring Framework 6.1 and Spring Boot 3.2 set for release in November 2023, this talk offered a forward-looking perspective on infrastructure enhancements already available in snapshots and slated for the first milestone in June 2023.
Modern Java and Jakarta EE Context
Juergen set the stage by outlining the Java and Jakarta EE landscape that Spring Framework 6.1 inhabits. Spring Framework 6.0 established a JDK 17 baseline, introducing language innovations like record types, sealed classes, and pattern matching, which provide a robust foundation for modern Java development. JDK 21, a long-term support release arriving in September 2023, builds on this with further enhancements, including sequence collections and virtual threads exiting preview. Spring Framework 6.1 aligns with JDK 21, enabling developers to adopt these features seamlessly. On the Jakarta EE front, Spring Framework 6.1 supports Jakarta EE 10, with plans to align with Jakarta EE 11’s anticipated JDK 21 requirement in 2024. This ensures compatibility with evolving servlet, JPA, and bean validation APIs, positioning Spring as a bridge to future Java ecosystems.
Virtual Threads: Scalability with Simplicity
A cornerstone of Juergen’s talk was Spring Framework 6.1’s integration with virtual threads, a transformative JVM feature in JDK 21. Virtual threads, lightweight and JVM-managed, allow blocking operations without tying up platform threads, enabling massive scalability with minimal resource overhead. Juergen explained that Spring’s task executor facilities, like SimpleAsyncTaskExecutor and a dedicated VirtualThreadTaskExecutor, are now virtual thread-ready. These allow developers to replace traditional thread pool configurations with virtual thread setups, requiring minimal code changes. In Spring MVC, virtual threads are configured at the servlet container level (e.g., Tomcat or Jetty), with upcoming Tomcat releases offering first-class support. For reactive stacks like Spring WebFlux, virtual threads serve as an escape hatch for integrating blocking operations, complementing the reactive model’s efficiency. Juergen emphasized that existing Spring MVC applications can achieve higher scalability or reduced memory footprints by adopting virtual threads, with no application code changes in ideal scenarios.
Checkpoint/Restore: Revolutionizing Startup Time
Juergen also delved into Spring Framework 6.1’s support for Project CRaC, which dramatically accelerates application startup through JVM snapshotting. By capturing a fully bootstrapped application state—potentially after warmup—and restoring it on demand, CRaC reduces startup times by a factor of at least 20 compared to traditional JVM bootstrapping. Spring Framework 6.1 integrates CRaC via its lifecycle model, using the 14-year-old Lifecycle interface to manage component stop and restart signals during checkpoint and restore phases. This ensures that embedded servers, message listeners, and other components pause cleanly and resume seamlessly. Juergen noted that most Spring applications are CRaC-compatible out of the box, though some components, like Tomcat adapters, require updates. Spring Boot 3.2 may introduce standard checkpoint options post-bootstrap, simplifying adoption. Currently supported by Azul’s OpenJDK and AWS Lambda SnapStart, CRaC promises significant benefits for dynamic scaling in Linux container deployments.
Practical Adoption and Future Outlook
Juergen underscored the practical readiness of these features, with Spring Framework 6.1 M1 slated for mid-June 2023 and Spring Boot 3.2 M1 for mid-July. Release candidates are planned for October, aligning with JDK 21’s September release, ensuring developers can adopt virtual threads and CRaC promptly. He encouraged experimentation, noting that benchmarking virtual threads or CRaC with existing applications is the best way to quantify benefits like scalability or reduced startup times. Looking ahead, Juergen highlighted a Spring.Next Buff session at Spring I/O for deeper discussions on these innovations. By aligning with JDK 21 and Jakarta EE 11, Spring Framework 6.1 positions developers to embrace the Java ecosystem’s future, delivering performance and efficiency without sacrificing Spring’s hallmark simplicity.