Posts Tagged ‘StructuredConcurrency’
[DevoxxBE2024] The Next Phase of Project Loom and Virtual Threads by Alan Bateman
At Devoxx Belgium 2024, Alan Bateman delivered a comprehensive session on the advancements in Project Loom, focusing on virtual threads and their impact on Java concurrency. As a key contributor to OpenJDK, Alan explored how virtual threads enable high-scale server applications with a thread-per-task model, addressing challenges like pinning, enhancing serviceability, and introducing structured concurrency. His talk provided practical insights into leveraging virtual threads for simpler, more scalable code, while detailing ongoing improvements in JDK 24 and beyond.
Understanding Virtual Threads and Project Loom
Project Loom, a transformative initiative in OpenJDK, aims to enhance concurrency in Java by introducing virtual threads—lightweight, user-mode threads that support a thread-per-task model. Unlike traditional platform threads, which are resource-intensive and often pooled, virtual threads are cheap, allowing millions to run within a single JVM. Alan emphasized that virtual threads enable developers to write simple, synchronous, blocking code that is easy to read and debug, avoiding the complexity of reactive or asynchronous models. Finalized in JDK 21 after two preview releases, virtual threads have been widely adopted by frameworks like Spring and Quarkus, with performance and reliability proving robust, though challenges like pinning remain.
The Pinning Problem and Its Resolution
A significant pain point with virtual threads is “pinning,” where a virtual thread cannot unmount from its carrier thread during blocking operations within synchronized methods or blocks, hindering scalability. Alan detailed three scenarios causing pinning: blocking inside synchronized methods, contention on synchronized methods, and object wait/notify operations. These can lead to scalability issues or even deadlocks if all carrier threads are pinned. JEP 444 acknowledged this as a quality-of-implementation issue, not a flaw in the synchronized keyword itself. JEP 491, currently in Early Access for JDK 24, addresses this by allowing carrier threads to be released during such operations, eliminating the need to rewrite code to use java.util.concurrent.locks.ReentrantLock. Alan urged developers to test these Early Access builds to validate reliability and performance, noting successful feedback from initial adopters.
Enhancing Serviceability for Virtual Threads
With millions of virtual threads in production, diagnosing issues is critical. Alan highlighted improvements in serviceability tools, such as thread dumps that now distinguish carrier threads and include stack traces for mounted virtual threads in JDK 24. A new JSON-based thread dump format, introduced with virtual threads, supports parsing for visualization and preserves thread groupings, aiding debugging of complex applications. For pinning, JFR (Java Flight Recorder) events now capture stack traces when blocking occurs in synchronized methods, with expanded support for FFM and JNI in JDK 24. Heap dumps in JDK 23 include unmounted virtual thread stacks, and new JMX-based monitoring interfaces allow dynamic inspection of the virtual thread scheduler, enabling fine-tuned control over parallelism.
Structured Concurrency: Simplifying Concurrent Programming
Structured concurrency, a preview feature in JDK 21–23, addresses the complexity of managing concurrent tasks. Alan presented a motivating example of aggregating data from a web service and a database, comparing sequential and concurrent approaches using thread pools. Traditional thread pools with Future.get() can lead to leaks or wasted cycles if tasks fail, requiring complex cancellation logic. The StructuredTaskScope API simplifies this by ensuring all subtasks complete before the main task proceeds, using a single join method to wait for results. If a subtask fails, others are canceled, preventing leaks and preserving task relationships in a tree-like structure. An improved API in Loom Early Access builds, planned for JDK 24 preview, introduces static factory methods and streamlined exception handling, making structured concurrency a powerful complement to virtual threads.
Future Directions and Community Engagement
Alan outlined Project Loom’s roadmap, focusing on JEP 491 for pinning resolution, enhanced diagnostics, and structured concurrency’s evolution. He emphasized that virtual threads are not a performance boost for individual methods but excel in scalability through sheer numbers. Misconceptions, like replacing all platform threads with virtual threads or pooling them, were debunked, urging developers to focus on task migration. Structured concurrency’s simplicity aligns with virtual threads’ lightweight nature, promising easier debugging and maintenance. Alan encouraged feedback on Early Access builds for JEP 491 and structured concurrency (JEP 480), highlighting their importance for production reliability. Links to JEP 444, JEP 491, and JEP 480 provide further details for developers eager to explore.
Links:
[KotlinConf2024] Lifecycles, Coroutines, and Scopes: Structuring Kotlin
At KotlinConf2024, Alejandro Serrano Mena, a JetBrains language evolution researcher, delivered a re-recorded talk on structured concurrency, illuminating how coroutine scopes bridge Kotlin’s concurrency model with framework lifecycles. Exploring Compose, Ktor, and Arrow libraries, Alejandro demonstrated how scopes ensure intuitive job cancellation and supervision. From view model scopes in Compose to request scopes in Ktor and resource management in Arrow, the talk revealed the elegance of scope-based designs, empowering developers to manage complex lifecycles effortlessly.
Structured Concurrency: A Kotlin Cornerstone
Structured concurrency, a pillar of Kotlin’s coroutine library, organizes jobs within parent-child hierarchies, simplifying cancellation and exception handling. Alejandro explained that unlike thread-based concurrency, where manual tracking is error-prone, scopes make concurrency a local concern. Jobs launched within a CoroutineScope are tied to its lifecycle, ensuring all tasks complete or cancel together. This model separates logical tasks (what to do) from execution (how to run), enabling lightweight job scheduling without full threads, as seen in dispatchers like Dispatchers.IO.
Coroutine Scopes: Parents and Children
A CoroutineScope acts as a parent, hosting jobs created via launch (fire-and-forget) or async (result-producing). Alejandro illustrated this with a database-and-cache example, where a root job spawns tasks like saveToDatabase and saveToCache, each with subtasks. Cancellation propagates downward—if the root cancels, all children stop. Exceptions bubble up, triggering default cancellation of siblings unless a SupervisorJob or CoroutineExceptionHandler intervenes. Scopes wait for all children to complete, ensuring no dangling tasks, a key feature for frameworks like Ktor.
Compose: View Model Scopes in Action
In Jetpack Compose, view model scopes tie coroutines to UI lifecycles. Alejandro showcased a counter app, where a ViewModel manages state and launches tasks, like fetching weather data, within its viewModelScope. This scope survives Android events like screen rotations but cancels when the activity is destroyed, preventing job leaks. Shared view models across screens maintain state during navigation, while screen-specific view models clear tasks when their composable exits, balancing persistence and cleanup in multiplatform UI development.
Ktor: Scopes for Requests and Applications
Ktor, a Kotlin HTTP framework, leverages coroutine scopes for server-side logic. Alejandro demonstrated a simple Ktor app with GET and WebSocket routes, each tied to distinct scopes. The PipelineContext scope governs individual requests, while the Application scope spans the server’s lifecycle. Launching tasks in the request scope ensures they complete or cancel with the request, enabling fast responses. Application-scoped tasks, like cache updates, persist beyond requests, offering flexibility but requiring careful cancellation to avoid lingering jobs.
Arrow: Beyond Coroutines with Resource and Saga Scopes
The Arrow library extends scope concepts beyond coroutines. Alejandro highlighted resource scopes, which manage lifecycles of database connections or HTTP clients, ensuring automatic disposal via install or autoCloseable. Saga scopes orchestrate distributed transactions, pairing actions (e.g., increment) with compensating actions (e.g., decrement). Unlike coroutine scopes, which cancel on exceptions, saga scopes execute compensations in reverse, ensuring consistent states across services. These patterns showcase scopes as a versatile abstraction for lifecycle-aware programming.
Dispatching: Scheduling with Flexibility
Coroutine scopes delegate execution to dispatchers, separating task logic from scheduling. Alejandro noted that default dispatchers inherit from parent scopes, but developers can specify Dispatchers.IO for I/O-intensive tasks or Dispatchers.Main for UI updates. This decoupling allows schedulers to optimize thread usage while respecting structured concurrency rules, like cancellation propagation. By choosing appropriate dispatchers, developers fine-tune performance without altering the logical structure of their concurrent code.
Conclusion: Think Where You Launch
Alejandro’s key takeaway—“think where you launch”—urges developers to consider scope placement. Whether in Compose’s view models, Ktor’s request handlers, or Arrow’s resource blocks, scopes simplify lifecycle management, making cancellation and cleanup intuitive. By structuring applications around scopes, Kotlin developers harness concurrency with confidence, ensuring robust, maintainable code across diverse frameworks.
Links:
[KotlinConf2019] Kotlin Coroutines: Mastering Cancellation and Exceptions with Florina Muntenescu & Manuel Vivo
Kotlin coroutines have revolutionized asynchronous programming on Android and other platforms, offering a way to write non-blocking code in a sequential style. However, as Florina Muntenescu and Manuel Vivo, both prominent Android Developer Experts then at Google, pointed out at KotlinConf 2019, the “happy path” is only part of the story. Their talk, “Coroutines! Gotta catch ’em all!” delved into the critical aspects of coroutine cancellation and exception handling, providing developers with the knowledge to build robust and resilient asynchronous applications.
Florina and Manuel highlighted a common scenario: coroutines work perfectly until an error occurs, a timeout is reached, or a coroutine needs to be cancelled. Understanding how to manage these situations—where to handle errors, how different scopes affect error propagation, and the impact of launch vs. async—is crucial for a good user experience and stable application behavior.
Structured Concurrency and Scope Management
A fundamental concept in Kotlin coroutines is structured concurrency, which ensures that coroutines operate within a defined scope, tying their lifecycle to that scope. Florina Muntenescu and Manuel Vivo emphasized the importance of choosing the right CoroutineScope for different situations. The scope dictates how coroutines are managed, particularly concerning cancellation and how exceptions are propagated.
They discussed:
* CoroutineScope: The basic building block for managing coroutines.
* Job and SupervisorJob: A Job in a coroutine’s context is responsible for its lifecycle. A key distinction is how they handle failures of child coroutines. A standard Job will cancel all its children and itself if one child fails. In contrast, a SupervisorJob allows a child coroutine to fail without cancelling its siblings or the supervisor job itself. This is critical for UI components or services where one failed task shouldn’t bring down unrelated operations. The advice often given is to use SupervisorJob when you want to isolate failures among children.
* Scope Hierarchy: How scopes can be nested and how cancellation or failure in one part of the hierarchy affects others. Understanding this is key to preventing unintended cancellations or unhandled exceptions.
Cancellation: Graceful Termination of Coroutines
Effective cancellation is vital for resource management and preventing memory leaks, especially in UI applications where operations might become irrelevant if the user navigates away. Florina and Manuel would have covered how coroutines support cooperative cancellation. This means that suspending functions in the kotlinx.coroutines library are generally cancellable; they check for cancellation requests and throw a CancellationException when one is detected.
Key points regarding cancellation included:
* Calling job.cancel() initiates the cancellation of a coroutine and its children.
* Coroutines must cooperate with cancellation by periodically checking isActive or using cancellable suspending functions. CPU-bound work in a loop that doesn’t check for cancellation might not stop as expected.
* CancellationException is considered a normal way for a coroutine to complete due to cancellation and is typically not logged as an unhandled error by default exception handlers.
Exception Handling: Catching Them All
Handling exceptions correctly in asynchronous code can be tricky. Florina and Manuel’s talk aimed to clarify how exceptions propagate in coroutines and how they can be caught.
They covered:
* launch vs. async:
* With launch, exceptions are treated like uncaught exceptions in a thread—they propagate up the job hierarchy. If not handled, they can crash the application (depending on the root scope’s context and CoroutineExceptionHandler).
* With async, exceptions are deferred. They are stored within the Deferred result and are only thrown when await() is called on that Deferred. This means if await() is never called, the exception might go unnoticed unless explicitly handled.
* CoroutineExceptionHandler: This context element can be installed in a CoroutineScope to act as a global handler for uncaught exceptions within coroutines started by launch in that scope. It allows for centralized error logging or recovery logic. They showed examples of how and where to install this handler effectively, for example, in the root coroutine or as a direct child of a SupervisorJob to catch exceptions from its children.
* try-catch blocks: Standard try-catch blocks can be used within a coroutine to handle exceptions locally, just like in synchronous code. This is often the preferred way to handle expected exceptions related to specific operations.
The speakers stressed that uncaught exceptions will always propagate, so it’s crucial to “catch ’em all” to avoid unexpected behavior or crashes. Their presentation aimed to provide clear patterns and best practices to ensure that developers could confidently manage both cancellation and exceptions, leading to more robust and user-friendly Kotlin applications.