Jonathan Lalou's Blog

Lecturer

Tomori is a Senior Specialist Solutions Architect at Amazon Web Services (AWS), specializing in high-performance computing (HPC) and distributed machine learning. Tomori’s research and professional focus center on the “physics” of training—optimizing the interaction between silicon, networking fabrics, and software frameworks to enable the training of models with trillions of parameters.

Abstract

Training state-of-the-art AI models has become a monumental engineering challenge that requires the precise orchestration of massive compute clusters. This article explores the technical foundations of training high-performing AI models at scale using AWS’s purpose-built infrastructure. It provides an in-depth analysis of Amazon SageMaker HyperPod and its role in distributed training, covering essential techniques such as data and model parallelism. The discussion highlights the shift from managed APIs to Kubernetes-based orchestration (EKS) and Slurm for high-performance job scheduling. Key innovations in automated model tuning, observability through Prometheus and Grafana, and efficient checkpoint management are detailed, showing how these strategies can reduce training time by up to 40% while optimizing costs for the next generation of foundation models.

Scaling Challenges in the Era of Foundation Models

The shift toward foundation models has fundamentally changed the requirements for AI training infrastructure. When a model’s memory footprint exceeds the capacity of a single GPU, developers must move beyond simple training scripts to sophisticated distributed systems. The challenges are three-fold: compute intensity, memory constraints, and networking bottlenecks.

At scale, the “efficiency of the cluster” becomes the primary metric. Organizations must navigate the trade-offs between different types of parallelism—Data Parallelism, where the dataset is split across nodes, and Model Parallelism (including Pipeline and Tensor parallelism), where the model itself is distributed. AWS addresses these challenges by providing a specialized infrastructure layer, specifically through P5 instances powered by NVIDIA H100 GPUs and high-bandwidth Elastic Fabric Adapter (EFA) networking. This hardware foundation is critical for maintaining the high-speed communication necessary for synchronization during the backward pass of training.

Orchestration Choice: Kubernetes vs. Slurm

A critical decision for any organization training at scale is the choice of orchestrator. AWS offers two primary paths for managing SageMaker HyperPod clusters, each catering to different operational philosophies.

1. Kubernetes (Amazon EKS): This is often the preferred choice for organizations that have adopted a cloud-native DevOps culture. Kubernetes provides a robust API for managing containers, and when integrated with SageMaker HyperPod, it allows for the orchestration of ML training jobs alongside other microservices. This path benefits from a rich ecosystem of observability tools like Prometheus and Grafana.
2. Slurm: Traditionally used in high-performance computing (HPC) environments, Slurm is a highly efficient job scheduler designed for massive, tightly coupled workloads. Organizations coming from research backgrounds or those requiring the absolute lowest overhead often choose Slurm.

Regardless of the orchestrator, SageMaker HyperPod provides the underlying “resiliency layer,” which includes automated node health monitoring and the ability to replace failing instances without human intervention. This ensures that the long-running training “epochs” are not disrupted by hardware variability.

Parallelism Strategies and Performance Optimization

Achieving a 40% reduction in training time requires more than just more GPUs; it requires optimized parallelism strategies. The discussion centers on three primary methodologies:

• Data Parallelism: The most common approach, where each GPU processes a different batch of data and gradients are synchronized.
• Tensor Parallelism: Splitting individual layers across multiple GPUs to handle extremely wide layers.
• Pipeline Parallelism: Dividing the model into stages and processing them in a “pipeline” fashion to maximize GPU utilization.

SageMaker’s distributed training libraries automate much of this complexity. By injecting “sharding” logic into the training script, these libraries allow developers to scale their models with minimal code changes. Furthermore, the use of automated model tuning—which systematically explores the hyperparameter space—ensures that the resulting model reaches its peak performance without manual “trial and error.”

Code Sample: Configuring Model Parallelism in SageMaker

from sagemaker.pytorch import PyTorch

'''
Configuration for SageMaker Model Parallelism (SMP)
'''
smp_options = {
    "enabled": True,
    "parameters": {
        "pipeline_parallel_degree": 2,
        "tensor_parallel_degree": 4,
        "optimize": "speed",
        "ddp": True
    }
}

'''
Training job configuration using HyperPod
'''
estimator = PyTorch(
    entry_point='train_llm.py',
    instance_type='ml.p5.48xlarge',
    instance_count=16, # Scaling to 128 GPUs
    distribution={"smdistributed": {"modelparallel": smp_options}},
    role=sagemaker.get_execution_role(),
    framework_version='2.1'
)

estimator.fit()

Observability and Fault Tolerance

In a cluster of hundreds of GPUs, “observability” is not a luxury—it is a requirement. Tomori emphasizes the importance of a centralized observability stack using Prometheus and Grafana. These tools allow engineers to monitor GPU utilization, memory pressure, and network throughput in real time. If a node begins to underperform or “throttles” due to heat, the observability stack provides the alerts necessary for proactive management.

Coupled with observability is the concept of efficient checkpoint management. By utilizing high-performance storage like Amazon FSx for Lustre, SageMaker HyperPod can save model “weights” and optimizer states rapidly. In the event of a failure, the system can reload the most recent checkpoint and resume training within minutes. This “resilient training” loop is what enables the massive foundation models of today to be trained in weeks rather than months.

Conclusion

Training high-performing AI models at scale is as much an infrastructure challenge as it is an algorithmic one. Through the integration of SageMaker HyperPod, optimized parallelism libraries, and robust orchestration via EKS or Slurm, AWS has created an environment where “the physics of scaling” can be mastered. By reducing training times and automating resiliency, these tools empower organizations to push the boundaries of what is possible, turning the promise of trillion-parameter models into a repeatable, cost-effective reality.

Links:

• Amazon SageMaker HyperPod Resource
• AWS Training Jobs GitHub Examples
• Distributed Training Libraries on AWS

Lecturers

Marit van Dijk is a highly regarded Java Champion, Docker Captain, and Developer Advocate specializing in IDE productivity enhancements, Kotlin ecosystem contributions, and making advanced topics approachable for broad audiences. Piotr Przybył serves as Senior Developer Advocate at Elastic, holding Java Champion and Docker Captain distinctions. He is widely recognized for deep explorations of Java performance characteristics, language evolution, and practical development methodologies.

Abstract

Java’s consistent release rhythm introduces valuable innovations that can sometimes feel daunting to integrate, particularly for developers working with established codebases or transitioning from prior versions. Marit van Dijk and Piotr Przybył present an engaging, experimentation-focused approach to adopting contemporary Java capabilities. Through extensive live coding demonstrations within IntelliJ IDEA, they cover a rich selection of features including compact source files, enhanced records, stream gatherers, structured concurrency primitives, vector API computations, and documentation improvements. The session emphasizes immediate practical value, IDE assistance, and the joy of discovery.

The Motivation for Staying Current with Java Evolution

Java maintains a predictable six-month release cycle, consistently delivering enhancements across performance optimizations, concurrency abstractions, security improvements, and developer experience refinements. While Long-Term Support (LTS) releases provide stability crucial for risk-averse enterprises requiring extended commercial backing, the speakers encourage thoughtful consideration of intermediate versions for teams positioned to capitalize on their advancements.

They readily acknowledge legitimate concerns around upgrade processes, dependency compatibility, and organizational inertia. However, they highlight maturing ecosystem support—including comprehensive IDE tooling, community resources, and backward compatibility efforts—that significantly eases transitions. The overarching message promotes proactive engagement rather than passive waiting, positioning Java as a continually improving platform rather than a static legacy technology.

Learning modern Java is portrayed as an active, playful process reminiscent of childhood discovery. This perspective transforms potentially intimidating language additions into opportunities for creative exploration and immediate productivity gains, supported powerfully by modern development environments.

Syntactic Refinements: Compact Sources, Records, and Module Imports

Java 25’s introduction of compact source files and instance main methods marks a notable step toward reduced ceremony. Traditional requirements for explicit class declarations and static main entry points are relaxed, enabling concise scripts ideal for learning exercises, rapid prototyping, command-line utilities, and exploratory development. This change particularly benefits educational contexts and situations requiring quick experimentation.

Records, stabilized since Java 16, function as transparent, immutable carriers of data. They automatically provide canonical constructors, accessor methods, sensible equals, hashCode, and toString implementations, along with pattern matching support. Perfect for DTOs, value objects, and request/response models, records substantially reduce boilerplate and associated error opportunities. Developers retain the ability to customize behavior through explicit overrides when domain requirements demand specialized semantics.

Module imports offer further conciseness by permitting declarations such as import module java.base;, which encompasses the foundational JDK packages. This eliminates lengthy lists of individual import statements while preserving code clarity and avoiding namespace pollution. The IO utility class simplifies basic console input/output operations, replacing more verbose patterns involving Scanner and direct System.out usage.

Collectively, these syntactic improvements reflect Java’s ongoing commitment to balancing power with approachability, making the language more welcoming without compromising its industrial strengths.

Expressive Data Processing: Stream Gatherers and Parallel Operations

The stream gatherers feature, finalized in Java 24, significantly expands the expressiveness of the Streams API. Developers can now define custom intermediate operations that encapsulate complex processing logic, replacing ad-hoc combinations of filter, map, flatMap, and similar methods. This promotes creation of reusable, composable, and semantically rich pipelines suited to sophisticated data transformations.

A particularly valuable gatherer, mapConcurrent, enables safe parallel processing by leveraging virtual threads, avoiding traditional fork-join pool exhaustion issues common with parallel() streams in certain contexts. The speakers demonstrate practical applications ranging from straightforward data enrichment to more involved computations, highlighting improved readability and maintainability.

Concurrency Paradigms: Structured Scopes and Lazy Initialization

Structured concurrency, available as a preview feature in Java 26, introduces a more disciplined approach to managing concurrent operations. By structuring tasks within a StructuredTaskScope, developers achieve clearer code organization, automatic resource cleanup, and built-in cancellation semantics. This represents a substantial improvement over error-prone patterns relying on CompletableFuture compositions, reducing risks of thread leaks and orphaned computations.

Lazy constants provide controlled deferral of initialization until first access, contributing to faster startup times and more efficient resource utilization. Expanded pattern matching capabilities for primitive types enhance the power and readability of switch expressions and type checks across broader scenarios.

Performance Acceleration: Vector API and Documentation Innovations

Project Panama’s Vector API unlocks access to hardware vector instructions (SIMD), enabling significant performance improvements for data-parallel workloads without requiring low-level assembly coding. Through high-level Java abstractions, developers can express vectorized operations on arrays and other data structures, achieving throughput gains particularly beneficial in scientific computing, financial modeling, image processing, and machine learning applications.

The speakers present concrete benchmarks comparing scalar and vectorized implementations, illustrating real-world speedups. Documentation enhancements, specifically Markdown support within Javadoc comments introduced in Java 23, modernize API documentation practices. Rich formatting capabilities render effectively in contemporary IDEs, reducing the maintenance burden historically associated with HTML-based Javadoc.

Ecosystem Synergies and Forward Outlook

The presentation extends beyond core language features to examine synergies with development tools, community resources, and evolving practices. Modern IDEs like IntelliJ IDEA provide sophisticated support for preview features, refactoring assistance, and code analysis that accelerate adoption. Community engagement through user groups, conferences, and open-source contributions further enriches the learning experience.

The speakers encourage deliberate experimentation within personal or side projects as a low-risk pathway to building confidence before enterprise-wide rollout. They stress the importance of updating not merely version numbers but also mental models and coding idioms to fully leverage Java’s capabilities.

Java’s ongoing development demonstrates its continued relevance as a powerful, productive platform for contemporary software engineering. By embracing curiosity and playful exploration, developers can harness these advancements to enhance both personal satisfaction and professional effectiveness.

Links:

• Video: Learning modern Java the playful way
• Marit van Dijk Website
• Piotr Przybył Website
• OpenJDK Project Amber
• Java 25 Release Notes
• JEP Catalog

In the digital domain, where inclusion is imperative yet often ignored, Elise Kristiansen, a senior front-end savant at Opa, exposes the exigencies of accessibility. With Norway’s mandates for universal design entrenched for years, Elise enumerates ten traps thwarting equitable access, offering actionable antidotes. Her lightning discourse, brisk yet brimming, beckons builders to bolster barriers, ensuring digital doors open for all.

Elise opens with urgency: WCAG’s wisdom, legislated for inclusivity, languishes as new sites sideline the sightless, the deaf, the distinct. Her mission: awaken awareness, arming architects with insights to integrate accessibility from inception.

Technical Traps: Text and Multimedia Missteps

Elise’s initial indictments: absent alt text for images, missing captions for multimedia—blind and deaf users barred from content. Her remedy: wield scanners—Pa11y pinpoints omissions—then craft concise, contextual descriptions. Videos demand subtitles, audio begs transcripts—ensuring equivalence for all.

Keyboard navigation, another nemesis, neglects non-mouse navigators. Elise urges: ensure focus flows logically—tab orders tailored, skip links supplied—empowering keyboard-bound users to traverse terrains unhindered.

Design and Development Deficits: Planning Pitfalls

Design’s disregard dooms accessibility: color contrasts collapse, focus indicators fade. Elise’s exhortation: adopt ARIA landmarks, enforce contrast ratios—WCAG’s 4.5:1 for text, 3:1 for graphics—ensuring visibility for visually impaired viewers.

Development’s delay—accessibility as afterthought—derails deadlines. Elise advocates: embed accessibility in estimates, engage all stakeholders—customers, designers, editors—in inclusive intent from planning’s outset.

Awareness and Economics: Overcoming Obstacles

Ignorance impedes: developers and designers, untrained, overlook accessibility’s essence. Elise’s solution: champion courses—hours invested yield inclusive interfaces. Her final foe: economic excuses, where overlays masquerade as solutions. She counters: simplicity scales—minimalist designs, sans compromise, serve all.

Elise’s entreaty: prioritize people, not profits—universal design unites, exclusion erodes.

Links:

• Opa company website
• Elise Kristiansen on LinkedIn

Lecturer
Anton Arhipov is a Developer Advocate at JetBrains, where he engages with developers and shares insights on Kotlin, IntelliJ IDEA, and related technologies. He frequently speaks at conferences and contributes to JetBrains initiatives around AI tooling and agent frameworks.

Abstract
Anton Arhipov’s session at Voxxed Days Amsterdam 2026 introduces the construction of AI agents directly on the JVM using Koog, a lightweight Kotlin-native framework developed at JetBrains. Rather than relying on Python-based solutions, the talk demonstrates building tool-using LLM agents in Kotlin, emphasizing seamless integration with existing JVM products and domain models. Through live coding and detailed examination of core concepts—including communication with language models, tool registration, interaction loops, and the distinctive graph-based strategy DSL—participants gain insight into creating robust, reproducible agent workflows. The presentation culminates in a coding agent example that generates, compiles, and tests small programs, highlighting both opportunities and challenges when working with local and remote LLMs.

Why Build AI Agents on the JVM? Context and Motivations

The emergence of large language models has spurred interest in autonomous AI agents capable of tool use and iterative reasoning. While Python has historically dominated this space, Anton argues that language choice should serve the embedding context. For JVM-based products such as IntelliJ IDEA, TeamCity, or YouTrack, implementing agents in Python introduces significant overhead: external process management, serialization costs, and duplication of domain models. A native JVM solution reuses existing infrastructure, types, and libraries, improving efficiency and maintainability.

Koog originated from internal JetBrains efforts to embed agents into their tools. The framework’s tech lead documented the transition from Python to Kotlin, sparking discussions around perceived “skill issues” versus genuine architectural advantages. Anton stresses that the decision prioritizes integration over language fashion.

Fundamentals of Agent Architecture and the Interaction Loop

At a high level, an AI agent comprises an environment, a strategy (or “brain”), model providers, and user interaction. The workflow begins with a user request. The agent forwards a prompt to an LLM via a prompt executor (supporting OpenAI, Google, Ollama, and others). The model may respond with:

• An assistant message (plain text, terminating the loop), or
• A tool call, prompting the agent to invoke registered tools, feed results back to the LLM, and iterate until an assistant message appears.

This forms the agentic loop. Koog models agents as simple classes with a tools registry, configurable strategy, prompt, model selection, iteration limits, and a prompt executor. Predefined models and capability checks (e.g., tool calling support) simplify setup.

Anton demonstrates a basic happy-path agent that writes a Kotlin program (initially a “Hello World,” later a Snake game using Java Swing). Early iterations reveal the need for tools: file system operations, shell execution via Gradle, and careful prompting to enforce compilation and verification steps. Live runs expose humorous LLM behaviors, such as generating wrapper files or hallucinating results, underscoring the importance of robust system prompts, validation, and verification.

Tool Integration, Local Models, and Observed Challenges

Tools in Koog can be registered via a registry or supplied as annotated functions with descriptions for better LLM guidance. The session explores interactions with local models (e.g., small Mistral variants via Ollama) versus remote APIs. Local models offer speed and privacy but exhibit unreliability in tool calling—hallucinating results, ignoring descriptions, or fabricating outputs when tool results appear implausible (such as unrealistic temperature values in Kelvin).

Examples illustrate how insufficient tool descriptors or conflicting instructions lead to unexpected behavior. Anton shows cases where models call tools multiple times, reject their own outputs, or produce inconsistent results across runs, even with identical prompts. These observations highlight that reliable tool calling remains critical for effective agents and often requires significant prompt engineering and workflow safeguards.

The Strategy DSL: Graph-Based Control for Determinism and Reproducibility

A standout feature of Koog is its strategy DSL, which models agent behavior as a reproducible graph rather than a purely LLM-driven loop. The default strategy handles basic LLM calls and tool invocation, but custom strategies allow sophisticated workflows.

Nodes represent actions (e.g., calling the LLM, processing tool results), while edges use predicates or conditions (e.g., “if tool call, proceed to execution node”). Graphs support sequencing, branching, and chaining multiple strategies—such as enforcing a planning stage before implementation and verification. Type safety in Kotlin ensures nodes and edges compose correctly, with the compiler assisting in workflow construction.

Anton walks through building a simple parsing-and-increment graph, then extends the concept to more realistic scenarios involving structured outputs for plans. By validating plans against domain models and potentially generating dynamic sub-strategies, developers introduce determinism into otherwise stochastic LLM processes. Structured outputs improve reliability when deserializing LLM responses into typed objects like plan items.

This graph approach distinguishes Koog from many other frameworks, enabling complex, auditable, and testable agent logic while leveraging Kotlin’s strengths.

Live Coding a Coding Agent and Practical Takeaways

The session features extensive live exploration, including attempts to generate a Snake game with requirements for Gradle, specific versions, compilation verification, and no mistakes. Successes and failures illustrate real-world agentic challenges: hallucinated tool calls, skipped verification steps, and the need for precise instructions. Anton shares that even with “make no mistakes” in the prompt, outcomes vary across models and runs.

The talk concludes with a repository of examples and reiterates Koog’s multi-platform nature, adapter support for various model providers, and the power of the strategy DSL. While 45 minutes cannot cover advanced topics, the presentation equips attendees with foundational knowledge to experiment with building JVM-native agents.

Code Sample: Basic Agent Setup in Koog (Kotlin)

val executor = OpenAI.promptExecutor { /* config */ }
val agent = Agent {
    promptExecutor = executor
    systemPrompt = "You are a helpful Kotlin developer..."
    model = OpenAIModels.GPT_4O
    tools = toolRegistry { 
        include(FileSystemProvider())
        include(ShellExecutor())
    }
}

agent.run("Write a Kotlin program to print Hello World")

Code Sample: Simple Strategy Graph

val strategy = strategy<String, Int> {
    start forwardingTo parseNode forwardingTo incrementNode
}

val parseNode = node { input: String -> input.toInt() }
val incrementNode = node { input: Int -> input + 1 }

These examples capture the framework’s emphasis on readability and type-safe composition.

In summary, Anton Arhipov provides a grounded, pragmatic overview of constructing AI agents on the JVM. Koog enables developers to harness LLM capabilities within familiar Kotlin and JVM ecosystems, with the strategy DSL offering a powerful mechanism for controlling complexity. While challenges with model reliability persist—especially locally—the framework’s design promotes integration, reproducibility, and incremental improvement of agentic systems.

Links:

• Video URL
• Koog Framework Information
• Anton Arhipov on JetBrains Blog

Lecturer

Victor Uria Valle is a Full Stack Software Engineer at CERN, specializing in modernizing and developing enterprise applications to support laboratory operations. He holds a degree from Universidad de Oviedo and pursues personal projects in artificial intelligence, including game-playing agents. Relevant links include his LinkedIn profile (https://ch.linkedin.com/in/vuria).

Abstract

This article scrutinizes Victor Uria Valle’s endeavor to construct a Deep Q-Network (DQN) in Java for mastering the board game Azul. It delineates the game’s mechanics, Q-learning fundamentals, neural network integration, and training methodologies. Through analytical exploration, it assesses environmental modeling, state representation, reward engineering, and performance implications, underscoring Java’s viability for reinforcement learning without specialized frameworks.

Contextualizing Azul and Reinforcement Learning

Victor commences by elucidating Azul’s rules, a tile-placement game involving a shared board with factories displaying colored tiles and a central area. Players select tiles of one color from a factory or center, placing them on their personal board’s pattern lines—a right-angled triangular grid. Completed lines transfer tiles to a wall, scoring points based on adjacency, while excess tiles incur penalties on the floor line. Code structures reflect this: enums define tile colors, factories hold tile lists, player boards track pattern lines, walls, floor lines, scores, and penalties. Legal moves constrain placements—tiles must match line colors, avoid filled wall positions, and handle overflows. This setup frames reinforcement learning (RL), where agents learn via trial-and-error interactions. Victor employs Q-learning, estimating action values (Q-values) in states to maximize cumulative rewards. Traditionally tabular, Q-learning suits discrete spaces but falters in Azul’s vast state space (exponential due to board configurations). Contextually, AI in games traces to chess and Go, but Victor opts for Java sans PyTorch or TensorFlow, demonstrating accessibility. Implications: Democratizes RL for Java developers, though lacks optimized libraries.

Deep Q-Network Architecture and Innovations

To address state complexity, Victor integrates deep neural networks, approximating Q-values via function approximation. The DQN processes states as inputs, outputting Q-values per action. State encoding vectorizes boards: binary representations for tile presence in factories, centers, pattern lines, walls, and floors. Actions encode selections (factory/center, color, line/floor). The neural network, a multi-layer perceptron, uses rectified linear units (ReLU) for non-linearity, trained via backpropagation with mean squared error loss. Innovations include experience replay—storing state-action-reward-next-state tuples in a buffer, sampling mini-batches to break correlations—and a target network, periodically updated to stabilize learning. Epsilon-greedy policy balances exploration (random actions) and exploitation (highest Q-value actions), decaying epsilon over episodes. Code snippets illustrate:

public double[] forward(double[] input) {
    // Input to hidden layer
    double[] hidden = new double[hiddenSize];
    for (int i = 0; i < hiddenSize; i++) {
        double sum = 0;
        for (int j = 0; j < inputSize; j++) {
            sum += input[j] * weightsInputHidden[j][i];
        }
        hidden[i] = Math.max(0, sum + biasHidden[i]); // ReLU
    }
    // Hidden to output
    double[] output = new double[outputSize];
    for (int i = 0; i < outputSize; i++) {
        double sum = 0;
        for (int j = 0; j < hiddenSize; j++) {
            sum += hidden[j] * weightsHiddenOutput[j][i];
        }
        output[i] = sum + biasOutput[i];
    }
    return output;
}

This forward pass computes Q-values, enabling action selection. Implications: Custom implementation fosters understanding, though less efficient than frameworks; suitable for educational prototypes.

Training Methodology and Reward Design

Training employs a DQN agent and trainer. The agent selects actions via epsilon-greedy, executes them, computes rewards, and stores experiences. Replay samples batches, updating the network; target updates occur periodically. Rewards equal score deltas minus amplified floor penalties, incentivizing efficient placements. Trainer parameters include episodes, steps per save, epsilon decay (initial 1.0 to minimum 0.05). Live demo showcases training: loading pre-trained models, executing moves in a terminal UI, observing random explorations (5% minimum) for continual learning. Methodologically, Victor iterates games, refining via replays. Contexts: Balances short-term gains (line completions) with long-term strategies (wall synergies). Implications: Reward shaping critical—penalizing floors curbs suboptimal habits; serialization enables resuming, vital for lengthy trainings.

Analytical Implications for Java-Based AI

Victor’s approach validates Java for RL sans external dependencies, leveraging enums, lists for game logic, arrays for neural computations. Innovations: Pure Java DQN demystifies black-box frameworks, aiding pedagogy. Challenges: Manual matrix operations less performant; state space demands efficient encoding. Future: Enhance UI, explore advanced variants (Double DQN). Broader ramifications: Empowers non-ML specialists in game AI; implications for scalable, interpretable agents in constrained environments.

Links:

• Lecture Video
• Victor Uria Valle on LinkedIn

Lecturer

Frank Greco is a prominent figure in the global technology landscape, serving as a Senior Consultant and Enterprise Architect with a specialized focus on Artificial Intelligence, Machine Learning, Cloud, and Edge computing. A long-standing Java Champion and educator, Frank is the founder and Chairman of the NYJavaSIG (New York Java Special Interest Group) and a co-author of JSR 381, the Java API for Visual Recognition. His extensive career includes developing innovations for financial systems and building developer communities worldwide.

Abstract

This article explores the fundamental shift from deterministic to probabilistic computing necessitated by the integration of Large Language Models (LLMs) into modern software engineering. By analyzing the internal mechanics of LLMs—specifically their nature as text pattern recognizers—this study identifies the critical distinction between generative models and traditional search engines. The analysis further investigates the collaborative synergy between language models and search infrastructure, providing a roadmap for Java developers to leverage these tools while mitigating the risks associated with non-deterministic outputs in production environments.

From Determinism to Probability: A Paradigm Shift

For decades, software development has been rooted in determinism—the principle that a specific input will consistently yield an identical, predictable output. The introduction of generative AI marks the first major transition toward probabilistic systems in mainstream application development. Unlike a weather forecast, which provides probabilities about future events, LLMs operate on internal probabilities to determine the next most likely token in a sequence. This shift requires developers to move beyond the expectation of 100% accuracy and instead design systems capable of managing uncertainty and “hallucinations”.

The Taxonomy of Large Language Models

A common misconception is treating an LLM as a traditional search engine. Technically, these models are sophisticated text pattern recognizers. They do not “know” facts in the way a database does; rather, they complete text based on patterns identified during training. While an LLM can provide accurate information if the query aligns with a path through its neural network that was heavily reinforced during training, it lacks a deterministic mechanism for factual verification.

Synergy Between Search Engines and Language Models

The true power of modern AI interfaces often stems from a hybrid approach where LLMs are integrated with search infrastructure. Major technology providers utilize search engines to retrieve relevant data points, which are then processed by a language model to formulate a coherent, human-like response. This methodology allows the model to act as a sophisticated interface layer that “understands” the user’s intent while relying on the search engine’s deterministic data retrieval to ground the response in reality.

Implications for Production Environments

Integrating probabilistic tools into production Java deployments introduces unique challenges. Developers must evaluate the risk threshold of their applications; if a single incorrect answer could cause significant harm or financial loss, a purely generative approach may be inappropriate. Understanding that LLMs are distinct from the deterministic tools of the past—such as compilers or SQL databases—is the first step toward building resilient AI-integrated systems that enhance productivity without compromising software integrity.

Links:

• Frank Greco on LinkedIn
• Frank Greco on Twitter/X
• NYJavaSIG Official Website
• JSR 381: Visual Recognition API
• Lecture Video

Lecturer

Chris Hollik is the General Manager of AI and Platform at Boomi. A veteran in the integration and automation space, Chris has been a driving force behind Boomi’s expansion into the AI agent market. He is a proponent of “agentic workflows,” arguing that the true ROI of AI comes not from individual chatbots, but from integrated, governed processes that span multiple enterprise systems.

Thomas Benjamin serves as the Senior Vice President of Engineering at Boomi. Thomas leads the technical development of Boomi’s AI-first platform, including the Agentstudio and Agent Control Tower. His focus is on architecting high-availability, multi-tenant systems that integrate with leading AI services like Amazon Bedrock to provide enterprise-grade security and observability for autonomous agents.

Abstract

The rapid proliferation of AI agents within the enterprise—a phenomenon known as “agent sprawl”—presents a significant challenge for modern IT organizations. As departments independently deploy agents from various vendors (e.g., Salesforce, Microsoft, OpenAI), the need for a centralized governance and orchestration layer becomes paramount. This article explores the Boomi Agentstudio and its integration with Amazon Bedrock, demonstrating how organizations can build and manage “agentic workflows.” These workflows connect multiple business systems—such as SAP, Oracle, and Salesforce—to execute complex tasks like manual journal entry reconciliation or regulatory compliance in pharma. Key to this strategy is the Boomi Agent Control Tower, which provides a unified interface for discovering, governing, and auditing agents in a multi-vendor environment, ensuring that the promise of hyperproductivity is matched by rigorous operational control.

The Imperative for AI Agent Governance

The “agentic revolution” is no longer a future concept; it is happening today at extreme scale. Boomi already has over 12,000 agents in production, with that number growing rapidly as organizations seek to unlock “hyperproductivity.” However, this rapid adoption has led to a critical problem: agent sprawl. Organizations find themselves in a multi-agent world where different departments use disparate tools from Anthropic, OpenAI, ServiceNow, and others.

Without a platform, this fragmented approach is unsustainable. An MIT study recently suggested that the potential for “shadow AI”—where agents operate outside the view of IT—could lead to significant security, compliance, and cost issues. For high-stakes industries like finance or pharmaceuticals, where auditors require immutable audit trails, the “black box” nature of ungoverned agents is a non-starter. The core question for the enterprise is: How do we become “enterprise class” in our AI strategy while still innovating at speed?

Boomi Agentstudio: Orchestrating the Multi-Agent Ecosystem

Boomi Agentstudio addresses these challenges by providing a dedicated environment for building and orchestrating AI agents. It is not merely a tool for creating single agents, but a platform for designing complex, multi-system workflows. Integrated with Amazon Bedrock and Q Business, Agentstudio allows users to leverage the best-of-breed foundation models while grounding them in the specific business context of their organization.

One of the platform’s unique strengths is its ability to stitch together master data from multiple applications—such as Salesforce, ServiceNow, Shopify, and SAP—to provide agents with the information they need to execute. This “connectivity tissue” ensures that an agent isn’t just generating text, but is actually performing actions like updating a downstream schema or moving data in a specific serialization format to prevent integration failure.

The system is built on open standards, being Open Telemetry compliant and contributing to protocols like the Model Context Protocol (MCP) and Agent-to-Agent (A2A). This openness ensures that as the AI landscape evolves, the Boomi platform remains compatible with any OTEL-compliant observability platform.

The Technical Foundation: Integration and Automation for AI

The effectiveness of an AI agent is fundamentally limited by the systems it can access and the data it can interpret. This is why Boomi’s heritage in Integration Platform as a Service (iPaaS) is so relevant. Integration provides the “common language” for systems to talk to each other, but it is automation that makes those processes work successfully at scale.

Boomi leverages a range of AWS services to operate at “four nines” (99.99%) of availability, including Amazon ECS, EKS, Fargate, and SQS. This infrastructure allows the platform to handle the intense computational and data-movement demands of real-time AI agents. For example, when a command is issued from the Control Tower to stop an agent, it is passed through SQS to a Lambda function that interacts directly with the Amazon Bedrock agent running in the customer’s VPC. This level of deep architectural integration ensures real-time feedback and control.

Code Sample: Agent Governance Call (Pseudo-code for Control Tower API)

{
  "command": "SUSPEND_AGENT",
  "agent_id": "FINANCE_RECON_AGENT_01",
  "reason": "Regulatory Audit in Progress",
  "governance_policy": {
    "stop_method": "GRACEFUL",
    "notify_admin": true,
    "log_level": "VERBOSE",
    "audit_persistence": "S3_IMMUTABLE_LOG"
  },
  "target_environment": {
    "cloud_provider": "AWS",
    "service": "Amazon Bedrock",
    "vpc_endpoint": "vpce-0a1b2c3d4e5f6g7h8"
  }
}

Agent Control Tower: Ensuring Auditability and Compliance

The Agent Control Tower is the “brain” of the Boomi AI ecosystem. It allows IT leaders to discover every agent running in their organization, regardless of the vendor or framework (such as LangGraph). It serves as a centralized policy enforcement point where security and operational rules are applied.

Key features of the Control Tower include:

• Discovery and Inventory: Seeing the “sprawl” in real time and bringing shadow agents under management.
• Operational Control: The ability to start, stop, or pause agents from a single dashboard.
• Immutable Audit Trails: Capturing every reasoning step and action taken by an agent. This is crucial for demonstrating to auditors how a specific financial decision or pharmaceutical report was generated.
• ROI Dashboarding: Tracking the performance and value delivery of agents to justify investment and prioritize further innovation.

Real-World Applications: From Finance to Pharma

The practical application of governed agents is vast. In the pharmaceutical industry, Boomi agents are used to manage complex regulatory workflows, ensuring that every step of a drug-approval process is documented and compliant. In finance, agents automate the “order to cash” or “hire to retire” processes, examining thousands of records in seconds to find discrepancies that would take a human days to uncover.

By providing a platform that handles the “boring” but essential tasks of governance, security, and integration, Boomi enables organizations to focus on the “interesting” part: innovating at speed. This approach ensures that AI is not just a experimental toy, but a reliable, enterprise-grade component of the modern business fabric.

Conclusion

As enterprises move from AI experimentation to full-scale production, the focus must shift from the capabilities of individual models to the governance of entire agentic ecosystems. The Boomi Agentstudio, supported by the scalability of AWS and the robust control of the Agent Control Tower, provides the necessary framework for this transition. By centralizing management in a multi-vendor world, Boomi ensures that organizations can harness the power of AI while maintaining the security and auditability required by the modern regulatory landscape.

Links:

• Boomi Agentstudio Overview
• Amazon Bedrock Service Page
• Boomi on AWS Marketplace

Lecturer

Pankaj Kumar Mishra serves as Head of Sales, Go-to-Market, and Generative AI at TCS AI.Cloud Unit, specializing in life sciences and healthcare for North America. With over seventeen years in information technology, Pankaj focuses on AWS-driven business growth and strategic partnerships. Bhavik Patel leads the architecture practice for the US employer segment at Cigna Healthcare, bringing twenty-three years of experience in technology roles within health insurance. Rajesh contributes to TCS’s cloud and modernization initiatives, emphasizing collaborative efforts in enterprise transformations.

Abstract

This in-depth examination traces Cigna Healthcare’s overhaul of sales and client onboarding processes through a cloud-native platform developed in partnership with Tata Consultancy Services and Amazon Web Services. It scrutinizes the operational hurdles in legacy systems, architectural strategies for streamlining workflows, and resultant enhancements in precision, speed, and stakeholder engagement. By analyzing the interplay of migration acceleration programs, financial optimization, and domain expertise, the article reveals how such collaborations yield measurable efficiencies and position organizations for sustained innovation in regulated sectors.

Legacy System Challenges and Strategic Partnership Formation

Cigna Healthcare, a prominent player in health insurance, operates across commercial, government, and international segments, serving millions through employer-based plans. The US employer division, central to this narrative, grappled with fragmented legacy architectures that hindered sales and onboarding agility. These systems, characterized by disparate data silos and manual interventions, protracted quote generation from days to weeks, impeding competitive responsiveness in a dynamic market.

The collaboration with TCS and AWS emerged from a shared recognition of these pain points. TCS’s domain proficiency in healthcare, coupled with AWS’s robust cloud ecosystem, facilitated a phased modernization under the AWS Migration Acceleration Program (MAP). This initiative aligned financial incentives with technical roadmaps, maximizing annual recurring revenue while offsetting migration costs. Initial assessments revealed opportunities for decommissioning outdated components, reducing technical debt without disrupting core operations.

Rajesh highlighted the emphasis on FinOps from inception, ensuring architectural reviews optimized service selection to avert over-provisioning. This proactive stance mitigated common pitfalls, such as escalating cloud spend, by embedding cost governance into design principles. The partnership’s structure—leveraging TCS’s industry insights and AWS’s scalable infrastructure—fostered a lower cost of change, enabling iterative enhancements rather than wholesale replacements.

Architectural Innovations and Implementation Methodologies

The resultant platform embodies a microservices-based, event-driven architecture on AWS, integrating services like Amazon API Gateway, Lambda, and DynamoDB for seamless data orchestration. This design supplanted rigid monolithic structures with modular components, allowing real-time quote processing and automated compliance checks. For instance, intelligent routing engines now direct requests across hybrid environments, ensuring data sovereignty and regulatory adherence under HIPAA.

Implementation adhered to MAP’s assess, mobilize, and migrate phases. The assess stage inventoried applications, prioritizing those with high business impact. Mobilization involved prototyping key workflows, such as client enrollment, using AWS Step Functions for orchestration. Migration executed via automated pipelines, transitioning workloads to Amazon Elastic Kubernetes Service (EKS) for resilience.

A pivotal innovation was the unified digital facade, providing stakeholders with intuitive interfaces via Amazon AppSync. This not only standardized capabilities but also embedded analytics for predictive insights, reducing onboarding timelines by forty percent. Bhavik Patel underscored the “bridge” concept—gradual capability reduction in legacy systems to ease adoption—alleviating business apprehensions around change.

Code sample depicting a simplified event-driven workflow in AWS Lambda for quote processing:

“`
import json
import boto3

dynamodb = boto3.resource(‘dynamodb’)
table = dynamodb.Table(‘Quotes’)

def lambda_handler(event, context):

Lecturer

Dustin Getz is the founder of Hyperfiddle and the primary architect behind Electric Clojure. His professional mission is to “collapse to zero” the cost of internal frontend development for business applications. With a background in distributed systems and concurrency, Dustin has spent years researching how to apply these low-level technical concepts to higher-level UI infrastructure. He is a frequent speaker at Clojure conferences, where he demonstrates how rethinking the computational structure of web applications can eliminate the complexities of traditional client-server architecture.

Abstract

Electric Clojure represents a paradigm shift in web architecture, moving away from the traditional separation of frontend and backend towards a unified, cloud-connected model. This article explores how Electric frames the user interface (UI) not as a series of manual network requests and state updates, but as a concurrency problem solvable through “differential dataflow”. By applying process supervision and structured concurrency, Electric achieves a UI = f(state) model that remains consistent across the network boundary. Through an analysis of Electric’s core principles, this article highlights how it addresses the “supervision problem” and optimizes network traffic through automatic differential updates, ultimately simplifying the development of complex, data-heavy applications.

Rethinking the Computational Structure of UIs

The fundamental premise of Dustin Getz’s work is that modern UI development is hindered by an incorrect computational structure. Most web applications rely on a manual bridge between the client (browser) and the server, involving REST or GraphQL APIs, manual state management (like Redux), and complex error handling for network failures.
Electric proposes a different structure based on differential dataflow. While React introduced the idea that the UI is a function of state, it is typically confined to the browser. Electric extends this concept across the entire network. In an Electric application, the code is written as a single, continuous program where the compiler automatically determines which parts of the code should run on the server and which should run on the client.

Solving the Network Boundary Problem

The “network boundary” is where most web applications fail or become overly complex. Electric addresses this by treating the connection between the client and server as a reactive data stream. Key innovations include:
1. Automatic Differential Updates: Instead of sending entire data objects over the wire, Electric only transmits the “diffs”—the specific changes in state. This high-frequency, high-fidelity communication is essential for maintaining a responsive UI in data-intensive environments.
2. Process Supervision: Electric applies Erlang-style supervision trees to UI components. If a server-side process fails or a network connection drops, the system can automatically recover and resume the UI state, solving the supervision problem that plagues traditional distributed web apps.
3. Reactive Back-pressure: By integrating differential dataflow, Electric manages the speed of data flow. If the client cannot keep up with the server’s updates, the system applies back-pressure to ensure the application remains stable and doesn’t “clobber” the browser’s resources.

Code Sample: Unified Frontend/Backend

(e/defn MyComponent [id]
  ;; This part runs on the server (accesses DB)
  (e/server
    (let [user (db/find-user id)]
      ;; This part runs on the client (renders UI)
      (e/client
        (dom/div
          (dom/h1 (str "Hello, " (:name user)))
          (dom/p "This data came directly from the DB!"))))))

Implications for Internal Tools and Enterprise Scaling

Dustin identifies a massive opportunity for this architecture in internal enterprise applications. Companies with thousands of microservices often struggle to build scalable UIs that can aggregate data from multiple sources. By placing an Electric “agent” in each microservice, developers can call backend Java or Clojure functions directly from a UI as if they were local calls.
This “UI Spreadsheet” model, as Dustin describes it, allows for native navigation of backend data structures. During his demonstration, he shows how a UI can query a git repository using the Java Git API (JGit) in real-time, displaying a git log through a scalable data browser without a single manually-written API endpoint. This approach collapses the traditional tiers of web development into a single, cohesive abstraction that composes across microservice boundaries.

Links:

• Electric Clojure: Differential Dataflow for UI (Lecture)
• Hyperfiddle Official Website
• Electric Clojure GitHub

Lecturer

Raquel Pau is a technical product manager at Broadcom (formerly VMware Tanzu). She brings extensive experience in Java developer tools, continuous-integration and continuous-delivery platforms, and internal developer platforms. Previously she worked as an engineering manager at Moderne, the company behind OpenRewrite, and held product-management roles at CloudBees focused on developer productivity. She has spoken at multiple Spring I/O editions as well as Devoxx, JavaConf and JavaZone. Her background combines deep technical knowledge of code-transformation tooling with product thinking about how large organizations can keep their application portfolios modern and consistent.

Abstract

Code modernization is not a single problem. Upgrading a Spring Boot application within the same major version, migrating from JAX-RS to Spring MVC, and rewriting a COBOL batch job into Spring Batch demand fundamentally different strategies. This article explores the taxonomy of modernization tasks proposed by Raquel Pau and the hybrid methodology that pairs OpenRewrite’s deterministic, type-aware recipes with the semantic reasoning power of large language models. Concrete demonstrations illustrate how upgrade plans are calculated from Maven metadata, how skills orchestrate recipe execution followed by LLM-driven semantic fixes, and how a structured DSL extracted from legacy code guides a full rewrite while preserving contracts and enabling incremental delivery.

Deterministic versus Non-Deterministic Transformations

Modernization tools fall into two broad categories. Deterministic tools always produce the identical output for a given input. Renaming a method, updating a package import, or replacing a deprecated Spring API are deterministic operations. OpenRewrite belongs to this category: it operates on a lossless semantic tree that retains type attribution obtained from the compiler, applies visitor-based recipes, and preserves the original formatting of the source. Because the transformation is deterministic, recipes can be unit-tested with high confidence and executed at scale across hundreds of repositories without surprise.

Non-deterministic problems admit many correct answers. Generating documentation, extracting the business intent of a filter, or inventing an idiomatic Spring Security configuration from a set of JAX-RS name-binding annotations are examples. Large language models excel here because they reason over patterns and can synthesize higher-level constructs that do not exist in the original code. The cost, however, is variability, the need for evaluation harnesses, and a tendency to hallucinate when internal libraries or proprietary APIs are outside the model’s training distribution.

OpenRewrite’s limitations are the mirror image of its strengths. It cannot perform runtime analysis; dependency injection and reflection mean that many object relationships become visible only after the application starts. It cannot invent new semantic abstractions; a mechanical translation of JAX-RS filters into Spring filters often leaves residual compilation errors or suboptimal configurations that require human or LLM insight. Cross-language migration is outside its design scope.

Coding agents partially compensate for these gaps by using pattern-based reasoning and by iterating until the project compiles. Yet they lack default type attribution, suffer from context-window constraints, and generate large volumes of tokens before reaching a stable state. The rational strategy is therefore hybrid: apply deterministic recipes first to shrink the problem, then invoke the LLM only for the residual semantic work.

Three Levels of Modernization

Pau organizes modernization into three progressively more demanding levels.

Upgrades remain inside the same framework family. A Spring Boot 3.3 application is moved to Spring Boot 4, simultaneously updating transitive dependencies such as Jackson and JUnit. Because Spring’s release train is not strictly linear and because organizations maintain internal frameworks with their own release cadences, a simple “latest version” recipe is insufficient. An upgrade-plan engine inspects Maven metadata, calculates a sequence of compatible intermediate steps, and emits a series of small, reviewable pull requests. Each step leaves the application in a buildable state. Tanzu’s Application Advisor exposes this capability via the cf repo upgrade plan and cf repo apply upgrade plan commands, demonstrating that continuous, low-risk upgrades can be embedded in CI pipelines.

Migrations change the underlying framework while preserving language and runtime. The canonical example is Jakarta JAX-RS to Spring Boot. Name-binding annotations that attach filters to resources have no direct counterpart; authentication filters must become Spring Security configurations; repositories must acquire @Repository annotations. The recommended skill therefore first executes the OpenRewrite recipes that perform the mechanical rewrite and any accompanying Spring Boot upgrade, then hands control to the coding agent to resolve remaining compilation errors and to map name-binding semantics onto Spring constructs. The result is both more complete and far less expensive in tokens than asking an unconstrained LLM to rewrite the entire application.

Full rewrites discard the original implementation while preserving contracts. A COBOL batch program that sorts records by date and amount must become a Spring Batch job that reads the same input format, produces identical output, and respects the same database schema if one is involved. Because legacy systems rarely possess comprehensive tests, the process begins by extracting a catalog of user stories, then a structured domain-specific language description of inputs, outputs, and processing steps. Only after the human reviewer validates the generated tests and the semantic model does the agent emit Spring code, typically seeded by a skeleton obtained from start.spring.io. Incremental delivery is essential: large monolithic rewrites cannot be reviewed or risk-managed in a single step.

Orchestrating OpenRewrite and LLM Agents

Three integration mechanisms allow a coding agent to invoke OpenRewrite without saturating its context window. Local MCP servers expose the rewrite CLI so that only the command and its concise output enter the conversation. Skills package the same CLI invocation and are loaded only when the agent decides the skill is relevant. Prompts can be registered with a remote MCP server, yet they must be fully present in every conversation and therefore scale poorly for complex migrations.

The hybrid skill for a JAX-RS migration therefore looks roughly as follows: calculate the upgrade plan that includes the JAX-RS recipes, execute the recipes, collect residual compilation diagnostics, and finally apply semantic transformations that replace name-binding filters with Spring Security and Spring MVC constructs. Because the deterministic phase has already performed the bulk of the mechanical work, the LLM operates on a far smaller residual problem and produces higher-quality results.

For full rewrites the skill is organized into three explicit phases. Phase one extracts a user-story catalog and stores it under version control so that subsequent runs reuse the analysis. Phase two materializes a structured DSL for a chosen story, including acceptance criteria, data models, and external contracts. Phase three generates the Spring implementation and correlating tests. Human validation remains mandatory; the agent cannot be trusted to invent missing requirements or to decide whether an original implementation was correct.

Practical Demonstrations and Organizational Implications

In the upgrade demonstration a Spring Petclinic application on Boot 3.3 is analyzed; the engine proposes coordinated upgrades of Spring Boot, Jackson and JUnit; successive apply steps produce small, reviewable diffs that leave the project green after each commit. In the migration demonstration a pure JAX-RS Petclinic is transformed: OpenRewrite rewrites the bulk of the code, the agent resolves compilation issues caused by signature changes, and name-binding annotations disappear in favor of proper Spring Security configuration. In the rewrite demonstration a simple COBOL sorter is analyzed, a single user story and its DSL are generated, a Spring Batch project is scaffolded, and the resulting executable produces byte-for-byte identical output.

The organizational payoff is standardization. When every application can be moved to a common Spring Boot baseline with low friction, teams share libraries, security configurations and operational practices. Token consumption drops dramatically because deterministic recipes eliminate the majority of mechanical work. Evaluation of non-deterministic skills becomes feasible because the residual problem set is smaller and more homogeneous.

Conclusion

Modernization success depends on matching the tool to the nature of the transformation. OpenRewrite supplies precision, testability and scalability for deterministic changes. Large language models supply the semantic insight required for migrations and rewrites. A carefully designed hybrid that keeps the LLM outside the hot path of routine upgrades, that constrains its context to residual problems, and that forces explicit contracts for full rewrites yields both higher quality and lower cost. Organizations that adopt this disciplined approach can keep large application portfolios current without sacrificing reviewability or operational safety.

Links:

• Video of the lecture
• Slides
• Raquel Pau on X
• Raquel Pau on GitHub
• Session page
• OpenRewrite
• Broadcom Tanzu