Jonathan Lalou's Blog

Lecturer

Sebastian Wallkötter is an AI builder and open-source maintainer. He holds a doctorate in social robotics and computer science from Uppsala University and maintains the widely used image-processing library Image.io, which records tens of millions of monthly downloads. After working as a product manager for a data platform at a Scandinavian media company he relocated to Boston; his current focus is the architectural patterns that underlie contemporary generative-AI systems.

Abstract

Beneath the proliferation of generative-AI frameworks lies a single, almost trivial control structure: a nested pair of while loops. The outer “turn” loop accumulates conversation history; the inner “trace” loop repeatedly invokes a language model until it ceases to request tools. This article dissects that core loop, shows how its presence or absence yields completions, chat assistants, agents, and co-pilots, and then examines the complementary discipline of context engineering—the principled management of the information that enters the model’s window. Together these two ideas supply a first-principles map that allows practitioners to construct sophisticated systems without mandatory dependence on any particular framework.

The Core Loop and Its Four Constituents

Every language-model call is defined by four elements: the chosen model, a system prompt that supplies instructions, a history of prior messages, and a list of available tools. History itself is an ordered sequence of typed messages—user, thinking, tool-call, tool-result, and assistant—whose legal transitions are strictly constrained. A user message is followed by an assistant message (or by intermediate thinking and tool activity); tool calls are answered by tool results; the entire block may repeat.

When these constituents are assembled into application logic the resulting control flow is remarkably simple. After initial setup the program waits for a user message (the turn loop). Upon receipt it enters an inner loop that repeatedly calls the model, executes any requested tools, appends the tool results to history, and continues until the model finally emits an assistant message (the trace loop). That assistant message is returned to the user, history is retained, and the outer loop resumes. The pattern is deliberately boring; its power derives from the fact that nearly every generative application is a variation on these two loops.

Taxonomy of Applications Derived from the Loop

Absence of both loops yields ordinary completions—single-shot tasks such as sentiment classification or automated replies. Presence of the turn loop alone produces a chat assistant capable of multi-turn conversation but unable to act on the external world. Presence of the trace loop alone produces a classical agent that can call tools yet does not accumulate conversational history across independent invocations. Presence of both loops yields what may be termed a co-pilot: an interactive assistant that both converses and acts, typical of coding agents and desktop assistants.

Because completions and pure agents possess clear start and end points they can themselves be wrapped as tools. An agent that calls other agents as tools is the elementary building block of multi-agent systems. Two dominant organizational patterns emerge. Workflows simply daisy-chain agents; a supervisor agent treats subordinate agents as ordinary tools and remains unaware of their internal structure. Reflection (an actor-critic arrangement) improves output quality by iteratively critiquing and revising intermediate results. All of these configurations remain intelligible once the underlying turn and trace loops are understood.

Context Engineering: Programming the Window

Context engineering is the disciplined practice of deciding what information enters the model’s context window so that the system remains reliable without unnecessary cost. Language models care primarily about the immediate next action—which tool to invoke or which assistant message to emit. Any material that does not contribute to that decision is surplus.

A practical system prompt therefore contains two principal sections: an ordered list of instructions and a compact knowledge base organized by descriptive headings. Tools are supplied separately, commonly via the Model Context Protocol. Contrary to frequent advice against “mega-prompts,” contemporary models tolerate surprisingly large static contexts; a window of one to two hundred thousand tokens is roughly the length of a short novel and is often sufficient for an entire application. Premature fragmentation should therefore be resisted until empirical evidence of degradation appears.

When static prompts eventually become untenable, four failure modes—collectively the “deadly sins”—signal the need for lazy loading. Context overload occurs when a long trace of prior tool calls teaches the model to repeat the same action indefinitely. Context poisoning arises when the window contains stale or incomplete information about the external world. Useless data appears when instructions or tools irrelevant to the current task remain visible and are erroneously followed. Inconsistency arises when contradictory facts coexist and occasionally trigger erratic behavior. Each of these pathologies is silent; the model simply begins to fail without raising an exception.

Lazy loading restores reliability by injecting material only at the moments it becomes necessary: at agent initialization for known constants, after the user message for information deducible from the request, and during tool execution for previously unknown dependencies. Retrieval-augmented generation, skills, sub-agents, and memory are simply named strategies for performing that lazy loading—of knowledge, of instructions, of tools, and of personalization data, respectively.

Implications for Design and Standardization

Understanding the core loop and the principles of context engineering liberates practitioners from framework lock-in. Systems can be constructed directly against model APIs, instrumented with ordinary logging, and evolved by the gradual introduction of retrieval, skills, or multi-agent coordination only when required. Standardization efforts such as the Model Context Protocol are valuable insofar as they lower the cost of experimentation, yet the field remains young; excessive early standardization risks freezing patterns that have not yet been fully explored. The most robust path is therefore to master the simple loop, manage context with deliberate frugality, and introduce complexity only when measurement demonstrates its necessity.

Links:

• Lecture video

Lecturer

William Montaz is a technical leader at Criteo, where he tackles complex production issues in large-scale distributed systems. With extensive experience in machine learning infrastructure and data pipelines, he emphasizes methodical investigation techniques and cognitive bias awareness in high-stakes debugging scenarios.

Abstract

Critical incidents trigger immediate pressure: inconsistent data, unpredictable behaviors, and alarmed stakeholders. The instinctive response often involves blaming the most recent change or following intuition, frequently without meaningful progress. William Montaz guides participants through a genuine technical investigation where Apache Spark serves merely as context for a deeper exploration of methodology. The session demonstrates constructing systematic inquiries in unfamiliar territory, avoiding confirmation bias, breaking down obscure problems into testable hypotheses, and distinguishing correlation from causation. Through this concrete case of corrupted Spark job outputs during a datacenter migration, attendees learn practical skills for effective production troubleshooting.

The High-Stakes Nature of Production Incidents

Production environments demand rapid, accurate diagnosis under intense pressure. When jobs begin producing corrupted data—duplicate rows alongside missing entries—stakeholders naturally seek quick explanations. The temptation to attribute issues to recent changes or visible symptoms often leads investigations astray. Effective troubleshooting requires disciplined methodology rather than reactive speculation.

In large-scale data platforms like Criteo’s, where millions of containers process petabytes daily, incidents can cascade rapidly. Understanding both technical details and cognitive pitfalls becomes essential for resolution.

MapReduce and Spark: Foundational Concepts for the Investigation

To appreciate the case, familiarity with core distributed processing concepts proves valuable. MapReduce provides a framework for parallel computation across large datasets. The classic word count example illustrates the pattern: a map phase extracts key-value pairs (words as keys, counts as values), followed by shuffling to group related keys, and a reduce phase aggregates results.

Spark builds upon these principles with optimizations for iterative processing and in-memory computation. Jobs compile into directed acyclic graphs (DAGs) of stages, where each stage typically ends with a shuffle operation. This architecture enables fault tolerance through recomputation of failed tasks but introduces complexities around data consistency during retries.

Understanding these mechanics helps frame hypotheses about potential failure modes in distributed environments.

The Incident: Corrupted Data During Datacenter Migration

The reported issue involved Spark jobs producing inconsistent outputs with both missing and duplicated rows. The problem appeared intermittently, resolving after multiple retries. Context proved crucial: the organization was migrating its massive compute infrastructure between datacenters under tight timelines.

Initial observations suggested possible connections to the migration. Data replication across sites introduced network latency and potential consistency challenges. However, jumping to conclusions risked confirmation bias. Multiple hypotheses required systematic evaluation.

One theory posited replication inconsistencies between datacenters causing read anomalies. Another focused on increased disk pressure from migration activities leading to timeouts in shuffle operations. A third examined custom components like Kyuubi and AQ used for performance optimizations in the final job stages.

Systematic Investigation: Avoiding Cognitive Biases

Effective diagnosis demands awareness of common cognitive traps. The availability heuristic might overemphasize recent migration activities. Authority bias could discourage questioning framework behavior. Confirmation bias risks pursuing convenient explanations while ignoring contradictory evidence.

The investigation proceeded by gathering concrete data rather than speculating. Reproduction attempts in controlled environments revealed patterns: corruption occurred specifically during partial recomputes of the final stage under certain timing conditions. This narrowed focus to repartition operations and shuffle mechanics.

Code examination and instrumentation provided further clarity. Custom logging and Redis-based tracking of hash codes during repartitioning exposed non-deterministic behavior in floating-point aggregations. Order-dependent summation of floats, combined with retry mechanisms, produced the observed duplicates and omissions.

Root Cause Analysis and Resolution

The true cause emerged from careful observation of Spark’s internal repartition logic. Non-deterministic shuffle ordering during retries, combined with floating-point summation sensitivities, created inconsistent aggregations. The system assumed deterministic behavior across recomputations, an assumption violated under failure conditions.

Resolution involved enforcing deterministic partitioning and explicit sorting before aggregations. This fix eliminated the corruption while maintaining performance characteristics.

The case highlights that symptoms (duplicates, missing rows) traced back through multiple layers: migration pressure, shuffle timeouts, retry mechanics, and subtle numerical properties. Premature conclusions would have missed this chain.

Key Lessons for Production Troubleshooting

Several principles emerge from this investigation. Avoid premature extrapolation and maintain intellectual humility regarding framework perfection. Actively seek disconfirming evidence rather than supporting data. Leverage general knowledge and orders of magnitude to identify anomalies. Reading source code, when accessible, provides invaluable insights.

Presenting partial findings to colleagues surfaces blind spots and strengthens reasoning. Treat production systems with appropriate respect for their complexity while systematically eliminating hypotheses.

These approaches scale beyond Spark to any distributed system. They transform troubleshooting from art to disciplined practice.

Conclusion

Production incidents test both technical knowledge and cognitive discipline. By following methodical investigation practices, acknowledging biases, and pursuing root causes through evidence rather than assumption, teams can resolve complex issues effectively. The Spark corruption case during datacenter migration exemplifies how apparent framework problems often trace to subtle interactions requiring patient, thorough analysis. These skills prove invaluable for maintaining reliable large-scale systems.

Links:

• Video: Production Troubleshooting : boostez vos skills, une étude de cas

Lecturer

Tomori is a Senior Specialist Solutions Architect at Amazon Web Services (AWS), specializing in high-performance computing (HPC) and distributed machine learning. Tomori’s research and professional focus center on the “physics” of training—optimizing the interaction between silicon, networking fabrics, and software frameworks to enable the training of models with trillions of parameters.

Abstract

Training state-of-the-art AI models has become a monumental engineering challenge that requires the precise orchestration of massive compute clusters. This article explores the technical foundations of training high-performing AI models at scale using AWS’s purpose-built infrastructure. It provides an in-depth analysis of Amazon SageMaker HyperPod and its role in distributed training, covering essential techniques such as data and model parallelism. The discussion highlights the shift from managed APIs to Kubernetes-based orchestration (EKS) and Slurm for high-performance job scheduling. Key innovations in automated model tuning, observability through Prometheus and Grafana, and efficient checkpoint management are detailed, showing how these strategies can reduce training time by up to 40% while optimizing costs for the next generation of foundation models.

Scaling Challenges in the Era of Foundation Models

The shift toward foundation models has fundamentally changed the requirements for AI training infrastructure. When a model’s memory footprint exceeds the capacity of a single GPU, developers must move beyond simple training scripts to sophisticated distributed systems. The challenges are three-fold: compute intensity, memory constraints, and networking bottlenecks.

At scale, the “efficiency of the cluster” becomes the primary metric. Organizations must navigate the trade-offs between different types of parallelism—Data Parallelism, where the dataset is split across nodes, and Model Parallelism (including Pipeline and Tensor parallelism), where the model itself is distributed. AWS addresses these challenges by providing a specialized infrastructure layer, specifically through P5 instances powered by NVIDIA H100 GPUs and high-bandwidth Elastic Fabric Adapter (EFA) networking. This hardware foundation is critical for maintaining the high-speed communication necessary for synchronization during the backward pass of training.

Orchestration Choice: Kubernetes vs. Slurm

A critical decision for any organization training at scale is the choice of orchestrator. AWS offers two primary paths for managing SageMaker HyperPod clusters, each catering to different operational philosophies.

1. Kubernetes (Amazon EKS): This is often the preferred choice for organizations that have adopted a cloud-native DevOps culture. Kubernetes provides a robust API for managing containers, and when integrated with SageMaker HyperPod, it allows for the orchestration of ML training jobs alongside other microservices. This path benefits from a rich ecosystem of observability tools like Prometheus and Grafana.
2. Slurm: Traditionally used in high-performance computing (HPC) environments, Slurm is a highly efficient job scheduler designed for massive, tightly coupled workloads. Organizations coming from research backgrounds or those requiring the absolute lowest overhead often choose Slurm.

Regardless of the orchestrator, SageMaker HyperPod provides the underlying “resiliency layer,” which includes automated node health monitoring and the ability to replace failing instances without human intervention. This ensures that the long-running training “epochs” are not disrupted by hardware variability.

Parallelism Strategies and Performance Optimization

Achieving a 40% reduction in training time requires more than just more GPUs; it requires optimized parallelism strategies. The discussion centers on three primary methodologies:

• Data Parallelism: The most common approach, where each GPU processes a different batch of data and gradients are synchronized.
• Tensor Parallelism: Splitting individual layers across multiple GPUs to handle extremely wide layers.
• Pipeline Parallelism: Dividing the model into stages and processing them in a “pipeline” fashion to maximize GPU utilization.

SageMaker’s distributed training libraries automate much of this complexity. By injecting “sharding” logic into the training script, these libraries allow developers to scale their models with minimal code changes. Furthermore, the use of automated model tuning—which systematically explores the hyperparameter space—ensures that the resulting model reaches its peak performance without manual “trial and error.”

Code Sample: Configuring Model Parallelism in SageMaker

from sagemaker.pytorch import PyTorch

'''
Configuration for SageMaker Model Parallelism (SMP)
'''
smp_options = {
    "enabled": True,
    "parameters": {
        "pipeline_parallel_degree": 2,
        "tensor_parallel_degree": 4,
        "optimize": "speed",
        "ddp": True
    }
}

'''
Training job configuration using HyperPod
'''
estimator = PyTorch(
    entry_point='train_llm.py',
    instance_type='ml.p5.48xlarge',
    instance_count=16, # Scaling to 128 GPUs
    distribution={"smdistributed": {"modelparallel": smp_options}},
    role=sagemaker.get_execution_role(),
    framework_version='2.1'
)

estimator.fit()

Observability and Fault Tolerance

In a cluster of hundreds of GPUs, “observability” is not a luxury—it is a requirement. Tomori emphasizes the importance of a centralized observability stack using Prometheus and Grafana. These tools allow engineers to monitor GPU utilization, memory pressure, and network throughput in real time. If a node begins to underperform or “throttles” due to heat, the observability stack provides the alerts necessary for proactive management.

Coupled with observability is the concept of efficient checkpoint management. By utilizing high-performance storage like Amazon FSx for Lustre, SageMaker HyperPod can save model “weights” and optimizer states rapidly. In the event of a failure, the system can reload the most recent checkpoint and resume training within minutes. This “resilient training” loop is what enables the massive foundation models of today to be trained in weeks rather than months.

Conclusion

Training high-performing AI models at scale is as much an infrastructure challenge as it is an algorithmic one. Through the integration of SageMaker HyperPod, optimized parallelism libraries, and robust orchestration via EKS or Slurm, AWS has created an environment where “the physics of scaling” can be mastered. By reducing training times and automating resiliency, these tools empower organizations to push the boundaries of what is possible, turning the promise of trillion-parameter models into a repeatable, cost-effective reality.

Links:

• Amazon SageMaker HyperPod Resource
• AWS Training Jobs GitHub Examples
• Distributed Training Libraries on AWS

Lecturers

Marit van Dijk is a highly regarded Java Champion, Docker Captain, and Developer Advocate specializing in IDE productivity enhancements, Kotlin ecosystem contributions, and making advanced topics approachable for broad audiences. Piotr Przybył serves as Senior Developer Advocate at Elastic, holding Java Champion and Docker Captain distinctions. He is widely recognized for deep explorations of Java performance characteristics, language evolution, and practical development methodologies.

Abstract

Java’s consistent release rhythm introduces valuable innovations that can sometimes feel daunting to integrate, particularly for developers working with established codebases or transitioning from prior versions. Marit van Dijk and Piotr Przybył present an engaging, experimentation-focused approach to adopting contemporary Java capabilities. Through extensive live coding demonstrations within IntelliJ IDEA, they cover a rich selection of features including compact source files, enhanced records, stream gatherers, structured concurrency primitives, vector API computations, and documentation improvements. The session emphasizes immediate practical value, IDE assistance, and the joy of discovery.

The Motivation for Staying Current with Java Evolution

Java maintains a predictable six-month release cycle, consistently delivering enhancements across performance optimizations, concurrency abstractions, security improvements, and developer experience refinements. While Long-Term Support (LTS) releases provide stability crucial for risk-averse enterprises requiring extended commercial backing, the speakers encourage thoughtful consideration of intermediate versions for teams positioned to capitalize on their advancements.

They readily acknowledge legitimate concerns around upgrade processes, dependency compatibility, and organizational inertia. However, they highlight maturing ecosystem support—including comprehensive IDE tooling, community resources, and backward compatibility efforts—that significantly eases transitions. The overarching message promotes proactive engagement rather than passive waiting, positioning Java as a continually improving platform rather than a static legacy technology.

Learning modern Java is portrayed as an active, playful process reminiscent of childhood discovery. This perspective transforms potentially intimidating language additions into opportunities for creative exploration and immediate productivity gains, supported powerfully by modern development environments.

Syntactic Refinements: Compact Sources, Records, and Module Imports

Java 25’s introduction of compact source files and instance main methods marks a notable step toward reduced ceremony. Traditional requirements for explicit class declarations and static main entry points are relaxed, enabling concise scripts ideal for learning exercises, rapid prototyping, command-line utilities, and exploratory development. This change particularly benefits educational contexts and situations requiring quick experimentation.

Records, stabilized since Java 16, function as transparent, immutable carriers of data. They automatically provide canonical constructors, accessor methods, sensible equals, hashCode, and toString implementations, along with pattern matching support. Perfect for DTOs, value objects, and request/response models, records substantially reduce boilerplate and associated error opportunities. Developers retain the ability to customize behavior through explicit overrides when domain requirements demand specialized semantics.

Module imports offer further conciseness by permitting declarations such as import module java.base;, which encompasses the foundational JDK packages. This eliminates lengthy lists of individual import statements while preserving code clarity and avoiding namespace pollution. The IO utility class simplifies basic console input/output operations, replacing more verbose patterns involving Scanner and direct System.out usage.

Collectively, these syntactic improvements reflect Java’s ongoing commitment to balancing power with approachability, making the language more welcoming without compromising its industrial strengths.

Expressive Data Processing: Stream Gatherers and Parallel Operations

The stream gatherers feature, finalized in Java 24, significantly expands the expressiveness of the Streams API. Developers can now define custom intermediate operations that encapsulate complex processing logic, replacing ad-hoc combinations of filter, map, flatMap, and similar methods. This promotes creation of reusable, composable, and semantically rich pipelines suited to sophisticated data transformations.

A particularly valuable gatherer, mapConcurrent, enables safe parallel processing by leveraging virtual threads, avoiding traditional fork-join pool exhaustion issues common with parallel() streams in certain contexts. The speakers demonstrate practical applications ranging from straightforward data enrichment to more involved computations, highlighting improved readability and maintainability.

Concurrency Paradigms: Structured Scopes and Lazy Initialization

Structured concurrency, available as a preview feature in Java 26, introduces a more disciplined approach to managing concurrent operations. By structuring tasks within a StructuredTaskScope, developers achieve clearer code organization, automatic resource cleanup, and built-in cancellation semantics. This represents a substantial improvement over error-prone patterns relying on CompletableFuture compositions, reducing risks of thread leaks and orphaned computations.

Lazy constants provide controlled deferral of initialization until first access, contributing to faster startup times and more efficient resource utilization. Expanded pattern matching capabilities for primitive types enhance the power and readability of switch expressions and type checks across broader scenarios.

Performance Acceleration: Vector API and Documentation Innovations

Project Panama’s Vector API unlocks access to hardware vector instructions (SIMD), enabling significant performance improvements for data-parallel workloads without requiring low-level assembly coding. Through high-level Java abstractions, developers can express vectorized operations on arrays and other data structures, achieving throughput gains particularly beneficial in scientific computing, financial modeling, image processing, and machine learning applications.

The speakers present concrete benchmarks comparing scalar and vectorized implementations, illustrating real-world speedups. Documentation enhancements, specifically Markdown support within Javadoc comments introduced in Java 23, modernize API documentation practices. Rich formatting capabilities render effectively in contemporary IDEs, reducing the maintenance burden historically associated with HTML-based Javadoc.

Ecosystem Synergies and Forward Outlook

The presentation extends beyond core language features to examine synergies with development tools, community resources, and evolving practices. Modern IDEs like IntelliJ IDEA provide sophisticated support for preview features, refactoring assistance, and code analysis that accelerate adoption. Community engagement through user groups, conferences, and open-source contributions further enriches the learning experience.

The speakers encourage deliberate experimentation within personal or side projects as a low-risk pathway to building confidence before enterprise-wide rollout. They stress the importance of updating not merely version numbers but also mental models and coding idioms to fully leverage Java’s capabilities.

Java’s ongoing development demonstrates its continued relevance as a powerful, productive platform for contemporary software engineering. By embracing curiosity and playful exploration, developers can harness these advancements to enhance both personal satisfaction and professional effectiveness.

Links:

• Video: Learning modern Java the playful way
• Marit van Dijk Website
• Piotr Przybył Website
• OpenJDK Project Amber
• Java 25 Release Notes
• JEP Catalog

In the digital domain, where inclusion is imperative yet often ignored, Elise Kristiansen, a senior front-end savant at Opa, exposes the exigencies of accessibility. With Norway’s mandates for universal design entrenched for years, Elise enumerates ten traps thwarting equitable access, offering actionable antidotes. Her lightning discourse, brisk yet brimming, beckons builders to bolster barriers, ensuring digital doors open for all.

Elise opens with urgency: WCAG’s wisdom, legislated for inclusivity, languishes as new sites sideline the sightless, the deaf, the distinct. Her mission: awaken awareness, arming architects with insights to integrate accessibility from inception.

Technical Traps: Text and Multimedia Missteps

Elise’s initial indictments: absent alt text for images, missing captions for multimedia—blind and deaf users barred from content. Her remedy: wield scanners—Pa11y pinpoints omissions—then craft concise, contextual descriptions. Videos demand subtitles, audio begs transcripts—ensuring equivalence for all.

Keyboard navigation, another nemesis, neglects non-mouse navigators. Elise urges: ensure focus flows logically—tab orders tailored, skip links supplied—empowering keyboard-bound users to traverse terrains unhindered.

Design and Development Deficits: Planning Pitfalls

Design’s disregard dooms accessibility: color contrasts collapse, focus indicators fade. Elise’s exhortation: adopt ARIA landmarks, enforce contrast ratios—WCAG’s 4.5:1 for text, 3:1 for graphics—ensuring visibility for visually impaired viewers.

Development’s delay—accessibility as afterthought—derails deadlines. Elise advocates: embed accessibility in estimates, engage all stakeholders—customers, designers, editors—in inclusive intent from planning’s outset.

Awareness and Economics: Overcoming Obstacles

Ignorance impedes: developers and designers, untrained, overlook accessibility’s essence. Elise’s solution: champion courses—hours invested yield inclusive interfaces. Her final foe: economic excuses, where overlays masquerade as solutions. She counters: simplicity scales—minimalist designs, sans compromise, serve all.

Elise’s entreaty: prioritize people, not profits—universal design unites, exclusion erodes.

Links:

• Opa company website
• Elise Kristiansen on LinkedIn

Lecturer
Anton Arhipov is a Developer Advocate at JetBrains, where he engages with developers and shares insights on Kotlin, IntelliJ IDEA, and related technologies. He frequently speaks at conferences and contributes to JetBrains initiatives around AI tooling and agent frameworks.

Abstract
Anton Arhipov’s session at Voxxed Days Amsterdam 2026 introduces the construction of AI agents directly on the JVM using Koog, a lightweight Kotlin-native framework developed at JetBrains. Rather than relying on Python-based solutions, the talk demonstrates building tool-using LLM agents in Kotlin, emphasizing seamless integration with existing JVM products and domain models. Through live coding and detailed examination of core concepts—including communication with language models, tool registration, interaction loops, and the distinctive graph-based strategy DSL—participants gain insight into creating robust, reproducible agent workflows. The presentation culminates in a coding agent example that generates, compiles, and tests small programs, highlighting both opportunities and challenges when working with local and remote LLMs.

Why Build AI Agents on the JVM? Context and Motivations

The emergence of large language models has spurred interest in autonomous AI agents capable of tool use and iterative reasoning. While Python has historically dominated this space, Anton argues that language choice should serve the embedding context. For JVM-based products such as IntelliJ IDEA, TeamCity, or YouTrack, implementing agents in Python introduces significant overhead: external process management, serialization costs, and duplication of domain models. A native JVM solution reuses existing infrastructure, types, and libraries, improving efficiency and maintainability.

Koog originated from internal JetBrains efforts to embed agents into their tools. The framework’s tech lead documented the transition from Python to Kotlin, sparking discussions around perceived “skill issues” versus genuine architectural advantages. Anton stresses that the decision prioritizes integration over language fashion.

Fundamentals of Agent Architecture and the Interaction Loop

At a high level, an AI agent comprises an environment, a strategy (or “brain”), model providers, and user interaction. The workflow begins with a user request. The agent forwards a prompt to an LLM via a prompt executor (supporting OpenAI, Google, Ollama, and others). The model may respond with:

• An assistant message (plain text, terminating the loop), or
• A tool call, prompting the agent to invoke registered tools, feed results back to the LLM, and iterate until an assistant message appears.

This forms the agentic loop. Koog models agents as simple classes with a tools registry, configurable strategy, prompt, model selection, iteration limits, and a prompt executor. Predefined models and capability checks (e.g., tool calling support) simplify setup.

Anton demonstrates a basic happy-path agent that writes a Kotlin program (initially a “Hello World,” later a Snake game using Java Swing). Early iterations reveal the need for tools: file system operations, shell execution via Gradle, and careful prompting to enforce compilation and verification steps. Live runs expose humorous LLM behaviors, such as generating wrapper files or hallucinating results, underscoring the importance of robust system prompts, validation, and verification.

Tool Integration, Local Models, and Observed Challenges

Tools in Koog can be registered via a registry or supplied as annotated functions with descriptions for better LLM guidance. The session explores interactions with local models (e.g., small Mistral variants via Ollama) versus remote APIs. Local models offer speed and privacy but exhibit unreliability in tool calling—hallucinating results, ignoring descriptions, or fabricating outputs when tool results appear implausible (such as unrealistic temperature values in Kelvin).

Examples illustrate how insufficient tool descriptors or conflicting instructions lead to unexpected behavior. Anton shows cases where models call tools multiple times, reject their own outputs, or produce inconsistent results across runs, even with identical prompts. These observations highlight that reliable tool calling remains critical for effective agents and often requires significant prompt engineering and workflow safeguards.

The Strategy DSL: Graph-Based Control for Determinism and Reproducibility

A standout feature of Koog is its strategy DSL, which models agent behavior as a reproducible graph rather than a purely LLM-driven loop. The default strategy handles basic LLM calls and tool invocation, but custom strategies allow sophisticated workflows.

Nodes represent actions (e.g., calling the LLM, processing tool results), while edges use predicates or conditions (e.g., “if tool call, proceed to execution node”). Graphs support sequencing, branching, and chaining multiple strategies—such as enforcing a planning stage before implementation and verification. Type safety in Kotlin ensures nodes and edges compose correctly, with the compiler assisting in workflow construction.

Anton walks through building a simple parsing-and-increment graph, then extends the concept to more realistic scenarios involving structured outputs for plans. By validating plans against domain models and potentially generating dynamic sub-strategies, developers introduce determinism into otherwise stochastic LLM processes. Structured outputs improve reliability when deserializing LLM responses into typed objects like plan items.

This graph approach distinguishes Koog from many other frameworks, enabling complex, auditable, and testable agent logic while leveraging Kotlin’s strengths.

Live Coding a Coding Agent and Practical Takeaways

The session features extensive live exploration, including attempts to generate a Snake game with requirements for Gradle, specific versions, compilation verification, and no mistakes. Successes and failures illustrate real-world agentic challenges: hallucinated tool calls, skipped verification steps, and the need for precise instructions. Anton shares that even with “make no mistakes” in the prompt, outcomes vary across models and runs.

The talk concludes with a repository of examples and reiterates Koog’s multi-platform nature, adapter support for various model providers, and the power of the strategy DSL. While 45 minutes cannot cover advanced topics, the presentation equips attendees with foundational knowledge to experiment with building JVM-native agents.

Code Sample: Basic Agent Setup in Koog (Kotlin)

val executor = OpenAI.promptExecutor { /* config */ }
val agent = Agent {
    promptExecutor = executor
    systemPrompt = "You are a helpful Kotlin developer..."
    model = OpenAIModels.GPT_4O
    tools = toolRegistry { 
        include(FileSystemProvider())
        include(ShellExecutor())
    }
}

agent.run("Write a Kotlin program to print Hello World")

Code Sample: Simple Strategy Graph

val strategy = strategy<String, Int> {
    start forwardingTo parseNode forwardingTo incrementNode
}

val parseNode = node { input: String -> input.toInt() }
val incrementNode = node { input: Int -> input + 1 }

These examples capture the framework’s emphasis on readability and type-safe composition.

In summary, Anton Arhipov provides a grounded, pragmatic overview of constructing AI agents on the JVM. Koog enables developers to harness LLM capabilities within familiar Kotlin and JVM ecosystems, with the strategy DSL offering a powerful mechanism for controlling complexity. While challenges with model reliability persist—especially locally—the framework’s design promotes integration, reproducibility, and incremental improvement of agentic systems.

Links:

• Video URL
• Koog Framework Information
• Anton Arhipov on JetBrains Blog

Lecturer

Victor Uria Valle is a Full Stack Software Engineer at CERN, specializing in modernizing and developing enterprise applications to support laboratory operations. He holds a degree from Universidad de Oviedo and pursues personal projects in artificial intelligence, including game-playing agents. Relevant links include his LinkedIn profile (https://ch.linkedin.com/in/vuria).

Abstract

This article scrutinizes Victor Uria Valle’s endeavor to construct a Deep Q-Network (DQN) in Java for mastering the board game Azul. It delineates the game’s mechanics, Q-learning fundamentals, neural network integration, and training methodologies. Through analytical exploration, it assesses environmental modeling, state representation, reward engineering, and performance implications, underscoring Java’s viability for reinforcement learning without specialized frameworks.

Contextualizing Azul and Reinforcement Learning

Victor commences by elucidating Azul’s rules, a tile-placement game involving a shared board with factories displaying colored tiles and a central area. Players select tiles of one color from a factory or center, placing them on their personal board’s pattern lines—a right-angled triangular grid. Completed lines transfer tiles to a wall, scoring points based on adjacency, while excess tiles incur penalties on the floor line. Code structures reflect this: enums define tile colors, factories hold tile lists, player boards track pattern lines, walls, floor lines, scores, and penalties. Legal moves constrain placements—tiles must match line colors, avoid filled wall positions, and handle overflows. This setup frames reinforcement learning (RL), where agents learn via trial-and-error interactions. Victor employs Q-learning, estimating action values (Q-values) in states to maximize cumulative rewards. Traditionally tabular, Q-learning suits discrete spaces but falters in Azul’s vast state space (exponential due to board configurations). Contextually, AI in games traces to chess and Go, but Victor opts for Java sans PyTorch or TensorFlow, demonstrating accessibility. Implications: Democratizes RL for Java developers, though lacks optimized libraries.

Deep Q-Network Architecture and Innovations

To address state complexity, Victor integrates deep neural networks, approximating Q-values via function approximation. The DQN processes states as inputs, outputting Q-values per action. State encoding vectorizes boards: binary representations for tile presence in factories, centers, pattern lines, walls, and floors. Actions encode selections (factory/center, color, line/floor). The neural network, a multi-layer perceptron, uses rectified linear units (ReLU) for non-linearity, trained via backpropagation with mean squared error loss. Innovations include experience replay—storing state-action-reward-next-state tuples in a buffer, sampling mini-batches to break correlations—and a target network, periodically updated to stabilize learning. Epsilon-greedy policy balances exploration (random actions) and exploitation (highest Q-value actions), decaying epsilon over episodes. Code snippets illustrate:

public double[] forward(double[] input) {
    // Input to hidden layer
    double[] hidden = new double[hiddenSize];
    for (int i = 0; i < hiddenSize; i++) {
        double sum = 0;
        for (int j = 0; j < inputSize; j++) {
            sum += input[j] * weightsInputHidden[j][i];
        }
        hidden[i] = Math.max(0, sum + biasHidden[i]); // ReLU
    }
    // Hidden to output
    double[] output = new double[outputSize];
    for (int i = 0; i < outputSize; i++) {
        double sum = 0;
        for (int j = 0; j < hiddenSize; j++) {
            sum += hidden[j] * weightsHiddenOutput[j][i];
        }
        output[i] = sum + biasOutput[i];
    }
    return output;
}

This forward pass computes Q-values, enabling action selection. Implications: Custom implementation fosters understanding, though less efficient than frameworks; suitable for educational prototypes.

Training Methodology and Reward Design

Training employs a DQN agent and trainer. The agent selects actions via epsilon-greedy, executes them, computes rewards, and stores experiences. Replay samples batches, updating the network; target updates occur periodically. Rewards equal score deltas minus amplified floor penalties, incentivizing efficient placements. Trainer parameters include episodes, steps per save, epsilon decay (initial 1.0 to minimum 0.05). Live demo showcases training: loading pre-trained models, executing moves in a terminal UI, observing random explorations (5% minimum) for continual learning. Methodologically, Victor iterates games, refining via replays. Contexts: Balances short-term gains (line completions) with long-term strategies (wall synergies). Implications: Reward shaping critical—penalizing floors curbs suboptimal habits; serialization enables resuming, vital for lengthy trainings.

Analytical Implications for Java-Based AI

Victor’s approach validates Java for RL sans external dependencies, leveraging enums, lists for game logic, arrays for neural computations. Innovations: Pure Java DQN demystifies black-box frameworks, aiding pedagogy. Challenges: Manual matrix operations less performant; state space demands efficient encoding. Future: Enhance UI, explore advanced variants (Double DQN). Broader ramifications: Empowers non-ML specialists in game AI; implications for scalable, interpretable agents in constrained environments.

Links:

• Lecture Video
• Victor Uria Valle on LinkedIn

Lecturer

Frank Greco is a prominent figure in the global technology landscape, serving as a Senior Consultant and Enterprise Architect with a specialized focus on Artificial Intelligence, Machine Learning, Cloud, and Edge computing. A long-standing Java Champion and educator, Frank is the founder and Chairman of the NYJavaSIG (New York Java Special Interest Group) and a co-author of JSR 381, the Java API for Visual Recognition. His extensive career includes developing innovations for financial systems and building developer communities worldwide.

Abstract

This article explores the fundamental shift from deterministic to probabilistic computing necessitated by the integration of Large Language Models (LLMs) into modern software engineering. By analyzing the internal mechanics of LLMs—specifically their nature as text pattern recognizers—this study identifies the critical distinction between generative models and traditional search engines. The analysis further investigates the collaborative synergy between language models and search infrastructure, providing a roadmap for Java developers to leverage these tools while mitigating the risks associated with non-deterministic outputs in production environments.

From Determinism to Probability: A Paradigm Shift

For decades, software development has been rooted in determinism—the principle that a specific input will consistently yield an identical, predictable output. The introduction of generative AI marks the first major transition toward probabilistic systems in mainstream application development. Unlike a weather forecast, which provides probabilities about future events, LLMs operate on internal probabilities to determine the next most likely token in a sequence. This shift requires developers to move beyond the expectation of 100% accuracy and instead design systems capable of managing uncertainty and “hallucinations”.

The Taxonomy of Large Language Models

A common misconception is treating an LLM as a traditional search engine. Technically, these models are sophisticated text pattern recognizers. They do not “know” facts in the way a database does; rather, they complete text based on patterns identified during training. While an LLM can provide accurate information if the query aligns with a path through its neural network that was heavily reinforced during training, it lacks a deterministic mechanism for factual verification.

Synergy Between Search Engines and Language Models

The true power of modern AI interfaces often stems from a hybrid approach where LLMs are integrated with search infrastructure. Major technology providers utilize search engines to retrieve relevant data points, which are then processed by a language model to formulate a coherent, human-like response. This methodology allows the model to act as a sophisticated interface layer that “understands” the user’s intent while relying on the search engine’s deterministic data retrieval to ground the response in reality.

Implications for Production Environments

Integrating probabilistic tools into production Java deployments introduces unique challenges. Developers must evaluate the risk threshold of their applications; if a single incorrect answer could cause significant harm or financial loss, a purely generative approach may be inappropriate. Understanding that LLMs are distinct from the deterministic tools of the past—such as compilers or SQL databases—is the first step toward building resilient AI-integrated systems that enhance productivity without compromising software integrity.

Links:

• Frank Greco on LinkedIn
• Frank Greco on Twitter/X
• NYJavaSIG Official Website
• JSR 381: Visual Recognition API
• Lecture Video

Lecturer

Chris Hollik is the General Manager of AI and Platform at Boomi. A veteran in the integration and automation space, Chris has been a driving force behind Boomi’s expansion into the AI agent market. He is a proponent of “agentic workflows,” arguing that the true ROI of AI comes not from individual chatbots, but from integrated, governed processes that span multiple enterprise systems.

Thomas Benjamin serves as the Senior Vice President of Engineering at Boomi. Thomas leads the technical development of Boomi’s AI-first platform, including the Agentstudio and Agent Control Tower. His focus is on architecting high-availability, multi-tenant systems that integrate with leading AI services like Amazon Bedrock to provide enterprise-grade security and observability for autonomous agents.

Abstract

The rapid proliferation of AI agents within the enterprise—a phenomenon known as “agent sprawl”—presents a significant challenge for modern IT organizations. As departments independently deploy agents from various vendors (e.g., Salesforce, Microsoft, OpenAI), the need for a centralized governance and orchestration layer becomes paramount. This article explores the Boomi Agentstudio and its integration with Amazon Bedrock, demonstrating how organizations can build and manage “agentic workflows.” These workflows connect multiple business systems—such as SAP, Oracle, and Salesforce—to execute complex tasks like manual journal entry reconciliation or regulatory compliance in pharma. Key to this strategy is the Boomi Agent Control Tower, which provides a unified interface for discovering, governing, and auditing agents in a multi-vendor environment, ensuring that the promise of hyperproductivity is matched by rigorous operational control.

The Imperative for AI Agent Governance

The “agentic revolution” is no longer a future concept; it is happening today at extreme scale. Boomi already has over 12,000 agents in production, with that number growing rapidly as organizations seek to unlock “hyperproductivity.” However, this rapid adoption has led to a critical problem: agent sprawl. Organizations find themselves in a multi-agent world where different departments use disparate tools from Anthropic, OpenAI, ServiceNow, and others.

Without a platform, this fragmented approach is unsustainable. An MIT study recently suggested that the potential for “shadow AI”—where agents operate outside the view of IT—could lead to significant security, compliance, and cost issues. For high-stakes industries like finance or pharmaceuticals, where auditors require immutable audit trails, the “black box” nature of ungoverned agents is a non-starter. The core question for the enterprise is: How do we become “enterprise class” in our AI strategy while still innovating at speed?

Boomi Agentstudio: Orchestrating the Multi-Agent Ecosystem

Boomi Agentstudio addresses these challenges by providing a dedicated environment for building and orchestrating AI agents. It is not merely a tool for creating single agents, but a platform for designing complex, multi-system workflows. Integrated with Amazon Bedrock and Q Business, Agentstudio allows users to leverage the best-of-breed foundation models while grounding them in the specific business context of their organization.

One of the platform’s unique strengths is its ability to stitch together master data from multiple applications—such as Salesforce, ServiceNow, Shopify, and SAP—to provide agents with the information they need to execute. This “connectivity tissue” ensures that an agent isn’t just generating text, but is actually performing actions like updating a downstream schema or moving data in a specific serialization format to prevent integration failure.

The system is built on open standards, being Open Telemetry compliant and contributing to protocols like the Model Context Protocol (MCP) and Agent-to-Agent (A2A). This openness ensures that as the AI landscape evolves, the Boomi platform remains compatible with any OTEL-compliant observability platform.

The Technical Foundation: Integration and Automation for AI

The effectiveness of an AI agent is fundamentally limited by the systems it can access and the data it can interpret. This is why Boomi’s heritage in Integration Platform as a Service (iPaaS) is so relevant. Integration provides the “common language” for systems to talk to each other, but it is automation that makes those processes work successfully at scale.

Boomi leverages a range of AWS services to operate at “four nines” (99.99%) of availability, including Amazon ECS, EKS, Fargate, and SQS. This infrastructure allows the platform to handle the intense computational and data-movement demands of real-time AI agents. For example, when a command is issued from the Control Tower to stop an agent, it is passed through SQS to a Lambda function that interacts directly with the Amazon Bedrock agent running in the customer’s VPC. This level of deep architectural integration ensures real-time feedback and control.

Code Sample: Agent Governance Call (Pseudo-code for Control Tower API)

{
  "command": "SUSPEND_AGENT",
  "agent_id": "FINANCE_RECON_AGENT_01",
  "reason": "Regulatory Audit in Progress",
  "governance_policy": {
    "stop_method": "GRACEFUL",
    "notify_admin": true,
    "log_level": "VERBOSE",
    "audit_persistence": "S3_IMMUTABLE_LOG"
  },
  "target_environment": {
    "cloud_provider": "AWS",
    "service": "Amazon Bedrock",
    "vpc_endpoint": "vpce-0a1b2c3d4e5f6g7h8"
  }
}

Agent Control Tower: Ensuring Auditability and Compliance

The Agent Control Tower is the “brain” of the Boomi AI ecosystem. It allows IT leaders to discover every agent running in their organization, regardless of the vendor or framework (such as LangGraph). It serves as a centralized policy enforcement point where security and operational rules are applied.

Key features of the Control Tower include:

• Discovery and Inventory: Seeing the “sprawl” in real time and bringing shadow agents under management.
• Operational Control: The ability to start, stop, or pause agents from a single dashboard.
• Immutable Audit Trails: Capturing every reasoning step and action taken by an agent. This is crucial for demonstrating to auditors how a specific financial decision or pharmaceutical report was generated.
• ROI Dashboarding: Tracking the performance and value delivery of agents to justify investment and prioritize further innovation.

Real-World Applications: From Finance to Pharma

The practical application of governed agents is vast. In the pharmaceutical industry, Boomi agents are used to manage complex regulatory workflows, ensuring that every step of a drug-approval process is documented and compliant. In finance, agents automate the “order to cash” or “hire to retire” processes, examining thousands of records in seconds to find discrepancies that would take a human days to uncover.

By providing a platform that handles the “boring” but essential tasks of governance, security, and integration, Boomi enables organizations to focus on the “interesting” part: innovating at speed. This approach ensures that AI is not just a experimental toy, but a reliable, enterprise-grade component of the modern business fabric.

Conclusion

As enterprises move from AI experimentation to full-scale production, the focus must shift from the capabilities of individual models to the governance of entire agentic ecosystems. The Boomi Agentstudio, supported by the scalability of AWS and the robust control of the Agent Control Tower, provides the necessary framework for this transition. By centralizing management in a multi-vendor world, Boomi ensures that organizations can harness the power of AI while maintaining the security and auditability required by the modern regulatory landscape.

Links:

• Boomi Agentstudio Overview
• Amazon Bedrock Service Page
• Boomi on AWS Marketplace

Lecturer

Pankaj Kumar Mishra serves as Head of Sales, Go-to-Market, and Generative AI at TCS AI.Cloud Unit, specializing in life sciences and healthcare for North America. With over seventeen years in information technology, Pankaj focuses on AWS-driven business growth and strategic partnerships. Bhavik Patel leads the architecture practice for the US employer segment at Cigna Healthcare, bringing twenty-three years of experience in technology roles within health insurance. Rajesh contributes to TCS’s cloud and modernization initiatives, emphasizing collaborative efforts in enterprise transformations.

Abstract

This in-depth examination traces Cigna Healthcare’s overhaul of sales and client onboarding processes through a cloud-native platform developed in partnership with Tata Consultancy Services and Amazon Web Services. It scrutinizes the operational hurdles in legacy systems, architectural strategies for streamlining workflows, and resultant enhancements in precision, speed, and stakeholder engagement. By analyzing the interplay of migration acceleration programs, financial optimization, and domain expertise, the article reveals how such collaborations yield measurable efficiencies and position organizations for sustained innovation in regulated sectors.

Legacy System Challenges and Strategic Partnership Formation

Cigna Healthcare, a prominent player in health insurance, operates across commercial, government, and international segments, serving millions through employer-based plans. The US employer division, central to this narrative, grappled with fragmented legacy architectures that hindered sales and onboarding agility. These systems, characterized by disparate data silos and manual interventions, protracted quote generation from days to weeks, impeding competitive responsiveness in a dynamic market.

The collaboration with TCS and AWS emerged from a shared recognition of these pain points. TCS’s domain proficiency in healthcare, coupled with AWS’s robust cloud ecosystem, facilitated a phased modernization under the AWS Migration Acceleration Program (MAP). This initiative aligned financial incentives with technical roadmaps, maximizing annual recurring revenue while offsetting migration costs. Initial assessments revealed opportunities for decommissioning outdated components, reducing technical debt without disrupting core operations.

Rajesh highlighted the emphasis on FinOps from inception, ensuring architectural reviews optimized service selection to avert over-provisioning. This proactive stance mitigated common pitfalls, such as escalating cloud spend, by embedding cost governance into design principles. The partnership’s structure—leveraging TCS’s industry insights and AWS’s scalable infrastructure—fostered a lower cost of change, enabling iterative enhancements rather than wholesale replacements.

Architectural Innovations and Implementation Methodologies

The resultant platform embodies a microservices-based, event-driven architecture on AWS, integrating services like Amazon API Gateway, Lambda, and DynamoDB for seamless data orchestration. This design supplanted rigid monolithic structures with modular components, allowing real-time quote processing and automated compliance checks. For instance, intelligent routing engines now direct requests across hybrid environments, ensuring data sovereignty and regulatory adherence under HIPAA.

Implementation adhered to MAP’s assess, mobilize, and migrate phases. The assess stage inventoried applications, prioritizing those with high business impact. Mobilization involved prototyping key workflows, such as client enrollment, using AWS Step Functions for orchestration. Migration executed via automated pipelines, transitioning workloads to Amazon Elastic Kubernetes Service (EKS) for resilience.

A pivotal innovation was the unified digital facade, providing stakeholders with intuitive interfaces via Amazon AppSync. This not only standardized capabilities but also embedded analytics for predictive insights, reducing onboarding timelines by forty percent. Bhavik Patel underscored the “bridge” concept—gradual capability reduction in legacy systems to ease adoption—alleviating business apprehensions around change.

Code sample depicting a simplified event-driven workflow in AWS Lambda for quote processing:

“`
import json
import boto3

dynamodb = boto3.resource(‘dynamodb’)
table = dynamodb.Table(‘Quotes’)

def lambda_handler(event, context):