Posts Tagged ‘DevoxxGR2026’
[DevoxxGR2026] GenAI on Kubernetes: Training, Inference, and Serving in Production Environments
Lecturer
Alessandro Vozza is a seasoned cloud-native advocate and technologist with deep expertise in Kubernetes and AI/ML operations. He contributes actively to open-source communities and focuses on practical, scalable deployments of generative AI workloads. As a speaker and practitioner, Alessandro emphasizes operational excellence, resource efficiency, and the integration of modern AI tools within established cloud-native platforms.
Abstract
In this hands-on tutorial at Devoxx Greece 2026, Alessandro Vozza guides developers through the complete lifecycle of running generative AI workloads on Kubernetes. From distributed training jobs with GPU scheduling to optimized inference and scalable model serving, the session demonstrates how to leverage operators, autoscaling, vector stores, and frameworks like KServe, Ray, vLLM, and Kubeflow. Attendees gain actionable insights into designing efficient GPU clusters, fine-tuning models securely, and deploying production-grade architectures that integrate seamlessly with existing Kubernetes expertise.
The Convergence of Kubernetes and Generative AI
Kubernetes has evolved into the de facto platform for orchestrating complex, resource-intensive workloads, including those powered by generative AI. Vozza begins by contextualizing the challenges: training large models demands massive parallel computation across GPUs, inference requires low-latency serving under variable traffic, and the entire pipeline must remain observable, secure, and cost-effective. Traditional approaches struggle with these demands, but Kubernetes patterns—scheduling, autoscaling, and declarative resource management—provide a robust foundation.
The session highlights how the community has responded with specialized tools. Projects like Kubeflow address the full ML lifecycle, while KServe and vLLM focus on high-performance inference. These build upon core Kubernetes capabilities, allowing teams to treat AI workloads with the same rigor applied to microservices.
Distributed Training and GPU Orchestration
Training generative models is computationally intensive and benefits enormously from Kubernetes’ scheduling strengths. Vozza demonstrates launching distributed training jobs, emphasizing GPU-aware scheduling through device plugins and resource requests. Nodes are labeled with GPU capacity, enabling the scheduler to place pods on suitable hardware.
The tutorial covers hyperparameter tuning with tools like Katib, which automates experimentation across multiple configurations. Fine-tuning involves augmenting base models with domain-specific data, a process that Kubernetes orchestrates reliably through persistent volumes and checkpointing. Attendees learn to monitor training progress using built-in observability and handle failures gracefully with retries and job controllers.
Resource efficiency emerges as a key theme. Techniques such as multi-instance GPU (MIG) partitioning allow a single physical GPU to support multiple smaller workloads, maximizing utilization without over-provisioning expensive hardware.
Inference Serving and Model Deployment
Once trained, models must be served efficiently. Vozza walks through deploying inference endpoints with KServe, which abstracts the complexities of scaling and routing. vLLM serves as the high-throughput inference engine, leveraging continuous batching and paged attention for superior performance.
The architecture supports multi-model serving, where a single deployment handles various models based on request characteristics. Gateway API extensions make the ingress layer LLM-aware, enabling intelligent routing based on factors like key-value cache state or model specialization. This ensures optimal resource allocation and minimal latency.
Autoscaling plays a critical role. Horizontal Pod Autoscaler (HPA) combined with KEDA reacts to custom metrics such as queue depth or tokens processed per second, dynamically adjusting replicas to match demand while controlling costs.
Operational Considerations and Best Practices
Production readiness demands comprehensive observability. Vozza integrates Prometheus exporters and logging to track token throughput, latency, and GPU utilization. Security best practices include least-privilege access for model endpoints and encrypted communication.
The tutorial addresses common pitfalls: managing model registries for versioning, handling cold starts through caching, and ensuring reproducibility across environments. By treating models as first-class Kubernetes citizens, teams achieve consistent deployments from development to production.
Practical Roadmap and Future Directions
Participants receive a working reference setup they can adapt immediately. Vozza encourages starting small—perhaps with a single-model inference service—before scaling to distributed training and multi-model architectures. The session reinforces that Kubernetes knowledge directly transfers to AI operations, lowering the barrier for traditional platform teams.
Looking ahead, evolving features like dynamic resource allocation and improved GPU topology awareness will further streamline GenAI workloads. The message is clear: Kubernetes is not merely compatible with generative AI; it is becoming the preferred operational layer for the entire lifecycle.
Links:
[DevoxxGR2026] Code That Moves the World: The Rise of Physical AI
Lecturer
Will Sentance is the founder of Standard Material and Codesmith, organizations at the forefront of physical AI infrastructure and AI/software engineering education. A speaker, educator, and practitioner, Sentance bridges software engineering expertise with emerging robotics and autonomous systems. He contributes to research at Oxford and leads initiatives training talent for the next wave of intelligent physical systems.
Abstract
In this forward-looking keynote at Devoxx Greece 2026, Will Sentance explores the profound convergence of software engineering and physical intelligence. Robots and autonomous systems are transitioning from specialized, brittle demonstrations to capable, generalizable agents operating in real-world environments. Sentance details the technological breakthroughs in hardware, data, and foundation models driving this transformation and argues that traditional software engineering skills are central to building the platforms, data pipelines, and integrations required for scalable physical AI deployment.
The Remarkable Progress in Physical Intelligence
Physical AI—systems that sense, understand, and act upon the physical world—has advanced dramatically. Robots now follow natural language instructions, handle novel objects, and demonstrate emergent capabilities. Foundation models for robotics enable zero-shot generalization and long-horizon planning across diverse embodiments.
Companies like Physical Intelligence, Agility Robotics, and others are moving from laboratory experiments to industrial and domestic applications. This shift is fueled by massive investment and rapid iteration.
Core Technological Enablers
Three key areas have transformed the landscape:
Hardware Revolution: Affordable, off-the-shelf components—from full humanoids to grippers and sensors—dramatically lower barriers. Edge computing platforms provide sufficient power for onboard inference.
Data Explosion: Teleoperation, simulation (including sophisticated world models), and real-world deployment generate multimodal datasets at unprecedented scale. Techniques like action chunking address real-time requirements.
AI Models: End-to-end learning replaces traditional control theory. Vision-language-action models predict continuous action trajectories, enabling flexible behavior without exhaustive manual programming.
The Physical AI Technology Stack
Sentance outlines a layered architecture:
- Real-time Control: Low-level, deterministic operations managing actuators and safety at high frequency.
- Platform and Middleware: Abstractions like ROS providing integration, simulation interfaces, and developer tools.
- Intelligence Layer: Foundation models processing vision, language, and proprioception to generate actions.
- Data and Learning Loop: Continuous collection, training, evaluation, and deployment cycle.
Opportunities for Software Engineers
Contrary to initial impressions, software engineers are perfectly positioned to lead this revolution. Approximately 80% of the required work involves familiar disciplines: systems architecture, platform engineering, data pipelines, low-level optimization, and agentic integration.
Roles at leading organizations emphasize scalable frameworks, reliable deployment, observability, and integration of AI models into production—skills honed in cloud-native and distributed systems development.
New challenges center on real-time constraints, physical dynamics, and managing massive multimodal datasets, but these build directly upon existing expertise.
Getting Started with Physical AI
Sentance encourages practical experimentation using affordable hardware like the SO-101 and open tools. Developers can quickly train policies for simple tasks such as closing a laptop lid, experiencing the full cycle from data collection to deployment.
The physical world represents the next major platform for code. Software engineers who embrace this frontier will shape the coming industrial transformation.
Links:
[DevoxxGR2026] Bootiful Spring Boot 4: Exploring the Latest Advancements with Java 25
Lecturer
Josh Long is a Spring Developer Advocate at VMware, widely recognized as one of the most prominent voices in the Spring ecosystem. Known affectionately as “Mr. Spring,” he is the author of numerous books and a prolific speaker who travels the globe sharing insights on modern Java development. Long co-hosts the “Coffee with a Java Champion” YouTube channel and continues to champion practical, production-ready Spring applications.
Abstract
In this engaging session at Devoxx Greece 2026, Josh Long showcases the transformative capabilities of Spring Boot 4 alongside Java 25. Through live coding of a dog adoption service, he demonstrates powerful new features including virtual threads, API versioning, modular architecture with Spring Modulith, resilient patterns, and seamless integration with AI capabilities. The presentation highlights how the Spring ecosystem empowers developers to build scalable, observable, secure, and intelligent applications with remarkable efficiency.
Java 25 and the Evolution of Spring Boot 4
Spring Boot 4 represents a significant generational leap, aligned with Spring Framework 7. Long emphasizes the decomposition of auto-configuration, resulting in leaner classpaths and faster startup times. Java 25 introduces compelling enhancements, most notably the ability to run simple applications with a single void main() method, effectively delivering the first truly elegant Java scripting experience.
These advancements set the stage for building modern, efficient services that leverage the full power of the JVM while maintaining developer productivity.
Building a Modular Dog Adoption Service
Long begins with a practical example: a service to help adopt dogs. Using Spring Initializr, he configures a project with PostgreSQL, Spring Data JDBC, web support, security, observability through OpenTelemetry and Actuator, and development tools.
The application employs a clean, feature-oriented package structure rather than traditional layered architecture. Records simplify domain modeling, while Spring Data repositories provide type-safe data access with compile-time query generation via AOT processing—beneficial for both JVM and native image deployments.
API Versioning and Resilience Features
To handle evolving requirements, Long demonstrates Spring’s new API versioning capabilities. Multiple endpoint versions coexist, with sensible defaults and header-based selection, ensuring backward compatibility.
Resilience4j integration showcases retryable methods and circuit breakers. Long simulates downstream failures to illustrate automatic recovery, highlighting how declarative resilience patterns simplify robust service design.
Modular Architecture with Spring Modulith
A standout demonstration involves refactoring into feature modules—dogs, cats, and veterinary services—using Spring Modulith. This enforces architectural boundaries at compile time while supporting event-driven communication between modules through ApplicationModuleListener and the outbox pattern for reliable, eventually consistent inter-module interactions.
The framework automatically generates documentation, C4 architecture diagrams, and verifies module dependencies, bridging the gap between intended design and runtime reality.
Security and Production Readiness
Security configuration leverages Spring Security 7’s additive customizers, preserving sensible defaults while enabling features like one-time token login and password migration. Passkeys (WebAuthn) integration provides passwordless authentication using biometrics, representing a significant usability and security improvement.
Observability is built-in through Actuator and OpenTelemetry, with production considerations like resource limits addressed from the start.
Integrating AI Capabilities
Long concludes by incorporating Spring AI 2.0, demonstrating how to augment the application with intelligent assistants. Using skills and tool calling, the service can answer domain-specific questions about dogs and cats, showcasing the natural convergence of Spring Boot with modern AI workflows.
The Bright Future of Java and Spring Development
Throughout the session, Long reinforces that the combination of Java’s efficiency, Spring’s comprehensive ecosystem, and new generative AI tools positions developers exceptionally well. Despite industry hype cycles, the fundamentals of solid engineering—modularity, resilience, observability, and security—remain paramount.
Spring Boot 4 and Java 25 deliver the tools necessary to build systems that are faster, more scalable, more maintainable, and more intelligent than ever before.