Jonathan Lalou's Blog

(long answer to this comment on LinkedIn)

As GraalVM continues to mature, a recurring question surfaces among architects and performance engineers: can it realistically outperform traditional C++ in real-time systems?

The answer is nuanced. While GraalVM represents a major leap forward for managed runtimes, it does not fundamentally overturn the performance model that gives C++ its edge in deterministic environments. However, the gap is narrowing in ways that materially change architectural decisions.

Reframing the Question: What Do We Mean by “Real-Time”?

Before comparing technologies, it is critical to define “real-time.” In engineering practice, this term is frequently overloaded.

There are two distinct categories:

• Hard real-time: strict guarantees on worst-case latency (e.g., missing a deadline is a system failure)
• Soft real-time: latency matters, but occasional deviations are acceptable

Most backend systems fall into the second category, even when they are described as “low-latency.” This distinction is essential because it directly determines whether GraalVM is even a viable candidate.

Execution Models: Native vs Managed

C++: Deterministic by Design

C++ provides a minimal abstraction over hardware:

• Ahead-of-time (AOT) compilation to native code
• No implicit garbage collection
• Full control over memory layout and allocation strategies
• Predictable interaction with CPU caches and NUMA characteristics

This enables precise control over latency, which is why C++ dominates in domains such as embedded systems, game engines, and high-frequency trading infrastructure.

GraalVM: A Spectrum of Execution Modes

GraalVM is not a single execution model but a platform offering multiple strategies:

• JIT mode (JVM-based): dynamic compilation with runtime profiling
• Native Image (AOT): static compilation into a standalone binary
• Polyglot execution: interoperability across languages

Each mode introduces different trade-offs in terms of startup time, peak performance, and latency stability.

JIT Compilation: Peak Performance vs Predictability

GraalVM’s JIT compiler is one of its strongest assets. It performs deep optimizations based on runtime profiling, including:

• Inlining across abstraction boundaries
• Escape analysis and allocation elimination
• Speculative optimizations with deoptimization fallback

In long-running services, this can produce highly optimized machine code that rivals native implementations.

However, this optimization model introduces variability:

• Warmup phase: performance improves over time
• Deoptimization events: speculative assumptions can be invalidated
• Compilation overhead: CPU cycles are consumed by the compiler itself

For systems requiring stable latency from the first request, this behavior is inherently problematic.

Native Image: Reducing the Gap

GraalVM Native Image shifts compilation to build time, eliminating JIT behavior at runtime. This results in:

• Fast startup times
• Lower memory footprint
• Reduced latency variance

However, these benefits come with trade-offs:

• Loss of dynamic optimizations available in JIT mode
• Restrictions on reflection and dynamic class loading
• Generally lower peak performance compared to JIT-optimized code

Even in this mode, C++ retains advantages in fine-grained memory control and instruction-level optimization.

Garbage Collection and Latency

Garbage collection is one of the most significant differentiators between GraalVM and C++.

Modern collectors (e.g., G1, ZGC, Shenandoah) have dramatically reduced pause times, but they do not eliminate them entirely. More importantly, they introduce uncertainty:

• Pause times may vary depending on allocation patterns
• Concurrent phases still compete for CPU resources
• Memory pressure can trigger unexpected behavior

In contrast, C++ allows engineers to:

• Use stack allocation or object pools
• Avoid heap allocation in critical paths
• Guarantee upper bounds on allocation latency

This difference is decisive in hard real-time systems.

Microarchitectural Considerations

At the highest level of performance engineering, factors such as cache locality, branch prediction, and instruction pipelines dominate.

C++ offers direct control over:

• Data layout (AoS vs SoA)
• Alignment and padding
• SIMD/vectorization strategies

While GraalVM’s JIT can optimize some of these aspects automatically, it operates under constraints imposed by the language and runtime. As a result, it cannot consistently match the level of control available in C++.

Latency Profiles: A Practical Comparison

From a systems perspective, the difference can be summarized as follows:

Characteristic	C++	GraalVM (JIT)	GraalVM (Native Image)
Startup Time	Fast	Slow	Very fast
Peak Throughput	Excellent	Excellent (after warmup)	Good
Latency Predictability	Excellent	Moderate	Good
Memory Control	Full	Limited	Limited

Where GraalVM Is a Strong Choice

Despite its limitations in strict real-time environments, GraalVM excels in several domains:

Low-Latency Microservices

Native Image significantly reduces cold start times and memory usage, making it ideal for containerized workloads and serverless environments.

High-Throughput Systems

In long-running services, JIT optimizations can deliver excellent throughput with acceptable latency characteristics.

Polyglot Architectures

GraalVM enables seamless interoperability across multiple languages, simplifying system design in heterogeneous environments.

Developer Productivity

Compared to C++, the Java ecosystem offers faster iteration, richer tooling, and lower cognitive overhead for most teams.

Where C++ Remains Unmatched

C++ continues to dominate in scenarios where performance constraints are absolute:

• Hard real-time systems (avionics, medical devices, robotics)
• High-frequency trading engines with microsecond budgets
• Game engines and real-time rendering pipelines
• High-performance computing (HPC)

In these domains, even minor unpredictability is unacceptable, and the control offered by C++ is indispensable.

Strategic Takeaway

The most important shift is not that GraalVM surpasses C++, but that it redefines the boundary where managed runtimes are viable.

Historically, many systems defaulted to C++ purely for performance reasons. Today, GraalVM enables teams to achieve sufficiently high performance with significantly better developer productivity and ecosystem support.

This changes the optimization calculus:

• Use C++ when you need guarantees
• Use GraalVM when you need performance and agility

Conclusion

GraalVM does not replace C++ in real-time systems—but it does erode its dominance in adjacent domains.

For hard real-time applications, C++ remains the gold standard due to its deterministic execution model and fine-grained control over system resources.

For everything else, the decision is no longer obvious. GraalVM offers a compelling middle ground, delivering strong performance while dramatically improving developer velocity.

In modern system design, that trade-off is often more valuable than raw speed alone.