Posts Tagged ‘NitroSystem’
[AWSReInvent2025] Maximizing Block Storage Performance for High-Intensity Workloads: A Technical Analysis of io2 Block Express and the Nitro System
Lecturer
Mark Olsen and Jody Berenblatt are distinguished engineering and product leaders at Amazon Web Services, specializing in high-performance block storage. Mark Olsen serves as a Principal Product Manager for Amazon EBS, where he focuses on the architectural evolution of Provisioned IOPS volumes to meet the demands of mission-critical enterprise applications. Jody Berenblatt, a Senior Technical Product Manager, brings extensive expertise in the integration of storage subsystems with the AWS Nitro System and the optimization of storage networking protocols. Their work has been pivotal in the development of io2 Block Express, a storage tier designed to provide SAN-like performance in the cloud.
Abstract
This article provides a comprehensive examination of the technical foundations and performance characteristics of high-intensity block storage within the Amazon Elastic Block Store (EBS) ecosystem. Centered on the io2 Block Express architecture, the analysis explores how the integration of the AWS Nitro System, the Scalable Reliable Datagram (SRD) protocol, and Multi-Attach NVMe reservations enables ultra-low latency and high-throughput capabilities for data-intensive workloads such as SAP HANA, Oracle, and Microsoft SQL Server. The discussion details the methodology for managing tail latency, the benefits of decoupled storage architectures, and the operational strategies required to maximize I/O performance in a distributed cloud environment.
Infrastructure Foundations: The Evolution of Provisioned IOPS
The landscape of enterprise computing has shifted toward workloads that demand not only high throughput but also extreme consistency in I/O operations per second (IOPS). For decades, on-premises Storage Area Networks (SANs) were the only viable option for these applications. However, the maturation of Amazon EBS, particularly the transition from io1 to the io2 Block Express architecture, has redefined the capabilities of cloud-native block storage. The fundamental challenge in high-intensity storage is the management of latency, which is often the primary bottleneck for database performance.
In traditional storage models, performance was often tethered to the physical limitations of the disk or the controller. In the modern AWS architecture, the storage is decoupled from the compute instance, connected via a dedicated high-speed network. This separation allows for independent scaling of compute and storage resources but introduces the necessity for highly optimized networking to maintain sub-millisecond latency. The io2 Block Express volumes are engineered to provide up to 256,000 IOPS and 4,000 MB/s of throughput per volume, offering a level of performance that satisfies even the most demanding transactional databases.
Architecture of io2 Block Express: Performance and Durability
The architecture of io2 Block Express represents a paradigm shift in how block storage is provisioned and managed. Unlike standard volumes, io2 Block Express is designed to handle “high-intensity” workloads, defined by their sensitivity to latency and their requirement for high durability. These volumes provide a durability rating of 99.999%, which is a ten-fold improvement over standard io1 volumes. This reliability is achieved through sophisticated replication techniques across multiple physical hardwares within an Availability Zone.
A critical innovation in this architecture is the way it handles I/O operations. By utilizing the Nitro System, the overhead of the hypervisor is removed, allowing the EBS service to communicate directly with the instance’s memory. This “Block Express” layer acts as a high-performance interface that minimizes the processing time required for each I/O request. For applications like SAP HANA, where the speed of logging and data loading is critical, the reduced overhead translates directly into faster business processing cycles.
Networking Innovations: Scalable Reliable Datagram (SRD)
Perhaps the most significant technical advancement in maximizing block storage performance is the implementation of the Scalable Reliable Datagram (SRD) protocol. Traditional TCP protocols, while reliable, are prone to “head-of-line blocking,” where a single lost packet can delay the entire stream of data. In a high-performance storage environment, this creates “tail latency”—spikes in response time that can disrupt database synchronization and performance.
SRD solves this by utilizing multipath routing. Instead of sending data down a single network path, SRD spreads the traffic across as many as 64 different paths simultaneously. If a specific network switch becomes congested or a link fails, the protocol automatically reroutes the data without the latency spikes associated with TCP retransmissions. This protocol is implemented directly in the Nitro Cards, ensuring that the heavy lifting of network management does not consume CPU cycles on the user’s EC2 instance. The result is a more consistent “p99” latency profile, which is essential for maintaining stable performance in clustered environments.
Multi-Attach NVMe Reservations and High Availability
For enterprise applications requiring high availability, the ability for multiple EC2 instances to attach to a single EBS volume is a critical requirement. io2 Block Express supports Multi-Attach, allowing up to 16 Nitro-based instances to access the same volume simultaneously. This feature is particularly valuable for clustered file systems and applications that require shared storage for failover or parallel processing.
To manage concurrent access without data corruption, AWS implemented Multi-Attach NVMe Reservations. Based on the NVMe standard for persistent reservations (similar to SCSI-3 PR), this technology allows one instance to “reserve” the volume, ensuring that only authorized nodes can perform write operations. In the event of an instance failure, the reservation can be quickly cleared and reassigned to a healthy node, minimizing downtime. This mechanism provides the coordination layer necessary for complex deployments like Oracle RAC or SAP environments, where data integrity across multiple nodes is non-negotiable.
Observability and Performance Tuning for Enterprise Workloads
Achieving maximum performance requires a sophisticated approach to observability. Many administrators focus on average latency, but in high-intensity workloads, the “outliers” or tail latency are what truly matter. AWS provides tools such as Amazon CloudWatch and EBS Volume Insights to monitor these metrics in real-time. A key metric is the “Queue Depth,” which represents the number of pending I/O requests for a volume. To reach the full potential of an io2 Block Express volume (e.g., 256,000 IOPS), the application must maintain a sufficient queue depth—often 128 or higher—to keep the storage pipeline full.
// Example AWS CLI command to modify an EBS volume to io2 with high provisioned IOPS
aws ebs modify-volume \
--volume-id vol-0123456789abcdef \
--volume-type io2 \
--iops 100000
Furthermore, the choice of the EC2 instance type is paramount. Performance is not solely a function of the storage volume; the instance must be “EBS-optimized” with sufficient dedicated bandwidth to handle the provisioned throughput. For instance, using an R5b or X2idn instance allows the application to utilize the full 4,000 MB/s throughput offered by Block Express. Failure to match the instance capability with the volume performance will lead to throttling at the instance level, regardless of how many IOPS are provisioned.
Links:
[AWSReInvent2025] Introducing Nitro Isolation Engine: Transparency through Mathematics
Lecturer
JD Bean is a principal architect in AWS’s compute and ML services organization, specializing in virtualization and security innovations. Kareem Raslan serves as a senior principal engineer in AWS’s Nitro hypervisor team, focusing on hardware-software integration for cloud security. Nathan Chong is a principal applied scientist in AWS’s automated reasoning group, with expertise in formal verification and mathematical proofs. Relevant links include JD Bean’s LinkedIn profile (https://www.linkedin.com/in/jdbean/) and Nathan Chong’s LinkedIn profile (https://www.linkedin.com/in/nathan-chong-aws/).
Abstract
This article explores the AWS Nitro Isolation Engine, an advancement in the Nitro System that employs formal verification to ensure mathematical certainty in workload isolation. It examines the evolution of Nitro’s design, the application of automated reasoning for proofs, and the implications for cloud security, emphasizing compartmentalization and transparency.
The Evolution of the AWS Nitro System
The AWS Nitro System has fundamentally transformed the landscape of cloud virtualization by prioritizing enhanced security, superior performance, and accelerated innovation. JD Bean traces its development back to 2012, explaining how it culminated in a public launch in 2017 that marked a departure from conventional hypervisors such as Xen. At its core, the system relies on a customized version of the KVM hypervisor tailored specifically for cloud environments, complemented by the sixth generation of proprietary Nitro Silicon. This infrastructure underpins all EC2 instances introduced since 2018, demonstrating AWS’s commitment to reimagining virtualization.
In earlier iterations, systems like Xen depended on a component known as Dom0, which essentially functioned as a general-purpose operating system to handle essential tasks such as input/output operations, orchestration, and monitoring. However, as AWS expanded its services and built deeper relationships with customers, the limitations of Xen became increasingly apparent. The team recognized the need to push beyond these constraints, leading to a comprehensive reinvention that eliminated superfluous elements and relocated AWS-specific functions to dedicated hardware. Consequently, the Nitro System features a streamlined host operating system reduced to a minimal kernel, which not only minimizes potential attack surfaces but also enforces a policy of zero operator access, thereby isolating customer data from AWS personnel.
Within this broader context, the rise of cloud adoption has amplified the demand for confidential computing, where sensitive workloads require robust protections against unauthorized access. The Nitro architecture addresses these needs by compartmentalizing only the most critical isolation functions, which in turn optimizes efficiency and reduces vulnerabilities. This design philosophy ensures that customers can leverage the cloud’s scalability without compromising on security, setting the stage for subsequent advancements like the Nitro Isolation Engine.
Design and Implementation of the Nitro Isolation Engine
Building upon the foundational principles of the Nitro System, the Nitro Isolation Engine introduces a compact and formally verified module that significantly bolsters isolation assurances. Kareem Raslan elaborates on its compartmentalization strategy, noting how non-essential operations are shifted to user space, leaving behind a concise kernel comprising fewer than 100,000 lines of code dedicated solely to vital activities such as memory allocation and interrupt handling.
This engine is currently implemented on the Graviton 5 processor, available in preview mode, and utilizes specialized hardware extensions to facilitate secure transitions across compartments. The implementation methodology centers on rigorous specification, where the engine’s expected behaviors—such as maintaining strict workload separation—are articulated through precise mathematical models. Subsequently, the team employs tools like Isabelle to prove that the actual code aligns perfectly with these specifications, thereby guaranteeing that no deviations occur.
Nathan Chong further illuminates the process of automated reasoning, beginning with intuitive examples like the formula for the sum of the first n natural numbers and progressing to sophisticated machine-checked proofs. For the engine, this approach extends to verifying properties over potentially infinite states, which ensures that unauthorized access paths are entirely eliminated. The result is a system that not only performs efficiently but also withstands rigorous scrutiny, providing customers with unparalleled confidence in their data’s protection.
The implications of this design are profound, as it substantially diminishes the risk of exploitation by confining the trusted computing base to a minimal footprint. By verifying a smaller codebase through automated means, the engine mitigates issues stemming from legacy components, paving the way for a more secure cloud ecosystem.
Automated Reasoning and Mathematical Proofs
Automated reasoning stands as a cornerstone of the Nitro Isolation Engine, offering what the presenters describe as “transparency through mathematics” by delivering incontrovertible assurances of isolation. Nathan Chong contrasts informal proofs and specifications with their machine-checked counterparts in the Isabelle theorem prover, where each logical step is mechanically validated to prevent errors.
At the heart of this process lie core concepts such as specifications, which define the precise behaviors a system must exhibit, and proofs, which consist of finite chains of reasoning that irrefutably establish desired properties. For domains involving infinite possibilities, such as the natural numbers, techniques like mathematical induction are employed: a base case confirms the property for the initial value, while the inductive step demonstrates its preservation across subsequent values, much like a cascade of falling dominoes.
Scaling these methods to the complexities of the Nitro Isolation Engine requires advanced mathematical frameworks, including separation logic for managing memory resources, refinement techniques for bridging abstraction levels, and theorem provers to automate verification. Drawing on decades of research in formal methods, this approach ensures comprehensive coverage of real-world scenarios, including concurrent operations that could otherwise introduce subtle vulnerabilities.
An analysis of this methodology reveals its inherent value: unlike traditional testing, which is confined to finite scenarios, mathematical proofs provide exhaustive guarantees, fostering a level of trust that is essential for confidential computing environments. This not only elevates security standards but also enables organizations to innovate with greater assurance.
Implications for Cloud Security and Future Innovations
The introduction of the Nitro Isolation Engine heralds a new era in cloud security, where mathematical proofs become the benchmark for verifying system integrity. By emphasizing compartmentalization, the engine effectively minimizes the trusted computing base, thereby reducing the potential for exploits and enhancing overall resilience. Currently available as an always-on feature on Graviton 5 processors in preview, it invites users to request access through designated AWS channels, signaling AWS’s proactive stance in deploying cutting-edge security measures.
On a broader scale, the consequences extend to industries with stringent privacy requirements, such as finance and healthcare, where verifiable isolation can mitigate compliance risks and build customer confidence. AWS’s ongoing commitment to elevating security standards—evident throughout the Nitro System’s history—suggests that future innovations will continue to prioritize robust protections, allowing for rapid advancements without sacrificing safety.
This transparency through mathematics not only demystifies complex systems but also empowers users to make informed decisions about their cloud strategies, ultimately contributing to a more secure digital landscape.
Conclusion
The Nitro Isolation Engine exemplifies AWS’s unwavering dedication to pioneering secure and innovative cloud infrastructure. Through the rigorous application of formal verification, it achieves mathematical certainty in workload isolation, thereby redefining transparency and trust in the realm of virtualization.
Links:
- https://www.youtube.com/watch?v=hqqKi3E-oG8
- https://www.linkedin.com/in/jdbean/
- https://www.linkedin.com/in/nathan-chong-aws/
[AWSReInforce2025] How AWS designs the cloud to be the most secure for your business (SEC201)
Lecturer
The presentation is delivered by AWS security engineering leaders who architect the foundational controls that underpin the global cloud infrastructure. Their expertise encompasses hardware security modules, hypervisor isolation, formal verification, and organizational separation of duties across planetary-scale systems.
Abstract
The exposition delineates AWS’s security design philosophy, demonstrating how deliberate architectural isolation, formal verification, and cultural reinforcement create a substrate that absorbs undifferentiated security burden. Through examination of Nitro System enclaves, independent control planes, and hardware-rooted attestation, it establishes that security constitutes the primary reliability pillar, enabling customers to prioritize application innovation over infrastructure protection.
Security as Cultural Imperative and Design Principle
Security permeates AWS culture as the paramount priority, manifesting in organizational structure and technical architecture. Every engineering decision undergoes security review; features ship only when security criteria are satisfied. This cultural commitment extends to compensation—security objectives weigh equally with availability and performance in promotion criteria.
The design principle of least privilege applies ubiquitously: services operate with minimal permissions, even internally. When compromise occurs, blast radius is constrained by default. This philosophy contrasts with traditional enterprises where security is bolted on; at AWS, it is the foundation upon which all else is built.
Hardware-Enforced Isolation via Nitro System
The Nitro System exemplifies security through custom silicon. Traditional servers commingle customer workloads with management firmware; Nitro segregates these domains into dedicated cards—compute, storage, networking—each with independent firmware update channels.
Customer VM → Nitro Hypervisor → Nitro Security Module → Physical CPU
↘ Independent Control Plane → Hardware Attestation
The Nitro Security Module (NSM) maintains cryptographic attestation of the entire software stack. Before a host accepts customer instances, NSM verifies firmware integrity against immutable measurements burned into one-time-programmable fuses. Any deviation prevents boot, eliminating persistent rootkits at the hardware layer.
Independent Control and Data Plane Separation
Control plane operations—API calls, console interactions—execute in isolated cells that never touch customer data. A misconfigured S3 bucket policy might grant public access from the data plane perspective, but the control plane maintains an independent audit stream that detects the anomaly within minutes. This separation ensures configuration drift cannot evade detection.
The demonstration illustrates a public bucket created intentionally for testing. Within 180 seconds, Amazon Macie identifies the exposure, GuardDuty generates a finding, and Security Hub triggers an automated remediation workflow via Lambda. The customer perceives no interruption, yet the risk is mitigated proactively.
Formal Verification and Provable Security
AWS employs mathematical proof for critical components. The s2n TLS library undergoes formal verification using SAW (Software Analysis Workbench), proving absence of memory safety errors in encryption pathways. Similarly, the Firecracker microVM—underpinning Lambda and Fargate—uses TLA+ specifications to verify isolation properties under concurrency.
These proofs extend to hardware: the Nitro enclave attestation protocol is verified using ProVerif, ensuring man-in-the-middle attacks are impossible even if the host OS is compromised. This rigor transforms empirical testing into mathematical certainty for security invariants.
Organizational Isolation and Compensating Controls
Beyond technical boundaries, AWS enforces organizational separation. Teams that manage customer data cannot access control plane systems, and vice versa. This dual-key approach prevents insider threats: a malicious storage engineer cannot modify billing logic.
Compensating controls provide defense in depth. Even if a service principal is compromised, VPC endpoints restrict traffic to authorized networks. Immutable infrastructure—AMI baking, Infrastructure as Code—ensures configuration drift triggers automated replacement rather than manual fixes.
Customer Outcomes and Shared Fate
The infrastructure absorbs complexity so customers need not replicate it. Organizations avoid building global DDoS mitigation, hardware security module fleets, or formal verification teams. Instead, they compose higher-order security patterns: cell-based architectures, zero-trust microsegmentation, and automated compliance evidence collection.
This shared fate model extends to innovation velocity. When AWS hardens the substrate—introducing post-quantum cryptography in KMS, or confidential computing in EC2—customers inherit these capabilities instantly across all regions. Security becomes a force multiplier rather than a drag coefficient.
Conclusion: Security as Substratum for Civilization-Scale Computing
AWS designs security not as a feature but as the invariant property of the computing substrate. Through hardware isolation, formal verification, cultural reinforcement, and independent control planes, it creates a platform where compromise is detected and contained before customer impact. This foundation liberates organizations to build transformative applications—genomic sequencing at population scale, real-time fraud detection for billions of transactions—confident that the underlying security posture is mathematically sound and operationally resilient.