Modern cloud virtualization

When we think about compute resources (AKA virtual machines) in the public cloud, most of us have the same picture in our head – operating system, above hypervisor, deployed above physical hardware.

Most public cloud providers build their infrastructure based on the same architecture.

In this post we will review traditional virtualization, and then explain the benefits of modern cloud virtualization.

‍

Introduction to hypervisors and virtualization technology

The idea behind virtualization is the ability to deploy multiple operating systems, on the same physical hardware, and still allow each operating system access to the CPU, memory, storage, and network resources.

To allow the virtual operating systems (AKA “Guest machines”) access to the physical resources, we use a component called a “hypervisor”.

There are two types of hypervisors:

Type 1 hypervisor – an operating system deployed on physical hardware (“bare metal” machine) and allows guest machines access to the hardware resources.
Type 2 hypervisor – software within an operating system (AKA “Host operating system”) deployed on physical hardware. The guest machines are installed above the host operating system. The host operating system hypervisor allows guest machines access to the underlying physical resources.

‍

The main drawbacks of current hypervisors:

There is no full isolation between multiple guest VMs deployed on the same hypervisor and the same host machine. All the network passes through the same physical NIC and same hypervisor network virtualization.
The more layers we add (either type 1 or type 2 hypervisors), we increase overhead on the host operating system and host hypervisor. This means the guest VMs will not be able to take full advantage of the underlying hardware.

‍

AWS Nitro System

In 2017 AWS introduced their latest generation of hypervisors.

The Nitro architecture, underneath the EC2 instances, made a dramatic change to the way we use hypervisors by offloading virtualization functions (such as network, storage, security, etc.) to dedicated software and hardware chips. This allows the customer to get much better performance, with much better security and isolation of customers’ data.

‍

Hypervisor prior to AWS Nitro:

Hypervisor based on AWS Nitro:

‍

The Nitro architecture is based on Nitro cards:

Nitro card for VPC – handles network connectivity to the customer’s VPC, and fast network connectivity using ENA (Elastic Network Adapter) controller
Nitro card for EBS – allows access to the Elastic Block Storage service
Nitro card for instance storage – allows access to the local disk storage
Nitro security chip – provides hardware-based root of trust

‍

In 2020, AWS introduced AWS Nitro Enclaves that allow customers to create isolated environments to protect customers’ sensitive data and reduce the attack surface.

‍

EC2 instance prior to AWS Nitro Enclaves:

EC2 instance with AWS Nitro Enclaves enabled:

The diagram below shows two EC2 instances on the same EC2 host. One of the EC2 instances has Nitro Enclaves enabled:

Additional references:

AWS Nitro System
https://aws.amazon.com/ec2/nitro/

Powering next-gen Amazon EC2: Deep dive into the Nitro system
https://www.youtube.com/watch?v=rUY-00yFlE4

Deep Dive Into AWS Nitro Enclaves
https://www.youtube.com/watch?v=K5PRNHaEdOw

Reinventing virtualization with the AWS Nitro System
https://www.allthingsdistributed.com/2020/09/reinventing-virtualization-with-aws-nitro.html

AWS Nitro System
https://perspectives.mvdirona.com/2019/02/aws-nitro-system/

AWS Nitro - What Are AWS Nitro Instances, and Why Use Them?
https://www.metricly.com/aws-nitro/

AWS Nitro Enclaves
https://aws.amazon.com/ec2/nitro/nitro-enclaves

AWS Nitro Enclaves – Isolated EC2 Environments to Process Confidential Data
https://aws.amazon.com/blogs/aws/aws-nitro-enclaves-isolated-ec2-environments-to-process-confidential-data

AWS Nitro Enclaves - Getting Started Video
https://www.youtube.com/watch?v=t-XmYt2z5S8

‍

Oracle’s Generation 2 (GEN2) Cloud Infrastructure

In 2018 Oracle introduced their second generation of cloud infrastructures.

Oracle’sGen2 cloud offers isolated network virtualization, using custom-designed SmartNIC (a special software and hardware card) which offers customers the following advantages:

Reduced attack surface.
Prevent lateral traversal between bare-metal, container or VM hosts.
Protection against Man-in-the-Middle attacks between hosts and guest VMs.
Protection against denial-of-service attacks against VM instances.

‍

First generation cloud hypervisors:

Oracle Second generation cloud hypervisor:

Oracle Cloud architecture differs from the rest of the public cloud providers in terms of CPU power.

In OCI, 1OCPU (Oracle Compute Unit) = 1 physical core, while other cloud providers use Intel hyperthreading technology, which calculates 2 vCPU = 1 physical core.
As a result, customers get better performance per each OCPU it consumes.

‍

Another characteristic that differentiates OCI architecture is no resource oversubscription, which means a customer will never share the same resource (CPU, memory, network) with another customer. This avoids a “noisy neighbor” scenario and allows the customer better and guaranteed performance.

‍