Confidential architecture

note

Not every Codesphere instance offers confidential compute. Ask your Codesphere administrator whether confidential compute is available on your instance. See Prerequisites.

With confidential compute, the whole platform infrastructure counts as untrusted. That includes the hosts, the hypervisor, the orchestration layer, storage, network and the people who operate them. Codesphere protects your data from this infrastructure in every state:

• In use. Each service runs in its own confidential micro-VM with encrypted memory.
• In transit. Traffic between confidential services is encrypted by a service mesh.
• At rest. Persistent data is encrypted inside the micro-VM before it reaches storage.

Overview

Corresponding accessible text description

The components play the following roles:

• Untrusted infrastructure. Hardware, hypervisor, the platform, storage and network. It schedules and runs services, but never sees data in plain text.
• Confidential micro-VM. The trusted wrapper around single confidential service.
• Service mesh. Encrypts and authenticates traffic between confidential services.
• Encrypted storage. Persistent data is encrypted inside the VM before it reaches the storage backend.

Confidential micro-VMs

In Codesphere, every confidential service runs inside its own lightweight virtual machine, called a micro-VM. The micro-VM runs in the host CPU's trusted execution environment and forms a trusted wrapper around the confidential service. Everything inside it, from the guest kernel to your application, is protected by hardware memory encryption.

Why a virtual machine

Regular services run as containers, which are processes on a shared host kernel. Kernel features such as namespaces and cgroups keep them apart. The kernel itself belongs to the operator, and whoever controls the kernel can inspect the memory of every service on the host.

One service consists of multiple processes, which are wrapped together in one confidential micro-VM with:

• Its own minimal guest kernel and user space
• Memory that is encrypted by the CPU
• A fixed, measurable boot image

The platform still schedules and manages the service like any other. Features such as networking and resource limits keep working.

What the host can and cannot see

The host still runs the micro-VM, so it keeps some visibility.

The host can:

• See that the VM exists and how much CPU, memory and disk it uses
• See encrypted network traffic and its endpoints
• See encrypted disk blocks
• Stop, pause or delete the VM

The host cannot:

• Read the memory of the VM
• Read decrypted disk contents or network traffic
• Change the guest image or service without failing attestation
• Run commands inside the VM through the container runtime

Service mesh

Encrypted memory protects data inside a service. As soon as data leaves the service, it crosses a network that belongs to the untrusted infrastructure. Confidential services therefore join a service mesh that encrypts and authenticates all traffic between them.

How services join the mesh

Every confidential VM includes a proxy sidecar that handles all incoming and outgoing traffic of the service. The service only talks to its local proxy. The proxy encrypts the traffic and sends it to the proxy of the receiving service, which decrypts it and passes it on.

Corresponding accessible text description

Mutual TLS between services

All traffic between confidential services uses mutual TLS (mTLS). Both sides present their certificate and verify the other one before any data flows.

• Traffic is encrypted from one micro-VM to the other. The operator sees only encrypted packets and their endpoints.
• The host or a non-attested service cannot join the mesh, impersonate another service or read traffic.
• Your application does not need to change. It talks to other services as usual, and the mesh encrypts the connection on the wire.

Traffic leaving the mesh

The mesh only covers traffic between confidential services. Plan for every other path explicitly.

• Ingress from the internet. Public traffic must be decrypted inside a micro-VM. If TLS terminates at a router outside the TEE, the operator can read requests in plain text.
• Egress to external services. The mesh does not cover calls to external APIs or databases. Use TLS and verify the server certificate.
• Non-confidential services. Traffic to regular services leaves the trust boundary. The receiving side is not protected, so treat it like an external service.

Encryption at rest

Data written to disk ends up on storage that the operator controls. Confidential services encrypt persistent data with LUKS (Linux Unified Key Setup) inside the micro-VM, before it reaches the storage backend.

1. The storage backend attaches a volume to the micro-VM as a block device.
2. The guest unlocks the volume with its disk encryption key.
3. The guest mounts the decrypted device and the service reads and writes files as usual.

Corresponding accessible text description

All encryption and decryption happens inside the TEE. The storage backend, the host and any snapshot of the raw volume only contain ciphertext.

Comparison with regular services

	Regular service	Confidential service
Isolation boundary	Shared host kernel	Own kernel in an encrypted VM
Memory	Readable by the host	Encrypted, unreadable by the host
Network traffic between services	Readable on the wire	Encrypted with mTLS through the mesh
Data on disk	Readable by the storage operator	Encrypted with LUKS inside the VM
Startup	Fast	Slower, because of VM boot and attestation
Resource overhead	Minimal	Extra memory per service for the guest kernel
Debugging access	Full	Restricted
Hardware	Any host	Hosts with TEE support

For the practical impact, see Limitations and gotchas.