AMD ESMTP Blocks SMT Threats on EPYC, Linux Support Begins

AMD has begun enabling a new mechanism that strengthens isolation for server virtualization. The target is virtual machines using SEV-SNP. Its name is Enhanced SMT Protection, abbreviated as ESMTP. Reporting by Phoronix’s Michael Larabel said a patch series for the Linux kernel was published on September 14, 2026. The feature aims to curb the risks associated with simultaneous multithreading co-residency on AMD EPYC virtual machines.

This publication is the first appearance of ESMTP on the kernel mailing list. AMD had published an ESMTP white paper on its own site earlier this year. At the time, it did not attract much attention. Based on timing and content, it has been suggested to be a feature of the EPYC 9006 “Venice” generation. At this point it remains speculation, but a link to next-generation products is in view.

SMT Threats and Countermeasures

Simultaneous multithreading is a technology that shares one physical core between two logical entities. Sharing resources improves processing performance. On the other hand, it is known that it can become a path for side-channel attacks. Past cases, including Intel Hyper-Threading, form the background to this concern.

Linux has an operational mechanism called core scheduling. It is a method of assigning one core and its sibling threads to the same virtual machine. It expresses co-residency relationships using identifiers. This is implemented as a host kernel policy. ESMTP differs in that it enforces this policy in hardware. It is designed to prevent sharing of resources with untrusted host-side processes.

Hardware-Enforced Isolation Mechanism

ESMTP provides guarantees for SEV-SNP virtual machines. While one vCPU is running in guest mode, it restricts the state of sibling threads. Only host idle state or execution of a sibling vCPU declared as legitimate is permitted. Hardware refuses co-residency with other guests or arbitrary host-side processes. The patch series description stating this is as follows.

Enhanced SMT Protection (ESMTP) allows an SEV-SNP VM to require that, while one of its vCPUs is in guest mode, every SMT sibling thread on that physical core is either idle in host mode or running a vCPU the guest itself has declared a legal sibling. This mitigates the side-channel risk of sharing core resources with untrusted host threads or with another guest. Unlike core scheduling, where co-residency is a host kernel policy expressed with cookies, ESMTP is enforced by hardware.

Sibling designations are stored in a sibling mask inside the VMSA. The host cannot run arbitrary kernel, user, or interrupt handling on the sibling side. KVM and the guest fully set VCPU_SIBLING_MASK. This places all of the guest’s vCPUs in a single group. Any two can be co-located. Combined with ASID checks, siblings during guest execution are limited to vCPUs of the same guest or host idle. Because enforcement is by hardware, host policy violations cannot occur by design.

Kernel Support Status for Enablement

The current kernel patch series consists of two patches. They cover acceptance on the KVM side and setup on the guest side. The kernel alone is not sufficient. Changes to upstream QEMU and OVMF are also required. Support across the entire virtualization stack is a prerequisite.

The requirement is use of SEV-SNP. It cannot be used in configurations that do not use SEV-SNP. The white paper also positions it as an extension for confidential virtual machines. As shown by Rippling Announces AI Spend Tracking Tool ‘AI Spend Console’, operational management of cloud infrastructure is becoming more fragmented. ESMTP is also being prepared as an option for cloud computing. As seen in Asahi Linux Adds M3 Support, New Acer and Lenovo Models, coordination between kernel and hardware is central to recent infrastructure development. This time as well, alignment between hardware specifications and kernel implementation will be key.

Performance Cost and Operational Options

ESMTP is not enabled by default. It is provided as an opt-in feature. The reason is performance cost. This is because VMRUN stalls until the sibling side is ready. The patch series description is as follows.

ESMTP is opt-in because it carries a performance cost as VMRUN stalls until the sibling runs work from a trusted vCPU or is in host idle.

Expected use cases are public clouds and running untrusted workloads. In multi-tenant environments, ownership of sibling threads becomes important. Stronger isolation comes at the cost of increased latency. Operators need to weigh safety against processing performance. This aligns with the growing interest in confidential computing shown by OpenAI Announces Private Safety Processing with Zero Data Retention. Hardware-enforced isolation will serve as material to meet audit and contractual requirements.

Editorial Opinion

On short-term impact. In the next 3 to 6 months, work to merge support into the kernel, QEMU, and OVMF will be the focus. Cloud providers will measure the impact of VMRUN stalls in test environments. Configuration guidelines and documentation for SEV-SNP users will proceed in parallel.

On the long-term view. In 1 to 3 years, hardware-enforced co-residency restrictions will be built into confidential computing designs. As the Venice generation spreads, whether to keep the default or enable it automatically will be debated. Operators will be required to design configurations that assume performance overhead.

Questions from the editors. The fact that measured performance degradation figures have not yet been shown is a validation issue. Whether maintaining opt-in is appropriate, or whether changes are needed in response to growing threats, will divide opinion. How far untrusted workloads should be isolated, and how to align cost bearers and beneficiaries, is the question at hand.

References

• “AMD Preparing Linux For Enhanced SMT Protection “ESMTP” For EPYC VMs”, by Michael Larabel — Phoronix, 2026-09-14T18:03:38.000Z (ARR)
• Source URL: https://www.phoronix.com/news/AMD-Enhanced-SMT-Protection