Microsoft has announced the official launch of Azure confidential virtual machines (VMs) featuring the NCC H100 v5 SKU equipped with NVIDIA Tensor Core GPUs. These VMs leverage 4th-generation AMD EPYC processors to offer hardware-based data protection alongside enhanced performance.
This general availability follows the VMs’ earlier preview last year. By integrating confidential computing capabilities with GPU technology, Azure provides customers expanded options for securely and efficiently executing workloads in the cloud. These virtual machines are particularly suitable for inferencing, training, and fine-tuning small to medium-sized models, including popular frameworks like Whisper, Stable Diffusion and its variants (SDXL, SSD), and various language models such as Zephyr, Falcon, GPT-2, MPT, Llama2, Wizard, and Xwin.
The NCC H100 v5 VM SKUs feature a hardware-based Trusted Execution Environment (TEE), enhancing security for guest VMs by protecting against unauthorized access to VM memory and state by the hypervisor and other host management systems. This safeguards against unauthorized operator access. Customers can perform attestation requests to confirm that their VMs are operating within a correctly configured TEE, which is crucial for managing keys and running sensitive applications.
In a LinkedIn post, Vikas Bhatia, head of product for Azure confidential computing, and Drasko Draskovic, founder and CEO of Abstract Machines, acknowledged the advancements while also highlighting that attestation remains a concern. They pointed out that existing attestation mechanisms from cloud service providers like Azure and GCP may rely on trust in the provider, which undermines the essence of Confidential Computing. They suggested that a bare metal approach appears to be the only reliable option at present, although this reduces the necessity of TEEs except in multi-party computation services.
Several businesses have started utilizing the Azure NCC H100 v5 GPU VMs for confidential applications such as audio-to-text inferencing with Whisper models, incident prevention through video analysis, ensuring data privacy with confidential computing, and managing sensitive design data for stable diffusion projects in the automotive industry.
In addition to Microsoft, other major cloud providers like AWS and Google also provide NVIDIA H100 Tensor Core GPUs. For example, AWS includes H100 GPUs in its EC2 P5 instances, which are optimized for high-performance computing and AI tasks.
According to a recent whitepaper by NVIDIA regarding the H100 Tensor Core GPU, this next-generation data center GPU aims to significantly outperform the prior generation A100 Tensor Core GPU for large-scale AI and high-performance computing (HPC) tasks. The design improvements focus on enhancing scaling efficiency for these workloads.
Currently, Azure NCC H100 v5 virtual machines are available exclusively in the East US2 and West Europe regions.