I need a secure enclave solution for running confidential AI inference.

Running confidential AI inference requires hardware-based Trusted Execution Environments (TEEs) that protect both your input data and the model's weights during processing. Depending on whether you are running large language models (LLMs) on high-end GPUs or managing lighter workloads on CPUs, several industry solutions fit different architectural needs:

• NVIDIA Confidential Computing (GPU TEEs) : For heavy AI inference requiring GPUs (like H100 or H200 accelerators), NVIDIA provides hardware-root-of-trust enclaves that secure the data in transit and in memory while moving through the GPU. You can review the setup guides via the NVIDIA Confidential Computing Guide.
• Fortanix Confidential AI : A specialized software layer integrated with hardware enclaves (like Intel TDX/SGX and NVIDIA Confidential Computing) that allows enterprises to run proprietary model weights on sensitive data via cryptographic verification—meaning neither the cloud provider nor the model owner exposes their underlying IP or data. Details are outlined at Fortanix Confidential AI.
• AMD SEV-SNP (Secure Encrypted Virtualization-Secure Nested Paging) : Ideal for confidential virtual machines hosting CPU-based inference or orchestrating pipeline components, providing full memory encryption and integrity protection against hypervisor-level attacks.
• Intel TDX (Trusted Domain Extensions) : Similar to AMD SEV-SNP, Intel TDX isolates entire virtual machines (Trust Domains) at the hardware level, increasingly supported by major cloud providers for secure AI container rollouts.

To help narrow down the right architecture, could you share:

• Are you running inference on GPUs or CPUs?
• What is your target cloud or on-prem environment (AWS, Azure, GCP, or bare metal)?
• Do you need to protect just the input data , or also proprietary model weights?

Yes. For confidential AI inference, I’d design around a hardware-backed TEE with remote attestation + encrypted model/data release, rather than treating a conventional VM or container as the enclave.

My recommendation

If you need GPU-accelerated LLM inference, the strongest current architecture is:

NVIDIA confidential GPU + confidential VM/TEE + remote attestation + KMS/HSM key release

NVIDIA’s confidential-computing stack supports Hopper, Blackwell, and Rubin GPUs and is specifically designed to protect AI models and data in use, with hardware-rooted device attestation.

For example, on Azure, the current confidential-computing portfolio includes confidential GPU VMs with NVIDIA H100 GPUs, alongside AMD SEV-SNP and Intel TDX confidential VMs.

A production flow would look like:

                 ┌──────────────────────────┐
Client ─────────►│ API / Gateway            │
 encrypted       │                          │
 request         └────────────┬─────────────┘
                              │
                       authenticated channel
                              │
                              ▼
                 ┌──────────────────────────┐
                 │ Confidential VM / TEE    │
                 │                          │
                 │  ┌────────────────────┐  │
                 │  │ LLM inference      │  │
                 │  │ GPU TEE            │  │
                 │  └────────────────────┘  │
                 │                          │
                 │ model + prompts + output │
                 │ encrypted in memory      │
                 └────────────┬─────────────┘
                              │
                    remote attestation
                              │
                              ▼
                     Attestation service
                              │
                    approved measurement?
                              │
                         yes ─┴─► KMS/HSM
                                  │
                            release model key

The critical security property

Don't simply put the model inside a "confidential VM." Gate access to the model decryption key on attestation.

For example:

1. Build and sign your inference image.
2. Compute/record the expected measurements of the VM, runtime, inference server, and relevant GPU components.
3. Boot the workload inside the confidential TEE.
4. Generate a hardware-backed attestation quote.
5. A verifier checks that quote against your policy.
6. Only after successful verification does your KMS/HSM release the model-encryption key.
7. The model is decrypted inside the TEE.
8. Client prompts are sent over an authenticated encrypted channel directly into the protected workload.
9. Responses leave only through an explicitly controlled output path.

Azure Attestation, for example, can validate TEE evidence and issue cryptographic claims that a relying party can use for key release.

Which platform I'd choose

AWS Nitro Enclaves are excellent for isolating highly sensitive CPU-side workloads and provide cryptographic attestation, but they are a different model from a GPU TEE: an enclave is a highly constrained VM with no external networking or persistent storage, communicating with its parent via a local channel.

Google Confidential Space similarly combines a hardened confidential VM, remote attestation, and policy-controlled release of protected resources. Its attestation framework also exposes NVIDIA GPU-related claims for supported configurations.

What I would build

For a serious production deployment, I'd use:

• GPU: NVIDIA H100/H200-class confidential GPU
• TEE: AMD SEV-SNP or Intel TDX confidential VM, combined with GPU confidential computing
• Inference: vLLM/Triton or your inference runtime, depending on compatibility requirements
• Secrets: cloud KMS backed by an HSM
• Attestation: hardware quote → policy verifier → conditional key release
• Network: mTLS with keys established only after successful attestation
• Model: encrypted at rest and encrypted until it reaches the TEE
• Logging: metadata/audit events only; never prompts, completions, or decrypted model material
• Operator isolation: cloud/platform administrators should not possess the keys needed to decrypt the model or inference data

The particularly important distinction is confidentiality from the cloud operator versus merely encryption at rest/in transit. Confidential computing protects data while it is actually being processed; Azure's documentation explicitly describes this model for confidential AI and remote verification of the inference environment.

If you tell me your cloud (AWS/Azure/GCP/on-prem), model size (e.g. 7B/70B/405B), throughput target, and threat model—especially whether you need protection from the cloud provider itself—I can give you a concrete architecture and deployment stack.

What I would build

The critical security property

Don't simply put the model inside a "confidential VM." Gate access to the model decryption key on attestation.

For example:

1. Build and sign your inference image.
2. Compute/record the expected measurements of the VM, runtime, inference server, and relevant GPU components.
3. Boot the workload inside the confidential TEE.
4. Generate a hardware-backed attestation quote.
5. A verifier checks that quote against your policy.
6. Only after successful verification does your KMS/HSM release the model-encryption key.
7. The model is decrypted inside the TEE.
8. Client prompts are sent over an authenticated encrypted channel directly into the protected workload.
9. Responses leave only through an explicitly controlled output path.

Azure Attestation, for example, can validate TEE evidence and issue cryptographic claims that a relying party can use for key release.

Running confidential AI inference requires hardware-backed Trusted Execution Environments (TEEs) to protect both your sensitive input data/prompts and the proprietary model weights while in active use.

Depending on whether you need CPU-based isolation or high-performance GPU acceleration, several leading solutions fit different architectural needs:

• NVIDIA Confidential Computing (with H100/H200 GPUs): Best for high-performance deep learning and large language model (LLM) inference. It extends hardware root-of-trust directly into the GPU memory and secures the PCIe link between the CPU and GPU. Available via cloud instances like Azure Confidential VMs.
• AMD SEV-SNP & Intel TDX Confidential VMs: Ideal for whole-VM memory encryption isolating standard CPU-bound inference workloads or smaller models. Major hyperscalers (Azure, GCP, AWS) offer specialized confidential VM families powered by these technologies.
• AWS Nitro Enclaves: Great if you are native to AWS and running lightweight or moderately sized models. It creates an isolated, CPU-and-memory-locked VM attached to an EC2 instance, though it lacks direct native high-end GPU passthrough for massive LLMs compared to full Confidential VMs.
• Fortanix Confidential AI: A specialized software and orchestration layer designed to manage secure enclaves, cryptographic attestation, and policy enforcement across multi-cloud or on-prem environments without exposing model weights or prompts.
• BlindAI: An open-source/enterprise tool built specifically for secure remote ML inference leveraging Intel SGX enclaves to ensure privacy during client-server interactions.

To help narrow down the ideal architecture, could you share:

• Are you running large language models (LLMs) requiring GPU acceleration , or smaller models on CPUs?
• Which cloud provider (AWS, Azure, GCP) or on-premises infrastructure are you targeting?

If your goal is confidential AI inference—the model, prompts, outputs, and encryption keys should remain protected even from the cloud host/operator—I’d recommend a GPU-backed confidential VM/TEE with remote attestation and attestation-gated key release, rather than a traditional CPU-only enclave.

Best practical architecture

Client → TLS → Confidential VM + confidential GPU → TLS → client

Inside the confidential environment:

1. Run your inference server (vLLM, TensorRT-LLM, Triton, etc.).
2. Use a TEE such as AMD SEV-SNP or Intel TDX for the VM.
3. Use a confidential GPU where available so model/data transfers to the GPU are protected.
4. Generate an attestation report containing measurements of the trusted boot/runtime environment.
5. Have a remote attestation service verify those measurements.
6. Release the model-encryption key only after successful attestation.
7. Keep prompts, model weights, intermediate tensors, and outputs inside the TEE.
8. Never bake plaintext model keys or secrets into the VM/container image.

This is essentially a “decrypt only inside an attested enclave” design.

My first choice: Azure Confidential GPU

For a new deployment, Azure Confidential GPU is particularly interesting because Microsoft offers VM configurations combining AMD SEV-SNP with NVIDIA H100 GPUs. Microsoft describes the TEE as spanning the confidential VM and attached GPU, protecting data, models, and computation during GPU offload.

Azure's confidential VMs support both AMD SEV-SNP and Intel TDX, with attestation and virtual TPM facilities.

A good architecture would therefore be:

                    ┌──────────────────────────────┐
                    │        Your KMS / HSM        │
                    │  Encrypted model key (KEK)   │
                    └──────────────┬───────────────┘
                                   │
                         release after attestation
                                   │
Client ──TLS──► ┌──────────────────▼─────────────────┐
                │       Confidential VM / TEE         │
                │                                     │
                │  ┌────────────┐   ┌──────────────┐ │
                │  │ Inference  │──►│ Confidential │ │
                │  │ server     │   │ NVIDIA H100  │ │
                │  └────────────┘   └──────────────┘ │
                │          │                          │
                │     plaintext model                 │
                │     + prompts + outputs             │
                │          │                          │
                └──────────┼──────────────────────────┘
                           │
                         TLS
                           │
                         Client

The important security property is that the KMS doesn't simply give the VM the key because it has the right IAM identity. It gives it the key because it can cryptographically demonstrate that the expected confidential workload is actually running.

Azure Attestation can validate TEE evidence and issue cryptographic claims, and Azure supports using those claims for secure key release.

AWS alternative

If you're already on AWS, Nitro Enclaves are an excellent option for particularly sensitive portions of the inference stack. They provide isolated CPU/memory, have no external networking or persistent storage, and support cryptographic attestation.

Their strongest pattern is:

EC2 parent
   │
   ├── networking / API
   │
   └── Nitro Enclave
          │
          ├── inference workload
          ├── secrets
          └── model/data
                │
                ▼
             AWS KMS
       "release only to
        this attested enclave"

AWS KMS can evaluate the enclave's attestation measurements and restrict decryption to an authorized enclave image.

The catch is that GPU-heavy AI inference makes Nitro Enclaves less straightforward than a purpose-built confidential-GPU VM, so I'd choose Azure's confidential-GPU route if GPU inference is the central requirement.

What I'd implement

For a production system I'd use:

• TEE: AMD SEV-SNP + confidential GPU
• GPU: NVIDIA H100-class confidential GPU
• Inference: vLLM or TensorRT-LLM
• Model: encrypted at rest
• Key management: customer-controlled KMS/HSM
• Trust: remote attestation
• Key release: conditional on measured boot + workload measurements
• Network: mTLS
• Application identity: workload identity inside the TEE
• Logging: explicitly prevent prompts/completions from entering ordinary cloud logs
• Images: reproducibly built and cryptographically signed
• Deployment: immutable image → measured boot → attestation → key release → inference

One important distinction

A confidential VM is not automatically a confidential AI system. You need to ensure the entire path is covered: CPU memory, GPU memory/transport, model keys, storage, networking, telemetry, crash dumps, and operational access.

For example, Azure's documentation explicitly distinguishes the confidential workload/TEE from components outside its trusted computing base.

If you tell me (1) AWS/Azure/on-prem, (2) which GPU, (3) model size, and (4) whether you need to protect against the cloud provider itself, I can give you a concrete reference architecture and deployment stack.

Azure Attestation can validate TEE evidence and issue cryptographic claims, and Azure supports using those claims for secure key release.

AWS alternative

If you're already on AWS, Nitro Enclaves are an excellent option for particularly sensitive portions of the inference stack. They provide isolated CPU/memory, have no external networking or persistent storage, and support cryptographic attestation.

Their strongest pattern is:

EC2 parent
   │
   ├── networking / API
   │
   └── Nitro Enclave
          │
          ├── inference workload
          ├── secrets
          └── model/data
                │
                ▼
             AWS KMS
       "release only to
        this attested enclave"

AWS KMS can evaluate the enclave's attestation measurements and restrict decryption to an authorized enclave image.

The catch is that GPU-heavy AI inference makes Nitro Enclaves less straightforward than a purpose-built confidential-GPU VM, so I'd choose Azure's confidential-GPU route if GPU inference is the central requirement.

What I'd implement

For a production system I'd use:

Yes. For confidential AI inference, I’d recommend a GPU-backed confidential computing architecture, rather than a traditional CPU-only enclave.

Best options

• Azure Confidential GPU + NVIDIA H100 — probably the easiest managed option. Azure’s NCCadsH100v5 combines an AMD SEV-SNP confidential VM with an NVIDIA H100 Confidential Computing GPU, so the CPU and GPU participate in the protected TEE.
• NVIDIA Confidential Computing on H100/Blackwell — best if you want maximum GPU flexibility or are building your own confidential-AI infrastructure. NVIDIA provides hardware-rooted GPU attestation and protection for models, prompts, and inference data.
• AWS Nitro Enclaves — excellent for protecting particularly sensitive pre/post-processing and key-management logic, but it isn't by itself a GPU confidential-inference solution. Nitro Enclaves are isolated CPU/memory environments with no external networking and strong cryptographic attestation.

Architecture I'd use

Client
  │
  │ TLS
  ▼
API / Gateway
  │
  │ encrypted request
  ▼
Confidential VM
  │
  ├── Attestation
  │       │
  │       └── KMS/HSM releases inference key
  │
  ├── Encrypted model
  │
  └── Confidential GPU (H100/Blackwell)
          │
          └── Model inference
                 │
                 ▼
          Encrypted response

The important part is attestation-gated key release. Don't simply put the model or encryption keys inside the VM image. The workload should prove cryptographically that the expected VM/GPU/software stack is running, and only then receive the key needed to decrypt the model or sensitive inputs. AWS, for example, supports binding KMS authorization to enclave measurements.

Security properties to require

For a serious deployment, I'd make these requirements non-negotiable:

1. CPU TEE: AMD SEV-SNP or Intel TDX.
2. Confidential GPU: NVIDIA H100 or newer GPU with Confidential Computing.
3. Remote attestation: verify both the host/VM and GPU before releasing secrets.
4. External KMS/HSM: model and data keys should not be permanently present in the enclave.
5. Encrypted model at rest and in transit.
6. No plaintext logging: prompts, outputs, tokens and model weights must be excluded from application/telemetry logs.
7. Measured boot / signed container images.
8. Key rotation and revocation tied to attestation measurements.
9. Minimal privileged software: keep the TCB as small as practical.
10. Explicit threat model: decide whether you're protecting against the cloud operator, hypervisor, host OS, other tenants, compromised application components, or all of them.

One subtle point: “confidential VM” does not automatically mean “confidential AI.” You need the GPU itself protected. Azure's H100 offering is interesting precisely because the TEE spans the confidential VM and attached GPU.

If you tell me (a) AWS/Azure/on-prem, (b) which model—e.g. Llama, Mistral, proprietary—and (c) whether you need to protect the model weights, user prompts, or both, I can give you a concrete production architecture, including attestation, KMS/HSM, container runtime, and deployment choices.

Running confidential AI inference requires Trusted Execution Environments (TEEs) or secure enclaves that protect both your input data (prompts/user data) and the model's proprietary weights from the underlying infrastructure, cloud operator, or host.

The industry landscape for secure AI inference spans hardware-accelerated GPU enclaves, Confidential VMs, and application-layer orchestration tools.

Major Enterprise Solutions

• NVIDIA Confidential Computing : Hardware-level isolation for GPU-accelerated AI inference. It secures data-in-use on the GPU, ensuring that memory utilized during large language model (LLM) or vision model inference is encrypted and hardware-protected against host-level tampering. Available via major cloud providers and on-premises.
• Google Cloud Confidential Computing : Offers Confidential VMs and Confidential GKE nodes powered by AMD processors and advanced NVIDIA GPUs (such as Blackwell/H100/L4 lines), paired with open-source prompt encryption SDKs for end-to-end secure inference channels.
• Azure Confidential Computing : Uses AMD SEV-SNP and Intel TDX combined with confidential GPU offerings. It allows you to isolate containerized AI inference workloads (e.g., via AKS) so that even root administrators or hypervisors cannot dump the memory containing model weights or active requests.
• Fortanix Confidential AI : A software layer that integrates with NVIDIA Confidential Computing to allow multi-party AI inference. It lets enterprises run proprietary third-party models on sensitive local data with cryptographic verification, ensuring the model vendor doesn't see user data and the user doesn't extract model weights.

Key Architectural Approaches

1. Confidential VMs (CVMs) / Full-Node TEEs : AMD SEV-SNP or Intel TDX wrap the entire virtual machine instance in memory encryption. This is ideal if you are lifting and shifting existing Triton or vLLM inference servers into a secure cloud environment.
2. GPU-Enforced TEEs : Essential for high-performance deep learning inference where CPU enclaves would introduce a severe performance bottleneck.
3. Attestation Services : Whichever path you choose, you will rely on a Remote Attestation Service (such as NVIDIA Attestation Solutions or cloud-native equivalents) to cryptographically prove to clients that the enclave running the model is genuine and uncompromised before releasing decryption keys or sensitive prompts.

To help narrow down the best architecture, could you share:

• Are you deploying on-premises or on a public cloud (AWS, Azure, GCP)?
• Do you need hardware acceleration (GPUs ), or are you running smaller CPU-based models?
• Is your primary concern protecting user data/prompts or protecting model IP/weights?

Yes. For confidential AI inference, I’d recommend a GPU-backed Trusted Execution Environment (TEE) rather than a traditional CPU-only enclave.

Recommended architecture

Client → attested gateway → Confidential VM/container → confidential GPU → model

The key pieces are:

• TEE: AMD SEV-SNP or Intel TDX protects the VM's CPU memory from the cloud host/hypervisor.
• Confidential GPU: NVIDIA H100/Hopper and newer supported GPUs extend the protection boundary into GPU memory, which is critical because model weights, prompts, and activations can reside in VRAM.
• Remote attestation: Your client verifies cryptographically that the expected VM image, workload, and GPU configuration are running before releasing the model/data encryption key. NVIDIA's Confidential Containers architecture specifically supports composite CPU + GPU attestation and attestation-based key release.
• Encrypted transport: Use mTLS/attested TLS between the client and inference workload.
• Key release: Keep model weights encrypted at rest and release the decryption key only after successful attestation.
• Minimal host trust: The cloud operator should not have access to plaintext prompts, outputs, model weights, or enclave memory.

Good implementation choices

NVIDIA's H100 confidential-computing support is mature enough for production: single-GPU passthrough became generally available with CUDA 12.4, with AMD SEV-SNP and Intel TDX supported as the CPU-side TEE.

AWS Nitro Enclaves are extremely isolated—they have no persistent storage, interactive access, or external networking, and even root on the parent instance cannot access enclave contents. They're therefore useful for things like key management, tokenization, or sensitive CPU-side inference, but they're a different model from a GPU TEE.

There are also turnkey offerings such as confidential.ai that expose OpenAI-compatible confidential inference while providing attestation mechanisms.

What I would build

If you're protecting enterprise prompts + proprietary model weights, I'd use:

H100/B200 → SEV-SNP/TDX Confidential VM → NVIDIA Confidential Computing → Kata/Confidential Containers → attestation/KBS → encrypted model weights → vLLM/SGLang inference

The important security property is not merely "the inference server is in an enclave." You want the entire path—CPU memory, GPU VRAM, model weights, prompts, intermediate activations, and keys—to remain inside an attested trust boundary. GPU confidential computing closes the otherwise significant VRAM gap.

If you tell me (1) cloud vs on-prem, (2) model size, (3) latency/throughput target, and (4) whether you need to protect the model, the prompts, or both, I can propose a concrete architecture and hardware/software stack.

Running confidential AI inference requires hardware-backed Trusted Execution Environments (TEEs) that secure your model weights, prompts, and KV-caches while in active use. Depending on whether you are running CPU-bound smaller models or heavy GPU-accelerated LLMs, several robust enterprise solutions fit this architecture.

• Fortanix Confidential AI : Specializes in multi-party and enterprise setups where proprietary model weights and sensitive client inputs remain encrypted and hidden from infrastructure operators using strong cryptographic guarantees.
• AWS Nitro Enclaves : Ideal if your infrastructure is on AWS and you are running lighter pre- or post-processing or smaller models. It allows you to isolate CPU compute and tie decryption keys directly via AWS Key Management Service (KMS) using PCR measurement attestation.
• NVIDIA Confidential Computing (H100/H200/B200 GPU TEEs) : The gold standard for heavy LLM inference. It extends hardware root-of-trust encryption directly into the GPU High Bandwidth Memory (HBM), keeping activations, weights, and inference outputs secure from the host OS and hypervisor.
• Red Hat OpenShift Sandboxed Containers : A cloud-native orchestration layer integrating hardware TEE isolation seamlessly into enterprise Kubernetes clusters.

To help narrow down the ideal stack, tell me:

• Are you deploying heavy LLMs requiring GPU acceleration (like NVIDIA H100s) or smaller CPU-based models?
• Is your infrastructure hosted on-premises or on a specific cloud provider (AWS, Azure, GCP)?

Running confidential AI inference inside a secure enclave requires a Trusted Execution Environment (TEE) that can handle heavy model weights, context windows, and—crucially—accelerated hardware like GPUs . Because standard CPU-only enclaves struggle with the massive parallel throughput required for LLMs, modern confidential AI architecture chains CPU TEEs with Confidential GPU TEEs (such as NVIDIA H100/H200 with confidential computing modes).

The top production-ready solutions and platforms available depend on whether you want a managed cloud infrastructure, specialized zero-trust platforms, or an enterprise on-premise/hybrid setup:

1. Dedicated Confidential Cloud & Zero-Trust Platforms

• Phala Cloud: Offers zero-trust TEE infrastructure specifically built for decentralized and confidential AI. It natively supports modern CPU TEEs (Intel TDX, AMD SEV-SNP ) alongside NVIDIA H100/H200 GPU TEEs , complete with public remote attestation and specialized developer SDKs (dstack).
• SCONE (by Seconply): A software-level confidential computing platform that allows you to run unmodified AI applications and frameworks inside TEE enclaves (like Intel SGX/TDX). It handles cryptographic shielding for data-in-use across hybrid and multi-cloud environments.

2. Hyperscaler Confidential Computing

• Microsoft Azure Confidential Computing: Generally considered the most mature hyperscaler ecosystem for end-to-end confidential AI. It combines AMD SEV-SNP or Intel TDX virtual machines with integrated Azure Attestation and expanding support for confidential GPU instances.
• AWS Nitro Enclaves: Excellent for isolated, hypervisor-level CPU security, but lacks native high-performance GPU TEE passthrough for heavy LLM inference. Best suited if your inference workload is lightweight or runs purely on CPU-based models.
• Google Cloud Confidential VMs: Powered primarily by AMD SEV/SEV-SNP, though native, end-to-end confidential GPU orchestration trails behind Azure and specialized providers.

3. Open-Source & Containerized Tooling

• Red Hat OpenShift Sandboxed Containers: Integrates hardware-backed TEE cryptographic boundaries into cloud-native Kubernetes environments, allowing you to orchestrate private data inference safely inside enterprise clusters.

To help narrow down the right architecture, let me know:

• Are you deploying on public cloud (Azure/AWS) or looking for a specialized/on-premise setup?
• What size and type of model are you running (e.g., small CPU-bound model vs. large multi-GPU LLM)?

Yes. For confidential AI inference, I’d recommend a GPU-backed Trusted Execution Environment (TEE) rather than a traditional CPU-only enclave.

Recommended architecture

Best general-purpose choice: NVIDIA H100 Confidential Computing + AMD SEV-SNP

• Run the inference server inside an AMD SEV-SNP Confidential VM.
• Use an NVIDIA H100 in Confidential Computing mode, extending the trust boundary into GPU memory and execution. NVIDIA supports this architecture for confidential inference and says applications generally need no code changes.
• Require remote attestation before releasing secrets—model decryption keys, API credentials, or data-encryption keys should only be released to an attested workload.
• Encrypt client → inference service traffic normally, and ideally use an application-level protocol such as HPKE so plaintext requests only exist inside the trusted environment.
• Keep the model weights encrypted at rest; decrypt them only after successful attestation.
• Disable or tightly control logging, debugging, shell access, crash dumps, and telemetry that could expose prompts or model outputs.

A useful trust flow is:

Client
  │
  │ encrypted request
  ▼
┌─────────────────────────────────────────────┐
│ Confidential VM / TEE                       │
│                                             │
│  ┌───────────────┐     ┌────────────────┐  │
│  │ Attested      │────▶│ Encrypted model│  │
│  │ inference     │     │ weights        │  │
│  │ service       │     └────────────────┘  │
│  └───────┬───────┘                          │
│          │ encrypted/integrity-protected    │
│          ▼                                  │
│     ┌─────────────┐                         │
│     │ H100 GPU    │  ← confidential mode   │
│     └─────────────┘                         │
└─────────────────────────────────────────────┘
          │
          │ encrypted response
          ▼
       Client

Where to deploy it

Azure is probably the easiest managed option. Azure offers Confidential VMs combining AMD SEV-SNP with NVIDIA H100 GPUs, with the TEE covering the CPU VM and GPU. Azure also documents remote attestation and confidential inferencing.

For an on-prem/hybrid deployment, NVIDIA's H100 confidential-computing stack is a strong choice. NVIDIA supports AMD SEV-SNP and Intel TDX-based platforms and provides hardware-rooted attestation.

What about AWS Nitro Enclaves?

AWS Nitro Enclaves are excellent for isolating highly sensitive CPU workloads and provide cryptographic attestation, but I wouldn't make them the primary architecture for large GPU inference. AWS's Nitro model is particularly attractive for things such as a key broker/attestation service sitting alongside your GPU inference infrastructure.

If you want a turnkey service

There are also specialized confidential-AI providers. For example, Confidential AI currently offers GPU VMs using AMD SEV-SNP or Intel TDX, including H100, B200 and B300 configurations.

My recommendation

For a production system handling highly sensitive prompts/data or proprietary model weights, I'd choose:

H100 Confidential Computing + SEV-SNP → remote attestation → KMS/HSM key release → encrypted model → inference → encrypted response.

The important part isn't merely "put the model in an enclave." The attestation + key-release policy is what prevents an untrusted host, hypervisor, or unauthorized workload from simply obtaining the model or plaintext data.

If you tell me your cloud/on-prem preference, model size (e.g. Llama 70B), throughput requirement, and threat model (cloud provider, infrastructure admin, other tenants, or all of them), I can design the concrete architecture and component choices.