Hardware and Sizing¶
Release binding: v0.1.0-enterprise-ready · AMI ami-010806d4d3445660e · 2026-05-19
GPU classes¶
| Class | GPUs | Support posture | Status |
|---|---|---|---|
| A | L4, A10 | Pilot and smoke, small private corpora, installer validation | Smoke-ready |
| B | L40S, A100 | Production pilot | Measured on 4x A100 PCIe |
| C | H100, H200, B200 | High-scale single node; qualify the deployment release and workload | H100 measured; B200 measured for the synthetic latent-only envelope; H200 remains gated |
| D | Multi-node | Strategic and private runtime | Gated |
Capacity statements for unmeasured hardware or workload combinations are estimates only. A benchmark on a GPU does not qualify every appliance image, driver, corpus, or serving policy on that GPU. See Performance Evidence for the measured B200 synthetic envelope and the separate H100 real-corpus results; confirm the supported deployment configuration during Private Appliance or pilot scoping.
Base OS and runtime baseline¶
| Component | Requirement |
|---|---|
| Base OS | Ubuntu Server 24.04 LTS |
| Python | 3.12 runtime in the appliance venv |
| CUDA runtime | CUDA 12 family |
| NVIDIA driver | Data-center branch compatible with CUDA 12 |
| CPU-only mode | Clean degradation and smoke mode only; not GPU serving |
Virtualization requirements¶
| Target | GPU mode | Requirement |
|---|---|---|
| KVM / Proxmox / OpenStack | PCIe passthrough | IOMMU enabled, NVIDIA device passed through to the guest, nvidia-smi visible |
| VMware ESXi / vSphere | PCIe passthrough or NVIDIA vGPU | VM hardware version and vGPU licensing must expose CUDA and NVML to the guest |
| Bare metal Ubuntu | Native NVIDIA driver | Controlled installer only |
| Kubernetes / OCI | NVIDIA Container Toolkit | GPU device plugin, runtime class, persistent volume, license and TLS secrets |
| MIG | Optional | Must be recorded in the GPU preflight output before MIG-specific claims |
| Secure Boot | Customer controlled | Supported when NVIDIA driver signing is validated for the environment |
AWS instance types¶
| Use case | Instance type | GPU | VRAM | RAM |
|---|---|---|---|---|
| Enterprise production (minimum) | g5.2xlarge |
A10G | 24 GB | 32 GB |
| High-throughput | g5.4xlarge and up |
A10G | 24 GB | 64 GB and up |
| Maximum corpus | p4d.24xlarge |
A100 | 320 GB | 1152 GB |
g5.xlarge is below the enterprise floor and must not be used for production. See Known Boundaries. CPU-only instances are not supported; the GPU preflight refuses to start without a compatible NVIDIA GPU.
Google Cloud machine types¶
| Machine type | GPU | GPU memory | Use case |
|---|---|---|---|
g2-standard-8 |
nvidia-l4 x 1 | 24 GB | small and medium production |
a2-highgpu-1g |
nvidia-tesla-a100 x 1 | 40 GB | GPU deployment candidate; size and validate the intended corpus |
a2-ultragpu-1g |
nvidia-a100-80gb x 1 | 80 GB | Larger GPU-memory candidate; size and validate the intended corpus |
n2-standard-8 |
none | n/a | evaluation only |
The instance names above are deployment references, not a claim that a given vector count fits. Dimension, rank, precision, SQ8 residency, query working memory, and release qualification determine the usable envelope. Confirm current cloud availability and machine quotas in the selected region.
Sizing¶
The appliance ships a sizing command. Every answer is labeled as one of: validated envelope, estimated envelope, or unsupported envelope.
hx-sdp sizing \
--rows <corpus_rows> \
--dim <embedding_dim> \
--gpu-vram-gb <gpu_vram> \
--host-pinned-gb <host_pinned_memory>
Validate hardware before sizing decisions:
The preflight records GPU visibility, driver branch, CUDA runtime, torch CUDA availability, VRAM, compute capability, MIG state, ECC, persistence mode, allocation, and a GEMM smoke result.