Skip to content

CPU and GPU Flavors#

concept

Workloads that do not need GPUs run on the base flavor; GPU workloads run on the GPU flavor. RDMA is its own layer, because storage and data-loading nodes need the fabric without GPUs.

Flavor Base Fabric GPU AI-Ops Typical nodes
k8or-os ● — — optional control plane, APIs, general services
k8or-os + fabric ● ● — optional storage clients (Weka, Lustre), CPU data loaders
k8or-os-gpu ● ● ● optional H100 / H200 training and inference nodes

All three come from one pipeline and one kernel, with one version number. They differ only in which layer images the node's config lists.

How Kubernetes places work#

  • The flavor is chosen when a node is provisioned: a GPU server boots with the GPU layer listed.
  • Node Feature Discovery labels nodes with their hardware and layers.
  • GPU nodes carry a taint; only pods that tolerate it and request GPUs land there.
  • General workloads never consume expensive accelerator capacity.
gpu-pod.yaml
spec:
  tolerations:
    - key: nvidia.com/gpu
      operator: Exists
      effect: NoSchedule
  containers:
    - name: train
      resources:
        limits:
          nvidia.com/gpu: 8

The NVIDIA GPU Operator, trimmed#

Because the OS already ships the driver and the container toolkit:

  • the Operator's driver and toolkit containers are disabled;
  • the device plugin and DCGM exporter stay, to advertise GPUs and export metrics;
  • nothing compiles or loads drivers at runtime, and nothing races kubelet.
gpu-operator values.yaml
driver:
  enabled: false     # the k8or GPU layer provides the driver
toolkit:
  enabled: false     # the k8or GPU layer provides the container toolkit
devicePlugin:
  enabled: true
dcgmExporter:
  enabled: true

Room for other accelerators#

Because flavors are compositions of layers, an AMD ROCm or Intel Gaudi layer later is one more layer on the same base, not a fourth operating system.