AI infrastructure architecture guides
Vendor-neutral explanations of the runtime and infrastructure layers behind production LLM inference on Kubernetes.
Explore a vendor-neutral AI platform architecture for production LLM inference, including control plane, data plane, runtime dependencies and GPUs.
Read guide →02 — Traffic entry and policyAI Gateway Architecture on KubernetesLearn how an AI gateway uses Gateway API, HTTPRoute, external processors, Service and InferencePool backends for production inference traffic.
Read guide →03 — Model serving runtimeLLM Inference ArchitectureUnderstand LLM inference architecture from request queue and scheduler through prefill, decode, KV-cache management and token streaming.
Read guide →04 — Runtime endpoint selectionInference-Aware Routing on KubernetesLearn why LLM workloads need inference-aware routing and how InferencePool and an Endpoint Picker select serving replicas using runtime signals.
Read guide →05 — Accelerator resourcesGPU Management on KubernetesSee how Kubernetes discovers, configures, partitions, shares and allocates GPU resources to model inference workloads.
Read guide →06 — Hardware executionHow GPU Inference WorksLearn how GPU inference uses high-bandwidth memory and parallel tensor compute, and why compute, bandwidth and memory capacity limit performance.
Read guide →07 — Platform operating modelControl Plane vs Data Plane in an AI PlatformUnderstand how an AI platform control plane configures and reconciles infrastructure while the data plane handles live model inference traffic.
Read guide →