AI Platform Architecture
A production AI platform separates platform configuration from live inference execution. This reference architecture shows how requests move through an AI gateway, model inference runtime and GPU layers while Kubernetes controllers continuously reconcile the desired platform state.
Explore the interactive architecture →Control plane and platform intent
The control plane turns version-controlled configuration into runtime resources. GitOps delivery and Kubernetes controllers create or update gateways, routes, model-serving resources and GPU policies, then keep actual state aligned with desired state.
Data plane and live inference traffic
The data plane handles each active request. It authenticates and governs traffic, chooses the right model and serving endpoint, executes inference, and returns generated tokens. Latency, throughput and availability depend directly on these request-time decisions.
Runtime dependencies
Identity, rate limits, guardrails, inference routing, vector databases, object storage and model registries provide specialized capabilities. They can be used during request processing or model lifecycle operations without becoming mandatory sequential stages of every inference request.
Basic LLM, RAG and agentic flows
A basic LLM flow routes a request to model inference. RAG adds retrieval from an external knowledge system before generation. An agentic flow adds a bounded runtime loop and authorized tool calls while preserving the same gateway, inference and infrastructure foundations.