GPU on Kubernetes: How AI Teams Run Workloads Without Breaking the Budget

GPUs are expensive — and most teams waste over 80% of what they pay for. In this session, Aditya Krishnakumar (Senior SRE at SentinelOne) shares a real-world story of running bursty AI workloads on Kubernetes at a fraction of the cost. You'll learn how to combine three upstream Kubernetes primitives — the NVIDIA GPU Operator, Dynamic Resource Allocation (DRA), and GPU time-slicing — to let multiple pods share a single physical GPU without sacrificing latency. The session covers how DRA enables declarative GPU sharing as a first-class Kubernetes API, how autoscaling delivers true scale-to-zero, and pitfalls to avoid along the way.

Real results: 65–70% reduction in GPU spend, 4× effective utilization, flat p95 latency.

Audience: Platform engineers and SREs running AI workloads on Kubernetes. Intermediate level.