February Virtual Meetup - Transforming KServe Into a Zero Trust

KServe is a mature platform for model serving. However, securing machine learning inference remains a fundamental challenge. ML models are like a black box. One can not just easily find if a model has been poisoned or tampered with.

The talk begins with an introduction to KServe. Next we will cover real-world case studies of ML security failures highlighting common attack vectors and production grade security issues in ML systems.

Next, we introduce KitOps, a CNCF project that enables a zero-trust approach to ML inference on KServe. By using ModelKits: signed, OCI-native bundles that package models, metadata, configurations, and dependencies, we can establish verifiable trust throughout the model lifecycle. ModelKits can be stored in any OCI-compatible registry and seamlessly integrated into existing CI/CD pipelines.

Finally, we will walk through production patterns for hardening KServe deployments using familiar cloud-native tools: cryptographic model signatures, policy engines for enforcement, and AI Bill of Materials (AIBOM) for auditability and compliance.

Thus you will learn how to harden KServe using the similar set of cloud native tools already at your disposal.