AI-Ready Microservice Architecture

If you've ever followed an AI tutorial, you may have noticed the absence of crucial aspects necessary for successfully deploying AI models in production. Implementing these missing elements as an afterthought can be challenging. However, deploying and evaluating AI models in production can become less challenging with a well-established microservice architecture. In this presentation, we will showcase how to evaluate and deploy models using an event-driven feedback loop through Knative, KServe, and Kubeflow. Event-driven architecture (EDA) is a design approach that effectively addresses the difficulties of deploying and managing AI models in a scalable manner within a production environment. EDA naturally complements AI workloads. During the talk, we will have a demo where we will utilize cameras to capture images or video streams. With the help of an event-driven architecture, we will perform real-time image recognition tasks and engage attendees in evaluating the models.