Laravel & AI

Currently Playing: Failover, Queues, and Middleware for AI Agents - Ship AI with Laravel EP9

Our platform works in development. But what happens when OpenAI goes down at 2am, when hundreds of customers hit the chat at once, or when you need to figure out why a response was wrong three weeks ago? This episode is about the infrastructure that keeps things running. First, provider failover. Right now the agent is locked to one provider, so an OpenAI outage means every customer gets an error. I show how to pass an array of providers so the SDK automatically retries with the next one if the first fails. I set it up at the call site first, then centralize the failover chain in config so you can change it without touching code. I also wire up a listener for the AgentFailedOver event so you get alerted the moment a provider goes down. Next, queues. Not every AI call needs to happen in real time. I take the TicketClassifier from Episode 2 and swap prompt for queue so classification runs in the background through Laravel's queue system, with failover support on the queued job too. Then, middleware. The SDK lets you intercept every prompt and response flowing through an agent, just like HTTP middleware. I build three: one for logging prompts and response metadata, one for per-user rate limiting at 10 prompts a minute, and one for cost tracking with token counts. Then I attach all three to the support agent in the right order, since rate limiting should run before you bother logging or tracking a request you're about to reject. ⭐ GitHub: https://github.com/harris21/ship-ai-with-laravel 👉 Full series playlist: https://www.youtube.com/playlist?list=PLsmlCYEtWNI_plnKPPO13XjfDJoMIfhLN 👉 Timestamps 00:00 Production Ready Goals 00:22 Provider Failover Setup 01:57 Configurable Failover Chain 03:20 Failover Alerts Listener 04:46 Queueing AI Workloads 06:08 Agent Middleware Basics 06:23 Logging Middleware 07:07 Rate Limiting Middleware 07:54 Cost Tracking Middleware 08:45 Attach Middleware to Agent 09:43 Next Episode Testing ---------------------------------------------------------------------- ➫ Business Inquiries: hello@laravel-news.com ---------------------------------------------------------------------- JOIN OUR WEEKLY LARAVEL NEWSLETTER 👉 https://laravel-news.com/newsletter ----------------------------------------------------------------------