Intro The moment your app calls more than one model provider, the plumbing gets ugly. Different SDKs, different auth, no shared view of spend, and when one provider has a bad afternoon your bot just goes dark. The fix is an LLM gateway: one endpoint in front of every provider that handles routing, fallbacks, cost …
Read more “LLM Routing Solutions: Open Source vs Commercial”
Intro When scaling AI models like DeepSeek or Qwen on Amazon EKS, engineering teams obsess over GPU utilization while quietly bleeding money on storage bloat. Because standard EBS volumes force a 1:1 replica-to-disk ratio, scaling a single 70GB model to 20 pods doesn’t cost 70GB, it forces you to provision 1.4 Terabytes of redundant EBS …
Read more “vLLM on EKS: Cut LLM Storage Costs by 95% with S3 Mountpoint”
Intro Deploying vLLM manually is fine for a lab, but running it in production means dealing with Kubernetes, autoscaling, GPU orchestration, and observability. Thatโs where the vLLM Production Stack comes in – a Terraform-based blueprint that delivers production-ready LLM serving with enterprise-grade foundations. In this post, we’ll deploy it on Amazon EKS, covering everything from …
Read more “vLLM Production Stack on Amazon EKS with Terraform๐ง๐ผโ๐”
Intro Did you know that 12 millions secrets were publicly exposed in GitHub in 2023 alone? This is additional evidence that leaked secrets rhyme with financial and reputation loss for users, organizations and even states. The worst thing to do is make it easy for hackers to infiltrate your company’s system. This is where workload …
Read more “Terraform Pipelines for Dummies Part2: GitHub Actions AWS Deploy (OIDC)”
Intro Recently, I passed the AWS SAA certification, and while sharing your achievement on socials after long months of preparation and grind can be gratifying, some Pearson Vue test takers, didnโt have the same smooth experience, for unexpected challenges due to OnVue software glitches.In this short post, Iโll take you through the surreal blunders of …
Read more “How to Survive Pearson Vue exam glitch nightmare”