Intro A few months back I deployed the vLLM Production Stack on Nebius Managed K8s with Terraform. This time itโs NVIDIA Dynamo on the same foundation, one terraform apply from an empty cloud project to a live OpenAI-compatible endpoint. Dynamo is a very different beast from vllm-production-stack: a Helm chart installs an operator, and each …
Read more “NVIDIA Dynamo on Nebius Managed Kubernetes with Terraform”
Intro AI capex 2026 is $401B, while average GPU utilization across 23k+ Kubernetes clusters sits near 5%. That gap is the ugly truth no one’s talking about.In my recent blog vLLM DeepSeek deployment, we sharded a 643GB whale across 16 H100s because it was too big to fit on one node. Today we’re facing the …
Intro In the LLM routing post we wired a single gate in front of every model and promised a follow up for the part that actually keeps you out of trouble: guardrails. A guardrail is the thing that reads a prompt or a response and decides whether it gets through, gets masked, or gets dropped. …
Read more “LLM Prompt Guardrails Solutions: Open Source vs Commercial”
Intro The moment your app calls more than one model provider, the plumbing gets ugly. Different SDKs, different auth, no shared view of spend, and when one provider has a bad afternoon your bot just goes dark. The fix is an LLM gateway: one endpoint in front of every provider that handles routing, fallbacks, cost …
Read more “LLM Routing Solutions: Open Source vs Commercial”
Intro Conference demos always run out of clock, you show one or two outputs and the rest stay on the cutting-room floor. That’s what happened at my last Conf42 talk, How vLLM-Omni Unifies Multimodal Inference, so to make up for it I promised to share the whole code and demo videos in a proper blog. …
Read more “vLLM-Omni on Nebius H100: Serving Z-Image, Wan2.2, Qwen3-TTS & NVIDIA Cosmos 3”