Category: AI

Intro Today, most of us have used nano banana, Midjourney, Kling AI, Luma, or Sora to generate silly videos or catchy images on socials. But what do they share in common? They all rely on Diffusion Models as their core engine, even the brand new Seedance. While many of these are proprietary, the open-source world …

Read more “Diffusion Models 101: From Noise to Pixels”

Intro A few weeks ago I wrote about why Intel Optane Persistent Memory was the ideal technology for LLM KV-cache offloading with a near-DRAM latency, and natively non-volatile. In other words, it behaved like memory but survived reboots. I also explained why CXL wasn’t quite the performance equivalent, due to higher latency and non persistence. But recently …

Read more “NVIDIA’s 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗠𝗲𝗺𝗼𝗿𝘆 𝗦𝘁𝗼𝗿𝗮𝗴𝗲 (CMX): The KV Cache War”

Intro Any-to-any multimodal models combining text, images, video, and audio are advancing AI, but their complex architectures, mixing autoregressive LLMs and diffusion transformers, make efficient serving very difficult. Current systems like OpenAI’s ChatGPT (text) and Sora (video) run as separate engines, lacking unified any-to-any pipelines. vLLM-Omni solves just that with a fully disaggregated serving system …

Read more “What is vLLM-Omni? Beginners Intro”

Intro In the previous post, we explored how the vLLM Production-Stack upgrades vanilla vLLM engine to an enterprise-grade platform. This time, we’ll crack open the Helm chart, decoding the key knobs in values.yaml and showing deployment recipes that span from a minimal install to full cloud setups. Acknowledgment: While authored independently, this series benefited from …

Read more “vLLM production-stack: Deployment in the cloud (part2)”