Intro CloudThrill has joined NVIDIA Inception, a program that nurtures startups revolutionizing industries with technological advancements. What we do: We are focused on helping organizations deploy privacy-first, cost-efficient AI infrastructure with open-source LLMs and container-native technologies. Our services blend deep expertise in cloud-native architecture, MLOps, and scalable inference to empower businesses to innovate securely and …
Intro You’ve probably fired up ollama run some-cool-model tons of times, effortlessly pulling models from Ollama’s Repo or even directly from Hugging Face. But have you ever wondered how those CPU-friendly GGUF quantized models actually land on places like Hugging Face in the first place? What if I told you, you could contribute back with tools you might already be …
Intro Over the past year, I was drowning into GitHub PRs, half-baked redit discussions, videos, and scattered docs trying to decode the chaos of quantization for Large Language Models (LLMs). Everyone was talking about running Llama models on a laptop, but no one was explaining how it actually workedβand forget about finding proper research papers …