Cloudthrill
<p>Expertise worth sharing ...</p>

Our Mission...Challenge your potential

Our Solutions

We are a consulting team, not a platform vendor. Our engineers deploy, maintain and optimize your own AI inference stack wherever you choose: any cloud, on-prem or hybrid. Open-source serving, built in your account, so your data stays with you and every GPU earns its keep.

25+

IT & cloud certifications

55+

customers over the last decade

20+

speaking engagements

2

languages we deliver in (EN / FR)

What we deliver

Four ways we work with you, from a first proof of concept to a private inference stack in production.

YOUR ENVIRONMENTprompts and data stay insideGatewayGPU nodesvLLMSGLangLlamaQwenYour data

01 · Deploy

Private AI Inference Deployment

Your inference stack, built in your environment by our engineers.

We design and deploy a private inference stack on your infrastructure, then run it with your team or hand it over. You own everything we build.

  • Private by design: no data leaves your environment
  • Compliance-ready (GDPR, HIPAA), documented for your auditors
  • Okta / SSO sign-in and one gateway with guardrails in front of every model
  • The open models you choose, served with vLLM, SGLang, llm-d, NVIDIA Dynamo or KServe
  • Ongoing support after go-live: upgrades, model roll-outs and monitoring
Read: LLM guardrails: open source vs commercial →
tokens / sGPU UTILIZATIONsame traffic, fewer GPUs

02 · Optimize

LLM Serving Optimization

Faster tokens, fewer GPUs.

An engagement on the stack you already run: we benchmark it on your traffic, tune it, and leave your team the playbook.

  • Quantization strategy for your models
  • KV-cache sizing and cache-aware routing
  • GPU sharing and right-sizing (MIG, HAMi, KAI)
  • A before-and-after benchmark report
p95 latency targetPASS$ terraform apply

03 · Prove

AI Inference Proof of Concept

See it run in your environment, with your data.

Our engineers deploy an open-source LLM stack in your environment and measure it against your latency and cost targets.

  • In your cloud account or on-prem
  • Your choice of open model
  • Built as Terraform, so it is reproducible
  • Clear findings and a path to production
Apply for the free vLLM PoC
any cloudor on-premKUBERNETESTERRAFORMCI/CDAI

04 · Build the base

AI-ready Cloud Foundations

The foundations under your AI.

Kubernetes, infrastructure as code, CI/CD, secrets and security, built and maintained with your team.

  • Kubernetes on AWS, Azure, Google Cloud, OCI, CoreWeave, Nebius or on-prem
  • Terraform landing zones aligned with CIS
  • CI/CD with supply-chain scanning
  • Secrets, workload identity (OIDC), compliance as code

Where we deploy

The same consulting team and the same open-source stack, wherever your GPUs live. Pick your setup to see what we build there.

GPU clouds

GPU capacity on demand, without waiting on hyperscaler quotas.

We deploy your serving stack on a GPU cloud's managed Kubernetes, as Terraform you keep. A good fit when you need recent NVIDIA GPUs quickly, or extra capacity beyond your main cloud.

CoreWeaveNebiusCivo

Read: vLLM Production Stack on CoreWeave →

What we set up

  • Managed Kubernetes cluster and GPU node pools, all as Terraform
  • vLLM, llm-d or NVIDIA Dynamo, autoscaled on real queue depth
  • Model weights cached on shared storage for fast cold starts
  • Private networking and single sign-on back to your main environment

AI data center & on-prem

Your own GPUs and racks, kept busy.

For data that cannot leave your walls, or GPUs you already own. We build Kubernetes on your hardware, NVIDIA or AMD, and share GPUs between models so none of them sit idle.

NVIDIAAMDKubernetes

Read: GPU sharing: KAI vs HAMi vs MIG →

What we set up

  • Kubernetes on bare metal with the NVIDIA or AMD GPU operator
  • GPU sharing and scheduling: MIG, time-slicing, KAI or HAMi
  • Local model registry and image mirror, ready for restricted networks
  • Dashboards for GPU use, latency and cost per model

Hybrid cloud

Sensitive data stays on-prem, extra demand goes to the cloud.

One Terraform codebase deploys the same stack in both places, and one LLM gateway in front decides where each request runs.

one Terraform codebaseone LLM gateway

Read: LLM routing: open source vs commercial →

What we set up

  • The same serving stack on-prem and in the cloud, from one codebase
  • An LLM gateway that routes by data sensitivity, cost or load
  • Private interconnect and one identity (OIDC / SSO) across sites
  • Burst to cloud GPUs at peaks, then scale back down

Public cloud

Private inference inside the cloud account you already have.

On EKS, AKS, GKE or OKE, the stack runs in your own account and network, under your existing landing zone, billing and security controls.

AWSAzureGoogle CloudOCI

Read: vLLM on EKS: cut storage costs by 95% →

What we set up

  • GPU node pools on EKS, AKS, GKE or OKE, as Terraform
  • Model storage on S3, Blob or GCS, mounted for fast loading
  • Private endpoints only: prompts and outputs never leave your network
  • A GPU purchasing plan (on-demand, committed, spot) that keeps spend in check

Technologies

vLLM logo
vLLM
SGLang logo
SGLang
llm-d logo
llm-d
NVIDIA Dynamo logo
NVIDIA Dynamo
KServe logo
KServe
KubeRay logo
KubeRay
NVIDIA logo
NVIDIA
AMD logo
AMD
Ollama logo
Ollama
Kubernetes logo
Kubernetes
Terraform logo
Terraform
GitHub Actions logo
GitHub Actions
CoreWeave logo
CoreWeave
Nebius logo
Nebius
Oracle Cloud (OCI) logo
Oracle Cloud (OCI)
AWS logo
AWS
Azure logo
Azure
Google Cloud logo
Google Cloud