Managed vLLM hosting

Pilae installs, patches, backs up and monitors vLLM for you, on your own servers or in any of 10+ Pilae Cloud regions, six of them in Switzerland and the EU.

On Business, from CHF 1,590EUR 1,690USD 1,990 a month, because vLLM needs dedicated hardware.

Talk to us about vLLMSee what it runs on

Runs on
Your own hardware, or a Pilae Cloud region
Replaces
OpenAI API, Azure OpenAI
Licence
Apache-2.0
Your serversYour teamAssistantAI provider

vLLM among the AI and LLM apps

Models and the interfaces your team talks to, inside your perimeter. Prompts and documents stay on your side of the tunnel.

vLLM system requirements

Before anything is deployed, this is what has to exist. We size it with you in the first session, and we say so when your own hardware is already enough.

Where it runs
Your hardwarebare metal, VMware, Proxmox, Hyper-V or Kubernetes, in your datacentrePilae Cloud regionzur1 fra1 fal1 gra1 ams1 hel1 …machines dedicated to you, in the region your contract namesAPI key + /v1allowlistprivate meshaccess by identityencrypted tunnelencrypted, offsitevLLMpinned, patched, watched01GPUNVIDIA, recent computecapability03Model storageLocal NVMe, weightsmirroredYour teamthe people you allowYour identity providerKeycloak or Entra IDPilae Agentplans, asks, applies, recordsPilae consoleyou approve each changeYour backup targetstorage you own, at asecond sitea second datacentre,same jurisdictionYour hardwareyour servers, your networkPilae Cloud regionzur1 fra1 fal1 gra1 …private meshby identitytunnelencryptedvLLMpinned, patched, watched01GPUNVIDIA, recentcompute capability03Model storageLocal NVMe, weightsmirroredYour teamthe people you allowYour sign-inKeycloak, Entra IDPilae Agentplans, asks, applies, recordsPilae consoleyour approvalsYour backupsa second sitea second datacentre
On your hardware: Pilae installs vLLM and what it needs on the machines you choose, the Pilae Agent runs beside it, and encrypted backups go to storage you own at a second location you pick.In a Pilae Cloud region: the same setup on machines dedicated to you, in the region your contract names, with encrypted backups in a second datacentre inside the same jurisdiction.
All 5 requirements, and why each is there
  1. GPUNVIDIA, recent compute capabilityWith memory for the weights plus the KV cache at the context length and concurrency you need. AMD and Intel GPUs are supported through their own builds.
  2. Driver and runtimeNVIDIA driver + Container ToolkitThe official image carries its own CUDA libraries, and the host driver has to be new enough for them. On datacentre GPUs the compatibility libraries in the image can bridge an older driver; otherwise, moving to a newer image can mean upgrading the driver first.
  3. Model storageLocal NVMe, weights mirroredWeights are read from disk at every start, and upstream documents the default loader as most efficient on local storage. They are copied once from Hugging Face with the revision pinned.
  4. Shared memoryipc: hostvLLM uses PyTorch, which passes data between processes through shared memory, above all for tensor parallelism. Upstream says to give the container the host IPC namespace or a --shm-size, and we use the host namespace.
  5. AccessAPI key + /v1 allowlistThe API key covers /v1 and a few other prefixes. Routes such as /invocations, /tokenize and /pause answer without it, so the gate passes only the routes clients use.

Running vLLM in production: what it takes

  1. Size, then pin and deploy

    A vLLM image we have run, the model from the local mirror, and the tensor parallel size, context length and memory share in one config file in your repository. Usage statistics reporting to the project is switched off.

  2. Put the API behind the gate

    vLLM answers only on the private network, with a key per application. The gate in front passes the /v1 routes your clients need and nothing else.

  3. Watch the queue and the KV cache

    Probes check /health every 60 seconds. The metrics endpoint reports KV cache usage, waiting requests, time to first token and preemptions, and an engineer is alerted when the queue grows faster than it drains.

  4. Back up the config and the weights

    vLLM keeps no database. The config, compose files, mirrored weights and any fine-tuned adapters go offsite daily, encrypted, and the weights only change when the model does. On Enterprise, once a month we rebuild the server from them in a scratch environment and send it a test prompt.

  5. Upgrade against your config

    vLLM ships a minor release about every two weeks, and a deprecated flag can be gone two minor releases later. The Pilae Agent tests each upgrade against your config and model on a copy, waits for your approval, applies it in your window and keeps the previous image ready.

From our runbook

A vLLM server, configured from one file in your repository

# /etc/vllm/chat-model.yaml -- acme-inference-01, zur1, 2 x 80 GB GPUs
# Started as: vllm serve --config /etc/vllm/chat-model.yaml
# VLLM_API_KEY comes from the secret store; VLLM_NO_USAGE_STATS=1 is set.

model: /models/chat-model         # local mirror, never pulled at start
served-model-name: chat-model     # the name clients send as "model"
tensor-parallel-size: 2           # one model split across both GPUs
gpu-memory-utilization: 0.90      # vLLM's share; what weights leave is KV cache
max-model-len: 32768              # longest prompt plus answer accepted
max-num-seqs: 64                  # sequences scheduled together
enable-auto-tool-choice: true
tool-call-parser: hermes
host: 10.20.0.14                  # private network address only
port: 8000
uvicorn-log-level: warning
An example config for acme, read at start by vllm serve --config. The values that decide whether a model fits on the GPUs sit in one reviewed file, so a rebuilt server comes back with the same memory budget, context length and model name the applications call.

Migrating from OpenAI API to vLLM

Code written against the OpenAI API usually moves by changing three settings: the base URL, the key and the model name. vLLM serves the same Chat Completions, Completions, Embeddings and Responses endpoints, and tool calls work once the right parser is set for the model. What does not come across is the model. OpenAI's hosted models cannot be downloaded, so the work is choosing an open-weight model and testing it on your own prompts, which were tuned for another model and will need changes. Stored files, vector stores, fine-tuned models and Azure's content filters stay behind. Embeddings come from a separate embedding model, and vectors from two models cannot be mixed, so every vector index is rebuilt. Changing the code is the short part; that evaluation is the long one.

  • AlternativesA ChatGPT Enterprise alternative you ownOne of 5 apps for teams leaving ChatGPT Enterprise.
  • ServiceSaaS migrationA fixed-scope migration off Google Workspace, Microsoft 365, Zapier, Airtable and ChatGPT to open-source apps, with a parallel run and a cutover plan.

Moving from OpenAI API to vLLM, step by step

  1. Read the usage before choosing a model: Which endpoints, models and features your applications call, with token volumes and per-minute request peaks from the OpenAI or Azure usage records. Concurrency and context length decide the GPUs, not the number of users.
  2. Shortlist and test open-weight models: Two or three candidates whose licences fit, run against a set of your real prompts in a sandbox and scored beside the answers you get today. Prompts are adjusted here, not after the switch.
  3. Mirror the weights and size the server: The chosen weights are downloaded once, with the revision and checksums recorded. Gated models need access granted on Hugging Face to one of your users first, and some authors approve requests by hand. Tensor parallel size, context length and memory share are set from the measured load.
  4. Change the base URL, keep a fallback: Applications switch by configuration, one at a time, with the old key valid for one cycle. Anything that stores embeddings is re-indexed with the new embedding model before it switches.

What vLLM is, and who runs it

An OpenAI-compatible inference server for open-weight models, run on GPUs in Switzerland, the EU or your own datacentre, so prompts and answers never reach a model vendor.

Self-hosted vLLM for serving open-weight models

vLLM is an inference and serving engine for large language models. It started at UC Berkeley’s Sky Computing Lab and is now a hosted project of the PyTorch Foundation. PagedAttention keeps the KV cache in small blocks instead of one reservation per request, and continuous batching adds requests to the running batch as others finish. Tensor parallelism splits one model across the GPUs in a machine. Applications reach it through an OpenAI-compatible API.

It is built for many people or applications calling the same model at once. For one GPU machine serving a team, or several smaller models on modest hardware, Ollama is simpler to run. In a private AI deployment, vLLM usually sits behind a chat front end such as Open WebUI or LibreChat. Your own applications can also call it directly in place of the OpenAI API, and Langfuse beside it records each prompt and answer, so a change of model is judged on your own traffic.

Read the full briefFold the brief

vLLM in production: model weights, API access and GPU memory

vLLM runs on GPU machines dedicated to it, yours or ours in a Pilae Cloud region: Zurich, Frankfurt, Falkenstein, Gravelines, Amsterdam or Helsinki inside Switzerland and the EU, or one of six further afield. The weights are mirrored locally with the revision recorded, so a restart never waits on a download. The server answers only on your private network. Its API key covers /v1 and a few other prefixes but leaves many routes open, including /invocations, which runs inference, so the gate in front passes only the routes your clients call.

The failures that matter are about memory. vLLM refuses to start when the context length asked for does not fit in the KV cache, so the context length is set from the sizing before the first start. Under a burst it queues or preempts requests instead of failing them, so an overload shows up as slow answers rather than errors. Probes check the server every 60 seconds, and a queue that grows faster than it drains alerts an engineer. The config, compose files, weights and adapters are backed up daily, encrypted, to an offsite location in the country your contract names, and on Enterprise a scratch server is rebuilt from them every month, with the test recorded in the console. With a release every couple of weeks, the Pilae Agent runs each one against your config and model on a copy before you approve it for your window, and records the change in the console.

vLLM licence and model licences

vLLM is Apache-2.0. There is no paid edition and no enterprise directory, so nothing is kept back from the open-source code. Two other licences sit next to it. The model weights carry their own: some are Apache-2.0 or MIT, while others, such as Meta’s Llama models, come with their own licence and acceptable use policy. We check the one you choose before it is deployed. The CUDA libraries in the image come from NVIDIA’s own base image and are under NVIDIA’s licence terms. Our operation follows the published plans. Talk to us about the model you want to serve.

What Pilae is responsible for

The same commitments on every app we run, vLLM included, and what stays yours. How the platform does it.

Ours

  • A pinned version: A version we have run, not whatever latest resolves to that day.
  • A runbook: What it depends on, how it fails, what to do about it. In your repository.
  • Daily backups: Encrypted and offsite, and on Enterprise restored every month. A backup nobody has restored is a file.
  • A patch window: Security updates in a window you agreed, with a rollback ready.
  • Someone watching: Every endpoint probed on the minute. An alert reaches a person, not a dashboard nobody opens.

Yours

  • The credentials: You hold them. Ours are separate, named, logged and revocable with one command.
  • The approval: We ask before anything changes outside an agreed window.
  • The exit: The machine, the data, the compose files and the runbook are already yours. Nothing stops when our access does.

Who runs it

vLLM needs dedicated hardware, so it runs on Business: CHF 1,590EUR 1,690USD 1,990 a month, one of up to three apps, on your own servers or in Pilae Cloud.

Compare the plans

vLLM: common questions

Is vLLM open source?

Yes. vLLM is Apache-2.0 and hosted by the PyTorch Foundation, with no paid edition and no features held back for one. The models you serve with it are a separate question: each set of weights carries its own licence, and some come with use restrictions.

Where do prompts and answers go?

To the GPU machine and no further: your own hardware, or a dedicated server in the Pilae region you picked, in ISO 27001-certified datacentres. vLLM keeps no conversation store by default, and we leave its Responses store and request logging off. Left at its defaults, it also sends anonymous usage statistics about hardware and configuration to the project; we switch that off.

Can our applications keep using the OpenAI SDK?

Yes, for Chat Completions, Completions, Embeddings and Responses. The OpenAI client libraries call vLLM once the base URL, key and model name are changed. Tool calling needs a parser set for the model family, and that sits in the server's config, not in your code. OpenAI's stored files, vector stores and fine-tuning have no equivalent in vLLM.

Should we run vLLM or Ollama?

vLLM for many concurrent requests to one model, Ollama for one GPU machine serving a team or several smaller models. Ollama is simpler to run: it loads models on demand and runs quantised models on modest hardware. vLLM reserves most of the GPU memory at start and holds one base model per server, so serving several models means several servers.

What hardware does vLLM need?

GPUs with enough memory for the weights plus the KV cache at your context length and concurrency. NVIDIA with a recent compute capability, as upstream lists it, is the main path, and AMD and Intel GPUs are supported through their own builds. We size it from your traffic before anything is installed, and say so when the model you want does not fit.

Ask the engineers who would run your vLLM

Thirty minutes on the deployment you already have, or the one you are about to start.

Talk to us about vLLM

Plan your vLLM deployment

Costs and savings

Product comparisons

Business workflows