https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/CloudronVersions.json
Description
vLLM is a high-throughput inference server for large language models with the most widely
supported OpenAI-compatible API of the self-hosted options. This package runs it as a
Cloudron app: one install serves one model at https://<location>/v1, protected by a
generated API key, with weights cached locally and every OpenAI-style client able to connect
unchanged.
This is CPU inference (amd64). It will not match GPU speed and it is not meant to: it gives integrations, automations and privacy-sensitive workloads a fully self-hosted OpenAI endpoint on the server you already run. Pair it with a chat frontend, an AI gateway, embeddings and a vector store to complete a private AI stack on Cloudron.
Unofficial community package. Not affiliated with the vLLM project or Cloudron.
Recent Changes
Initial release: vLLM 0.26.0 OpenAI-compatible inference server, CPU backend (amd64, AVX2 minimum)
One model per install, default Qwen/Qwen3-0.6B, configurable via LLM_MODEL
Generated API key protects /v1; health, readiness and landing page open
Model cache on a persistent path outside backups; restores stay small and fast
Full gate ladder passed: install, auth, streamed flows through the platform proxy, backup/restore, memory sizing (10 GiB limit from measured 7.4 GiB steady footprint)