Community Apps

vLLM

By OrcVole

High-throughput OpenAI-compatible LLM inference server

0.28.0 10240MB Updated 2 Sep 2026
Package Website Documentation
ai llm inference openai
Install this app
In your Cloudron dashboard, open the App Store, choose "Community apps" and paste this URL:
https://raw.githubusercontent.com/OrcVole/vllm-cloudron/main/CloudronVersions.json

Description

`0.28.0

vLLM is a high-throughput inference server for large language models with the most widely supported OpenAI-compatible API of the self-hosted options. This package runs it as a Cloudron app: one install serves one model at https://<location>/v1, protected by a generated API key, with weights cached locally and every OpenAI-style client able to connect unchanged.

This is CPU inference (amd64). It will not match GPU speed and it is not meant to: it gives integrations, automations and privacy-sensitive workloads a fully self-hosted OpenAI endpoint on the server you already run. Pair it with a chat frontend, an AI gateway, embeddings and a vector store to complete a private AI stack on Cloudron.

Unofficial community package. Not affiliated with the vLLM project or Cloudron.

Screenshot

Recent Changes

  • Update vLLM 0.27.1 -> 0.28.0

  • Security: fixes prevent denial of service via forged audio sample rates and oversized image inputs, unauthenticated access to API endpoints, arbitrary code execution from untrusted model repositories, and resource exhaustion from unbounded generation requests

  • Breaking: bitsandbytes quantization moved to an out-of-tree plugin; Transformers dependency bumped to 5.15.0; removed flags calculate_kv_scales and override_attention_dtype; reasoning_content no longer included in output; MoE legacy code paths removed; cache_salt now required to be non-empty

  • Behaviour changes: max_num_batched_tokens default raised to 16384; prefix caching now enabled by default for Mamba models; Blackwell CUDA graph capture default raised to 1024; KV offload tiering metrics renamed from block to chunk

  • Packaging: pin moved; /dev/shm detection in start.sh unchanged and still required