Run more models than your GPU can hold.
CapyCTL deploys, parks, wakes and switches models on vLLM, SGLang and TensorFold, on GPU machines you own, behind one OpenAI-compatible endpoint.
$ capyctl list deployments NAME STATE READY REVISION HOSTS coding-small ready 1/1 3 gpu-box chat-large ready 1/1 1 workstation vision-small parked 0/1 2 gpu-box
Example output. A parked model releases its GPU memory and wakes on the next request for it.
- More models than memoryMost GPU machines hold one or two models at a time. CapyCTL parks the idle ones and wakes the one a request asks for, making room by parking or stopping an idle model.
- One endpointOpenAI-compatible chat completions with streaming and tool calls, behind an API key. Your apps keep one base URL and pick a model by name.
- Your engines, your weightsRegister the vLLM, SGLang or TensorFold installation you already have. Models come from a directory or from the Hub, pinned to a commit when you deploy.
Quick start
On Linux with an NVIDIA GPU and a vLLM, SGLang or TensorFold environment:
curl -fsSL https://edurdias.github.io/capyctl/install.sh | sh
capyctl engine add ~/venvs/vllm # a vLLM, SGLang or TensorFold environment
capyctl start standalone
In another shell, deploy a model from the Hub. Three fields are enough:
# qwen3-4b.yaml
name: qwen3-4b
engine: vllm
model: {hf: Qwen/Qwen3-4B-Instruct-2507}
capyctl deploy model --file qwen3-4b.yaml --activate --wait
Any OpenAI client can now send qwen3-4b as the model name (Make a request). To free its GPU memory, run capyctl park deployment qwen3-4b; the next request wakes it. One server can also drive several GPU machines (Run on several machines).
Measured recipes
Deployment files run through CapyCTL, each sized so the model fits in 32 GiB once ready. Measured on a GB10 on 2026-10-07.
| Model and checkpoint | Engine | Decode, one stream | More streams |
|---|---|---|---|
| Qwen3.6-35B-A3B, MLX 4-bit, MTP drafts | TensorFold 0.6.5 | 164 tokens/s | 368 tokens/s at 8 streams |
| Qwen3.6-35B-A3B, NVFP4 | vLLM 0.30.0 | 76.7 tokens/s | 180 tokens/s at 4 streams |
| Qwen3.6-35B-A3B, NVFP4 | SGLang 0.5.21 | 83.5 tokens/s | 186 tokens/s at 4 streams |
| Nemotron 3.5 Lightning 30B-A3B, MLX 4-bit, MTP drafts | TensorFold 0.6.5 | 137 tokens/s | one request at a time |
| Nemotron 3.5 Lightning 30B-A3B, NVFP4 | vLLM 0.30.0 | 69.9 tokens/s | 164 tokens/s at 4 streams |
| Nemotron 3.5 Lightning 30B-A3B, NVFP4 | SGLang 0.5.21 | 69.9 tokens/s | 163 tokens/s at 4 streams |
The TensorFold and NVFP4 rows serve different checkpoints, so they are not a one-to-one engine comparison. Each recipe lists the model revision, the commands, ready times, memory and a capyctl-bench report; the recipes repository has gpt-oss-20b, Gemma 4, Qwen3.8-27B and more.
In this organization
Checkpoints converted so a recipe can serve them. Each card says what changed from the original and how it was made.
- FrogNano-4B-2609-MLX-4bit: MLX 4-bit, text only, for TensorFold on NVIDIA GPUs.
- FrogNano-4B-2609-NVFP4: NVFP4 with FP8 attention, for vLLM, SGLang and TensorFold.
Status
0.1.2 is early software: one GPU per model, CLI only, and breaking changes are possible between minor versions while the version starts with 0. Models that span several GPUs or machines are coming; they are not in a release yet. Questions and bug reports go to GitHub. Licensed under Apache-2.0.