first commit
This commit is contained in:
8
apps/vllm-server/README.md
Normal file
8
apps/vllm-server/README.md
Normal file
@ -0,0 +1,8 @@
|
||||
# vllm-server
|
||||
|
||||
OpenAI-compatible model serving with vLLM.
|
||||
|
||||
The base is CPU-safe YAML. Add `components/gpu-nvidia` in environments that
|
||||
provide NVIDIA GPUs, and let the instance overlay patch model name, resources,
|
||||
and cache size.
|
||||
|
||||
Reference in New Issue
Block a user