first commit
This commit is contained in:
7
.gitignore
vendored
Normal file
7
.gitignore
vendored
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
.ocdp-builds/
|
||||||
|
build/
|
||||||
|
dist/
|
||||||
|
tmp/
|
||||||
|
*.rendered.yaml
|
||||||
|
*.secret.yaml
|
||||||
|
|
||||||
44
AGENTS.md
Normal file
44
AGENTS.md
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
# AGENTS.md
|
||||||
|
|
||||||
|
This file is the execution guide for `ocdp-workload-manifests`.
|
||||||
|
|
||||||
|
## Working Boundary
|
||||||
|
|
||||||
|
- Treat this project as a plain app manifests repository.
|
||||||
|
- Do not store OCDP WorkloadTemplate records here; they belong in `ocdp-server`
|
||||||
|
PostgreSQL.
|
||||||
|
- Do not store user WorkloadClaim instances here; they belong in target-cluster
|
||||||
|
CRs created through `ocdp-server`.
|
||||||
|
- Keep reusable Kubernetes resources under `apps/*/base`.
|
||||||
|
- Keep optional Kustomize components under `apps/*/components`.
|
||||||
|
- Keep per-deployment runtime values out of this repository. They belong in
|
||||||
|
temporary source files or runtime specs generated by `ocdp-server`.
|
||||||
|
- Use `tests/kustomize/*` only for generic validation overlays, never for real
|
||||||
|
user instances.
|
||||||
|
- Do not commit real passwords, tokens, customer hostnames, private NFS paths, or
|
||||||
|
private registry credentials.
|
||||||
|
|
||||||
|
## Runtime Rules
|
||||||
|
|
||||||
|
- Do not add a global catalog index unless the server explicitly needs one later.
|
||||||
|
- `ocdp-server` WorkloadTemplate records should reference apps with
|
||||||
|
`repositoryUrl`, `ref`, and `path`.
|
||||||
|
- Runtime source generation may create Secret, ConfigMap, and patch files, then
|
||||||
|
run Kustomize.
|
||||||
|
- Bases should stay template-free YAML.
|
||||||
|
- App bases should keep Services as `ClusterIP`; expose apps from
|
||||||
|
WorkloadTemplate values such as `exposure=nodeport` or `exposure=loadbalancer`.
|
||||||
|
Components such as `service-nodeport`, `service-loadbalancer`, or `ingress`
|
||||||
|
are implementation building blocks, not the user-facing contract.
|
||||||
|
- Environment overlays are platform/admin choices for cluster- or site-level
|
||||||
|
differences such as StorageClass, IngressClass, GPU runtime, registry prefix,
|
||||||
|
pull secrets, node selectors, tolerations, and site labels.
|
||||||
|
- NodePort numbers belong in per-claim values handled by `ocdp-server`, not in
|
||||||
|
reusable app bases or components.
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
- Run `make validate` after changing workload bases, components, or test
|
||||||
|
overlays.
|
||||||
|
- If a change intentionally affects generated resource names, check references
|
||||||
|
such as Secret refs, PVC claim names, Service backends, and ConfigMap mounts.
|
||||||
18
Makefile
Normal file
18
Makefile
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
.PHONY: validate list
|
||||||
|
|
||||||
|
KUSTOMIZE ?= kubectl kustomize
|
||||||
|
|
||||||
|
TEST_OVERLAYS := \
|
||||||
|
tests/kustomize/code-server-nodeport \
|
||||||
|
tests/kustomize/vllm-server-loadbalancer \
|
||||||
|
tests/kustomize/litellm-ingress
|
||||||
|
|
||||||
|
list:
|
||||||
|
@printf '%s\n' $(TEST_OVERLAYS)
|
||||||
|
|
||||||
|
validate:
|
||||||
|
@set -eu; \
|
||||||
|
for dir in $(TEST_OVERLAYS); do \
|
||||||
|
printf '==> %s\n' "$$dir"; \
|
||||||
|
$(KUSTOMIZE) "$$dir" >/dev/null; \
|
||||||
|
done
|
||||||
125
README.md
Normal file
125
README.md
Normal file
@ -0,0 +1,125 @@
|
|||||||
|
# ocdp-workload-manifests
|
||||||
|
|
||||||
|
Standalone Kubernetes manifests for OCDP workloads.
|
||||||
|
|
||||||
|
This repository is intentionally just a Git repository of app manifests. There is
|
||||||
|
no global catalog index and no dependency on the Gitea API. `ocdp-server` can
|
||||||
|
read a workload by building a Kustomize target directly from Git.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
OCDP keeps the responsibilities split:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Git repo
|
||||||
|
apps/<app>/base
|
||||||
|
apps/<app>/components
|
||||||
|
|
||||||
|
ocdp-server PostgreSQL
|
||||||
|
WorkloadTemplate metadata
|
||||||
|
WorkloadTemplate source.repositoryUrl/ref/path
|
||||||
|
user-facing values schema / parameters
|
||||||
|
environment overlay/profile metadata
|
||||||
|
access bindings
|
||||||
|
no WorkloadClaim instance storage
|
||||||
|
|
||||||
|
target cluster
|
||||||
|
WorkloadClaim CR as the canonical user claim
|
||||||
|
Workload CR
|
||||||
|
runtime Kubernetes resources
|
||||||
|
```
|
||||||
|
|
||||||
|
This repository stores only the Git base and reusable components. It does not
|
||||||
|
store `WorkloadTemplate` records and does not store user `WorkloadClaim`
|
||||||
|
instances.
|
||||||
|
|
||||||
|
An admin creates or updates a `WorkloadTemplate` in `ocdp-server`. That template
|
||||||
|
can point at one of these Git paths:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
templateType: kustomize
|
||||||
|
source:
|
||||||
|
type: gitKustomize
|
||||||
|
repositoryUrl: https://gitea.example.com/ocdp/ocdp-workload-manifests.git
|
||||||
|
ref: code-server-v0.1.0
|
||||||
|
path: apps/code-server/base
|
||||||
|
```
|
||||||
|
|
||||||
|
After the template is stored in PostgreSQL and assigned to users or groups, users
|
||||||
|
call `ocdp-server` to create a claim by submitting `templateId`, `workspaceId`,
|
||||||
|
and values. `ocdp-server` resolves the template from PostgreSQL, reads the Git
|
||||||
|
base, generates any temporary source files or runtime specs outside this repo,
|
||||||
|
then writes a `WorkloadClaim` CR into the target Kubernetes cluster. The final
|
||||||
|
user-created claim lives in Kubernetes, not in PostgreSQL.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```text
|
||||||
|
apps/
|
||||||
|
code-server/
|
||||||
|
base/
|
||||||
|
components/
|
||||||
|
vllm-server/
|
||||||
|
base/
|
||||||
|
components/
|
||||||
|
litellm/
|
||||||
|
base/
|
||||||
|
components/
|
||||||
|
tests/
|
||||||
|
kustomize/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Server Usage
|
||||||
|
|
||||||
|
`ocdp-server` should store a normal `gitKustomize` reference:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
source:
|
||||||
|
type: gitKustomize
|
||||||
|
repositoryUrl: https://gitea.example.com/ocdp/ocdp-workload-manifests.git
|
||||||
|
ref: code-server-v0.1.0
|
||||||
|
path: apps/code-server/base
|
||||||
|
```
|
||||||
|
|
||||||
|
For one deployment, `ocdp-server` should generate temporary source files outside
|
||||||
|
this repository. Those files can point at the Git base and add generated
|
||||||
|
Secrets, ConfigMaps, components, and patches.
|
||||||
|
|
||||||
|
## Exposure
|
||||||
|
|
||||||
|
Base services are `ClusterIP`. User-facing exposure choices such as
|
||||||
|
`clusterip`, `nodeport`, and `loadbalancer` belong in the WorkloadTemplate
|
||||||
|
values schema. `ocdp-server` can translate that value into runtime Service
|
||||||
|
configuration or a generated patch.
|
||||||
|
|
||||||
|
Reusable components are still useful implementation building blocks:
|
||||||
|
|
||||||
|
- `components/ingress`: keep the app Service internal and route through an
|
||||||
|
ingress controller.
|
||||||
|
- `components/service-loadbalancer`: change the app Service to `LoadBalancer`.
|
||||||
|
- `components/service-nodeport`: change the app Service to `NodePort`.
|
||||||
|
|
||||||
|
When `nodeport` is selected by the user, `ocdp-server` may generate an instance
|
||||||
|
patch or runtime Service field for an explicit port:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- op: add
|
||||||
|
path: /spec/ports/0/nodePort
|
||||||
|
value: 30080
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not hard-code shared NodePort values in app bases or reusable components.
|
||||||
|
|
||||||
|
Environment overlays are different from user choices. They are selected by the
|
||||||
|
platform from cluster, workspace, or customer profile information and can carry
|
||||||
|
things like StorageClass, IngressClass, GPU runtime class, registry prefix,
|
||||||
|
pull-secret wiring, node selectors, tolerations, and site-specific labels.
|
||||||
|
|
||||||
|
## Validate
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make validate
|
||||||
|
```
|
||||||
|
|
||||||
|
`make validate` runs `kubectl kustomize` against non-user test overlays under
|
||||||
|
`tests/kustomize`.
|
||||||
7
apps/code-server/README.md
Normal file
7
apps/code-server/README.md
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
# code-server
|
||||||
|
|
||||||
|
The base deploys code-server with a PVC, Service, and password Secret reference.
|
||||||
|
|
||||||
|
The Secret is generated by the instance overlay. Do not commit real passwords or
|
||||||
|
tokens to this catalog.
|
||||||
|
|
||||||
65
apps/code-server/base/deployment.yaml
Normal file
65
apps/code-server/base/deployment.yaml
Normal file
@ -0,0 +1,65 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: code-server
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: code-server
|
||||||
|
app.kubernetes.io/component: ide
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
annotations: {}
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/name: code-server
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: code-server
|
||||||
|
app.kubernetes.io/component: ide
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
securityContext:
|
||||||
|
fsGroup: 1000
|
||||||
|
containers:
|
||||||
|
- name: code-server
|
||||||
|
image: codercom/code-server:latest
|
||||||
|
imagePullPolicy: IfNotPresent
|
||||||
|
args:
|
||||||
|
- --bind-addr
|
||||||
|
- 0.0.0.0:8080
|
||||||
|
- --auth
|
||||||
|
- password
|
||||||
|
- /home/coder/project
|
||||||
|
env:
|
||||||
|
- name: PASSWORD
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: code-server-auth
|
||||||
|
key: password
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
containerPort: 8080
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /
|
||||||
|
port: http
|
||||||
|
livenessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /
|
||||||
|
port: http
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "500m"
|
||||||
|
memory: 1Gi
|
||||||
|
limits:
|
||||||
|
cpu: "2"
|
||||||
|
memory: 4Gi
|
||||||
|
volumeMounts:
|
||||||
|
- name: workspace
|
||||||
|
mountPath: /home/coder/project
|
||||||
|
volumes:
|
||||||
|
- name: workspace
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: code-server-data
|
||||||
|
|
||||||
7
apps/code-server/base/kustomization.yaml
Normal file
7
apps/code-server/base/kustomization.yaml
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
resources:
|
||||||
|
- pvc.yaml
|
||||||
|
- deployment.yaml
|
||||||
|
- service.yaml
|
||||||
|
|
||||||
15
apps/code-server/base/pvc.yaml
Normal file
15
apps/code-server/base/pvc.yaml
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
metadata:
|
||||||
|
name: code-server-data
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: code-server
|
||||||
|
app.kubernetes.io/component: workspace-storage
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
accessModes:
|
||||||
|
- ReadWriteOnce
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: 20Gi
|
||||||
|
|
||||||
16
apps/code-server/base/service.yaml
Normal file
16
apps/code-server/base/service.yaml
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: code-server
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: code-server
|
||||||
|
app.kubernetes.io/component: ide
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app.kubernetes.io/name: code-server
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
port: 80
|
||||||
|
targetPort: http
|
||||||
|
|
||||||
21
apps/code-server/components/ingress/ingress.yaml
Normal file
21
apps/code-server/components/ingress/ingress.yaml
Normal file
@ -0,0 +1,21 @@
|
|||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: Ingress
|
||||||
|
metadata:
|
||||||
|
name: code-server
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: code-server
|
||||||
|
app.kubernetes.io/component: ingress
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
rules:
|
||||||
|
- host: code-server.example.local
|
||||||
|
http:
|
||||||
|
paths:
|
||||||
|
- path: /
|
||||||
|
pathType: Prefix
|
||||||
|
backend:
|
||||||
|
service:
|
||||||
|
name: code-server
|
||||||
|
port:
|
||||||
|
name: http
|
||||||
|
|
||||||
5
apps/code-server/components/ingress/kustomization.yaml
Normal file
5
apps/code-server/components/ingress/kustomization.yaml
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
resources:
|
||||||
|
- ingress.yaml
|
||||||
|
|
||||||
8
apps/code-server/components/nfs/kustomization.yaml
Normal file
8
apps/code-server/components/nfs/kustomization.yaml
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
patches:
|
||||||
|
- path: pvc-rwx-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
name: code-server-data
|
||||||
|
|
||||||
4
apps/code-server/components/nfs/pvc-rwx-patch.yaml
Normal file
4
apps/code-server/components/nfs/pvc-rwx-patch.yaml
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
- op: replace
|
||||||
|
path: /spec/accessModes/0
|
||||||
|
value: ReadWriteMany
|
||||||
|
|
||||||
@ -0,0 +1,8 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
patches:
|
||||||
|
- path: service-loadbalancer-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: Service
|
||||||
|
name: code-server
|
||||||
|
|
||||||
@ -0,0 +1,4 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/type
|
||||||
|
value: LoadBalancer
|
||||||
|
|
||||||
@ -0,0 +1,8 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
patches:
|
||||||
|
- path: service-nodeport-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: Service
|
||||||
|
name: code-server
|
||||||
|
|
||||||
@ -0,0 +1,4 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/type
|
||||||
|
value: NodePort
|
||||||
|
|
||||||
8
apps/litellm/README.md
Normal file
8
apps/litellm/README.md
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
# litellm
|
||||||
|
|
||||||
|
LiteLLM gateway with ConfigMap-backed model routing and Secret-backed runtime
|
||||||
|
keys.
|
||||||
|
|
||||||
|
The instance overlay should generate `litellm-secrets` and patch
|
||||||
|
`config.yaml` for the selected upstream model.
|
||||||
|
|
||||||
20
apps/litellm/base/configmap.yaml
Normal file
20
apps/litellm/base/configmap.yaml
Normal file
@ -0,0 +1,20 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: litellm-config
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: litellm
|
||||||
|
app.kubernetes.io/component: config
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
data:
|
||||||
|
config.yaml: |
|
||||||
|
model_list:
|
||||||
|
- model_name: default
|
||||||
|
litellm_params:
|
||||||
|
model: openai/default
|
||||||
|
api_base: os.environ/LITELLM_UPSTREAM_API_BASE
|
||||||
|
api_key: os.environ/LITELLM_UPSTREAM_API_KEY
|
||||||
|
litellm_settings:
|
||||||
|
drop_params: true
|
||||||
|
request_timeout: 600
|
||||||
|
|
||||||
80
apps/litellm/base/deployment.yaml
Normal file
80
apps/litellm/base/deployment.yaml
Normal file
@ -0,0 +1,80 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: litellm
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: litellm
|
||||||
|
app.kubernetes.io/component: gateway
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
annotations: {}
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/name: litellm
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: litellm
|
||||||
|
app.kubernetes.io/component: gateway
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: litellm
|
||||||
|
image: docker.litellm.ai/berriai/litellm:main-stable
|
||||||
|
imagePullPolicy: IfNotPresent
|
||||||
|
args:
|
||||||
|
- --config
|
||||||
|
- /app/config.yaml
|
||||||
|
- --port
|
||||||
|
- "4000"
|
||||||
|
- --num_workers
|
||||||
|
- "2"
|
||||||
|
env:
|
||||||
|
- name: LITELLM_MASTER_KEY
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: litellm-secrets
|
||||||
|
key: masterKey
|
||||||
|
- name: LITELLM_SALT_KEY
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: litellm-secrets
|
||||||
|
key: saltKey
|
||||||
|
- name: LITELLM_UPSTREAM_API_KEY
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: litellm-secrets
|
||||||
|
key: upstreamApiKey
|
||||||
|
- name: LITELLM_UPSTREAM_API_BASE
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: litellm-secrets
|
||||||
|
key: upstreamApiBase
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
containerPort: 4000
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health/readiness
|
||||||
|
port: http
|
||||||
|
livenessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health/liveliness
|
||||||
|
port: http
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "250m"
|
||||||
|
memory: 512Mi
|
||||||
|
limits:
|
||||||
|
cpu: "1"
|
||||||
|
memory: 1Gi
|
||||||
|
volumeMounts:
|
||||||
|
- name: config
|
||||||
|
mountPath: /app/config.yaml
|
||||||
|
subPath: config.yaml
|
||||||
|
volumes:
|
||||||
|
- name: config
|
||||||
|
configMap:
|
||||||
|
name: litellm-config
|
||||||
|
|
||||||
7
apps/litellm/base/kustomization.yaml
Normal file
7
apps/litellm/base/kustomization.yaml
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
resources:
|
||||||
|
- configmap.yaml
|
||||||
|
- deployment.yaml
|
||||||
|
- service.yaml
|
||||||
|
|
||||||
16
apps/litellm/base/service.yaml
Normal file
16
apps/litellm/base/service.yaml
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: litellm
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: litellm
|
||||||
|
app.kubernetes.io/component: gateway
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app.kubernetes.io/name: litellm
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
port: 4000
|
||||||
|
targetPort: http
|
||||||
|
|
||||||
21
apps/litellm/components/ingress/ingress.yaml
Normal file
21
apps/litellm/components/ingress/ingress.yaml
Normal file
@ -0,0 +1,21 @@
|
|||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: Ingress
|
||||||
|
metadata:
|
||||||
|
name: litellm
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: litellm
|
||||||
|
app.kubernetes.io/component: ingress
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
rules:
|
||||||
|
- host: litellm.example.local
|
||||||
|
http:
|
||||||
|
paths:
|
||||||
|
- path: /
|
||||||
|
pathType: Prefix
|
||||||
|
backend:
|
||||||
|
service:
|
||||||
|
name: litellm
|
||||||
|
port:
|
||||||
|
name: http
|
||||||
|
|
||||||
5
apps/litellm/components/ingress/kustomization.yaml
Normal file
5
apps/litellm/components/ingress/kustomization.yaml
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
resources:
|
||||||
|
- ingress.yaml
|
||||||
|
|
||||||
@ -0,0 +1,8 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
patches:
|
||||||
|
- path: service-loadbalancer-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: Service
|
||||||
|
name: litellm
|
||||||
|
|
||||||
@ -0,0 +1,4 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/type
|
||||||
|
value: LoadBalancer
|
||||||
|
|
||||||
@ -0,0 +1,8 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
patches:
|
||||||
|
- path: service-nodeport-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: Service
|
||||||
|
name: litellm
|
||||||
|
|
||||||
@ -0,0 +1,4 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/type
|
||||||
|
value: NodePort
|
||||||
|
|
||||||
8
apps/vllm-server/README.md
Normal file
8
apps/vllm-server/README.md
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
# vllm-server
|
||||||
|
|
||||||
|
OpenAI-compatible model serving with vLLM.
|
||||||
|
|
||||||
|
The base is CPU-safe YAML. Add `components/gpu-nvidia` in environments that
|
||||||
|
provide NVIDIA GPUs, and let the instance overlay patch model name, resources,
|
||||||
|
and cache size.
|
||||||
|
|
||||||
58
apps/vllm-server/base/deployment.yaml
Normal file
58
apps/vllm-server/base/deployment.yaml
Normal file
@ -0,0 +1,58 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: vllm-server
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: vllm-server
|
||||||
|
app.kubernetes.io/component: model-server
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
annotations: {}
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/name: vllm-server
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: vllm-server
|
||||||
|
app.kubernetes.io/component: model-server
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: vllm
|
||||||
|
image: vllm/vllm-openai:latest
|
||||||
|
imagePullPolicy: IfNotPresent
|
||||||
|
args:
|
||||||
|
- --host
|
||||||
|
- 0.0.0.0
|
||||||
|
- --port
|
||||||
|
- "8000"
|
||||||
|
- --model
|
||||||
|
- Qwen/Qwen2.5-7B-Instruct
|
||||||
|
- --served-model-name
|
||||||
|
- default
|
||||||
|
env:
|
||||||
|
- name: HF_TOKEN
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: vllm-secrets
|
||||||
|
key: hfToken
|
||||||
|
optional: true
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
containerPort: 8000
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: http
|
||||||
|
initialDelaySeconds: 20
|
||||||
|
periodSeconds: 10
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "2"
|
||||||
|
memory: 12Gi
|
||||||
|
limits:
|
||||||
|
cpu: "4"
|
||||||
|
memory: 24Gi
|
||||||
|
|
||||||
6
apps/vllm-server/base/kustomization.yaml
Normal file
6
apps/vllm-server/base/kustomization.yaml
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
resources:
|
||||||
|
- deployment.yaml
|
||||||
|
- service.yaml
|
||||||
|
|
||||||
16
apps/vllm-server/base/service.yaml
Normal file
16
apps/vllm-server/base/service.yaml
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: vllm-server
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: vllm-server
|
||||||
|
app.kubernetes.io/component: model-server
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app.kubernetes.io/name: vllm-server
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
port: 8000
|
||||||
|
targetPort: http
|
||||||
|
|
||||||
7
apps/vllm-server/components/gpu-nvidia/gpu-patch.yaml
Normal file
7
apps/vllm-server/components/gpu-nvidia/gpu-patch.yaml
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/template/spec/runtimeClassName
|
||||||
|
value: nvidia
|
||||||
|
- op: add
|
||||||
|
path: /spec/template/spec/containers/0/resources/limits/nvidia.com~1gpu
|
||||||
|
value: 1
|
||||||
|
|
||||||
10
apps/vllm-server/components/gpu-nvidia/kustomization.yaml
Normal file
10
apps/vllm-server/components/gpu-nvidia/kustomization.yaml
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
patches:
|
||||||
|
- path: gpu-patch.yaml
|
||||||
|
target:
|
||||||
|
group: apps
|
||||||
|
version: v1
|
||||||
|
kind: Deployment
|
||||||
|
name: vllm-server
|
||||||
|
|
||||||
21
apps/vllm-server/components/ingress/ingress.yaml
Normal file
21
apps/vllm-server/components/ingress/ingress.yaml
Normal file
@ -0,0 +1,21 @@
|
|||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: Ingress
|
||||||
|
metadata:
|
||||||
|
name: vllm-server
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: vllm-server
|
||||||
|
app.kubernetes.io/component: ingress
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
rules:
|
||||||
|
- host: vllm.example.local
|
||||||
|
http:
|
||||||
|
paths:
|
||||||
|
- path: /
|
||||||
|
pathType: Prefix
|
||||||
|
backend:
|
||||||
|
service:
|
||||||
|
name: vllm-server
|
||||||
|
port:
|
||||||
|
name: http
|
||||||
|
|
||||||
5
apps/vllm-server/components/ingress/kustomization.yaml
Normal file
5
apps/vllm-server/components/ingress/kustomization.yaml
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
resources:
|
||||||
|
- ingress.yaml
|
||||||
|
|
||||||
@ -0,0 +1,17 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/template/spec/containers/0/env/-
|
||||||
|
value:
|
||||||
|
name: HF_HOME
|
||||||
|
value: /cache/huggingface
|
||||||
|
- op: add
|
||||||
|
path: /spec/template/spec/containers/0/volumeMounts
|
||||||
|
value:
|
||||||
|
- name: model-cache
|
||||||
|
mountPath: /cache
|
||||||
|
- op: add
|
||||||
|
path: /spec/template/spec/volumes
|
||||||
|
value:
|
||||||
|
- name: model-cache
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: vllm-cache
|
||||||
|
|
||||||
12
apps/vllm-server/components/pvc-cache/kustomization.yaml
Normal file
12
apps/vllm-server/components/pvc-cache/kustomization.yaml
Normal file
@ -0,0 +1,12 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
resources:
|
||||||
|
- pvc.yaml
|
||||||
|
patches:
|
||||||
|
- path: deployment-cache-patch.yaml
|
||||||
|
target:
|
||||||
|
group: apps
|
||||||
|
version: v1
|
||||||
|
kind: Deployment
|
||||||
|
name: vllm-server
|
||||||
|
|
||||||
15
apps/vllm-server/components/pvc-cache/pvc.yaml
Normal file
15
apps/vllm-server/components/pvc-cache/pvc.yaml
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
metadata:
|
||||||
|
name: vllm-cache
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: vllm-server
|
||||||
|
app.kubernetes.io/component: model-cache
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
spec:
|
||||||
|
accessModes:
|
||||||
|
- ReadWriteOnce
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: 100Gi
|
||||||
|
|
||||||
@ -0,0 +1,8 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
patches:
|
||||||
|
- path: service-loadbalancer-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: Service
|
||||||
|
name: vllm-server
|
||||||
|
|
||||||
@ -0,0 +1,4 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/type
|
||||||
|
value: LoadBalancer
|
||||||
|
|
||||||
@ -0,0 +1,8 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
patches:
|
||||||
|
- path: service-nodeport-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: Service
|
||||||
|
name: vllm-server
|
||||||
|
|
||||||
@ -0,0 +1,4 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/type
|
||||||
|
value: NodePort
|
||||||
|
|
||||||
28
tests/kustomize/code-server-nodeport/deployment-patch.yaml
Normal file
28
tests/kustomize/code-server-nodeport/deployment-patch.yaml
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/requests/cpu
|
||||||
|
value: "1"
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/requests/memory
|
||||||
|
value: 2Gi
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/limits/cpu
|
||||||
|
value: "2"
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/limits/memory
|
||||||
|
value: 4Gi
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/args/4
|
||||||
|
value: /home/coder/workspace
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/volumeMounts/0/mountPath
|
||||||
|
value: /home/coder/workspace
|
||||||
|
- op: add
|
||||||
|
path: /spec/template/spec/containers/0/env/-
|
||||||
|
value:
|
||||||
|
name: OCDP_USERNAME
|
||||||
|
value: test-user
|
||||||
|
- op: add
|
||||||
|
path: /spec/template/spec/containers/0/env/-
|
||||||
|
value:
|
||||||
|
name: OCDP_GIT_REPO
|
||||||
|
value: https://example.com/ocdp/test-workspace.git
|
||||||
11
tests/kustomize/code-server-nodeport/generated-secret.yaml
Normal file
11
tests/kustomize/code-server-nodeport/generated-secret.yaml
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: code-server-auth
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: code-server
|
||||||
|
app.kubernetes.io/component: auth
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
type: Opaque
|
||||||
|
stringData:
|
||||||
|
password: test-password
|
||||||
25
tests/kustomize/code-server-nodeport/kustomization.yaml
Normal file
25
tests/kustomize/code-server-nodeport/kustomization.yaml
Normal file
@ -0,0 +1,25 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
namespace: ocdp-workload-test
|
||||||
|
nameSuffix: -test
|
||||||
|
resources:
|
||||||
|
- ../../../apps/code-server/base
|
||||||
|
- generated-secret.yaml
|
||||||
|
components:
|
||||||
|
- ../../../apps/code-server/components/nfs
|
||||||
|
- ../../../apps/code-server/components/service-nodeport
|
||||||
|
patches:
|
||||||
|
- path: deployment-patch.yaml
|
||||||
|
target:
|
||||||
|
group: apps
|
||||||
|
version: v1
|
||||||
|
kind: Deployment
|
||||||
|
name: code-server
|
||||||
|
- path: pvc-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
name: code-server-data
|
||||||
|
- path: service-nodeport-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: Service
|
||||||
|
name: code-server
|
||||||
4
tests/kustomize/code-server-nodeport/pvc-patch.yaml
Normal file
4
tests/kustomize/code-server-nodeport/pvc-patch.yaml
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
- op: replace
|
||||||
|
path: /spec/resources/requests/storage
|
||||||
|
value: 10Gi
|
||||||
|
|
||||||
@ -0,0 +1,4 @@
|
|||||||
|
- op: add
|
||||||
|
path: /spec/ports/0/nodePort
|
||||||
|
value: 30080
|
||||||
|
|
||||||
13
tests/kustomize/litellm-ingress/configmap-patch.yaml
Normal file
13
tests/kustomize/litellm-ingress/configmap-patch.yaml
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
- op: replace
|
||||||
|
path: /data/config.yaml
|
||||||
|
value: |
|
||||||
|
model_list:
|
||||||
|
- model_name: qwen
|
||||||
|
litellm_params:
|
||||||
|
model: openai/qwen2-5-7b
|
||||||
|
api_base: os.environ/LITELLM_UPSTREAM_API_BASE
|
||||||
|
api_key: os.environ/LITELLM_UPSTREAM_API_KEY
|
||||||
|
litellm_settings:
|
||||||
|
drop_params: true
|
||||||
|
request_timeout: 600
|
||||||
|
|
||||||
13
tests/kustomize/litellm-ingress/deployment-patch.yaml
Normal file
13
tests/kustomize/litellm-ingress/deployment-patch.yaml
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/requests/cpu
|
||||||
|
value: "500m"
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/requests/memory
|
||||||
|
value: 1Gi
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/limits/cpu
|
||||||
|
value: "2"
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/limits/memory
|
||||||
|
value: 2Gi
|
||||||
|
|
||||||
14
tests/kustomize/litellm-ingress/generated-secret.yaml
Normal file
14
tests/kustomize/litellm-ingress/generated-secret.yaml
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: litellm-secrets
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: litellm
|
||||||
|
app.kubernetes.io/component: auth
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
type: Opaque
|
||||||
|
stringData:
|
||||||
|
masterKey: sk-test-master
|
||||||
|
saltKey: sk-test-salt
|
||||||
|
upstreamApiKey: sk-no-key-required
|
||||||
|
upstreamApiBase: http://vllm-server-test:8000/v1
|
||||||
3
tests/kustomize/litellm-ingress/ingress-patch.yaml
Normal file
3
tests/kustomize/litellm-ingress/ingress-patch.yaml
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
- op: replace
|
||||||
|
path: /spec/rules/0/host
|
||||||
|
value: litellm.test.local
|
||||||
24
tests/kustomize/litellm-ingress/kustomization.yaml
Normal file
24
tests/kustomize/litellm-ingress/kustomization.yaml
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
namespace: ocdp-workload-test
|
||||||
|
nameSuffix: -test
|
||||||
|
resources:
|
||||||
|
- ../../../apps/litellm/base
|
||||||
|
- generated-secret.yaml
|
||||||
|
components:
|
||||||
|
- ../../../apps/litellm/components/ingress
|
||||||
|
patches:
|
||||||
|
- path: configmap-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: ConfigMap
|
||||||
|
name: litellm-config
|
||||||
|
- path: deployment-patch.yaml
|
||||||
|
target:
|
||||||
|
group: apps
|
||||||
|
version: v1
|
||||||
|
kind: Deployment
|
||||||
|
name: litellm
|
||||||
|
- path: ingress-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: Ingress
|
||||||
|
name: litellm
|
||||||
@ -0,0 +1,19 @@
|
|||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/args/5
|
||||||
|
value: Qwen/Qwen2.5-7B-Instruct
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/args/7
|
||||||
|
value: qwen2-5-7b
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/requests/cpu
|
||||||
|
value: "4"
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/requests/memory
|
||||||
|
value: 24Gi
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/limits/cpu
|
||||||
|
value: "8"
|
||||||
|
- op: replace
|
||||||
|
path: /spec/template/spec/containers/0/resources/limits/memory
|
||||||
|
value: 48Gi
|
||||||
|
|
||||||
@ -0,0 +1,12 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: vllm-secrets
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: vllm-server
|
||||||
|
app.kubernetes.io/component: auth
|
||||||
|
app.kubernetes.io/part-of: ocdp-workload
|
||||||
|
type: Opaque
|
||||||
|
stringData:
|
||||||
|
hfToken: ""
|
||||||
|
|
||||||
22
tests/kustomize/vllm-server-loadbalancer/kustomization.yaml
Normal file
22
tests/kustomize/vllm-server-loadbalancer/kustomization.yaml
Normal file
@ -0,0 +1,22 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
namespace: ocdp-workload-test
|
||||||
|
nameSuffix: -test
|
||||||
|
resources:
|
||||||
|
- ../../../apps/vllm-server/base
|
||||||
|
- generated-secret.yaml
|
||||||
|
components:
|
||||||
|
- ../../../apps/vllm-server/components/gpu-nvidia
|
||||||
|
- ../../../apps/vllm-server/components/pvc-cache
|
||||||
|
- ../../../apps/vllm-server/components/service-loadbalancer
|
||||||
|
patches:
|
||||||
|
- path: deployment-patch.yaml
|
||||||
|
target:
|
||||||
|
group: apps
|
||||||
|
version: v1
|
||||||
|
kind: Deployment
|
||||||
|
name: vllm-server
|
||||||
|
- path: pvc-patch.yaml
|
||||||
|
target:
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
name: vllm-cache
|
||||||
4
tests/kustomize/vllm-server-loadbalancer/pvc-patch.yaml
Normal file
4
tests/kustomize/vllm-server-loadbalancer/pvc-patch.yaml
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
- op: replace
|
||||||
|
path: /spec/resources/requests/storage
|
||||||
|
value: 200Gi
|
||||||
|
|
||||||
Reference in New Issue
Block a user