first commit
This commit is contained in:
7
.gitignore
vendored
Normal file
7
.gitignore
vendored
Normal file
@ -0,0 +1,7 @@
|
||||
.ocdp-builds/
|
||||
build/
|
||||
dist/
|
||||
tmp/
|
||||
*.rendered.yaml
|
||||
*.secret.yaml
|
||||
|
||||
44
AGENTS.md
Normal file
44
AGENTS.md
Normal file
@ -0,0 +1,44 @@
|
||||
# AGENTS.md
|
||||
|
||||
This file is the execution guide for `ocdp-workload-manifests`.
|
||||
|
||||
## Working Boundary
|
||||
|
||||
- Treat this project as a plain app manifests repository.
|
||||
- Do not store OCDP WorkloadTemplate records here; they belong in `ocdp-server`
|
||||
PostgreSQL.
|
||||
- Do not store user WorkloadClaim instances here; they belong in target-cluster
|
||||
CRs created through `ocdp-server`.
|
||||
- Keep reusable Kubernetes resources under `apps/*/base`.
|
||||
- Keep optional Kustomize components under `apps/*/components`.
|
||||
- Keep per-deployment runtime values out of this repository. They belong in
|
||||
temporary source files or runtime specs generated by `ocdp-server`.
|
||||
- Use `tests/kustomize/*` only for generic validation overlays, never for real
|
||||
user instances.
|
||||
- Do not commit real passwords, tokens, customer hostnames, private NFS paths, or
|
||||
private registry credentials.
|
||||
|
||||
## Runtime Rules
|
||||
|
||||
- Do not add a global catalog index unless the server explicitly needs one later.
|
||||
- `ocdp-server` WorkloadTemplate records should reference apps with
|
||||
`repositoryUrl`, `ref`, and `path`.
|
||||
- Runtime source generation may create Secret, ConfigMap, and patch files, then
|
||||
run Kustomize.
|
||||
- Bases should stay template-free YAML.
|
||||
- App bases should keep Services as `ClusterIP`; expose apps from
|
||||
WorkloadTemplate values such as `exposure=nodeport` or `exposure=loadbalancer`.
|
||||
Components such as `service-nodeport`, `service-loadbalancer`, or `ingress`
|
||||
are implementation building blocks, not the user-facing contract.
|
||||
- Environment overlays are platform/admin choices for cluster- or site-level
|
||||
differences such as StorageClass, IngressClass, GPU runtime, registry prefix,
|
||||
pull secrets, node selectors, tolerations, and site labels.
|
||||
- NodePort numbers belong in per-claim values handled by `ocdp-server`, not in
|
||||
reusable app bases or components.
|
||||
|
||||
## Validation
|
||||
|
||||
- Run `make validate` after changing workload bases, components, or test
|
||||
overlays.
|
||||
- If a change intentionally affects generated resource names, check references
|
||||
such as Secret refs, PVC claim names, Service backends, and ConfigMap mounts.
|
||||
18
Makefile
Normal file
18
Makefile
Normal file
@ -0,0 +1,18 @@
|
||||
.PHONY: validate list
|
||||
|
||||
KUSTOMIZE ?= kubectl kustomize
|
||||
|
||||
TEST_OVERLAYS := \
|
||||
tests/kustomize/code-server-nodeport \
|
||||
tests/kustomize/vllm-server-loadbalancer \
|
||||
tests/kustomize/litellm-ingress
|
||||
|
||||
list:
|
||||
@printf '%s\n' $(TEST_OVERLAYS)
|
||||
|
||||
validate:
|
||||
@set -eu; \
|
||||
for dir in $(TEST_OVERLAYS); do \
|
||||
printf '==> %s\n' "$$dir"; \
|
||||
$(KUSTOMIZE) "$$dir" >/dev/null; \
|
||||
done
|
||||
125
README.md
Normal file
125
README.md
Normal file
@ -0,0 +1,125 @@
|
||||
# ocdp-workload-manifests
|
||||
|
||||
Standalone Kubernetes manifests for OCDP workloads.
|
||||
|
||||
This repository is intentionally just a Git repository of app manifests. There is
|
||||
no global catalog index and no dependency on the Gitea API. `ocdp-server` can
|
||||
read a workload by building a Kustomize target directly from Git.
|
||||
|
||||
## Architecture
|
||||
|
||||
OCDP keeps the responsibilities split:
|
||||
|
||||
```text
|
||||
Git repo
|
||||
apps/<app>/base
|
||||
apps/<app>/components
|
||||
|
||||
ocdp-server PostgreSQL
|
||||
WorkloadTemplate metadata
|
||||
WorkloadTemplate source.repositoryUrl/ref/path
|
||||
user-facing values schema / parameters
|
||||
environment overlay/profile metadata
|
||||
access bindings
|
||||
no WorkloadClaim instance storage
|
||||
|
||||
target cluster
|
||||
WorkloadClaim CR as the canonical user claim
|
||||
Workload CR
|
||||
runtime Kubernetes resources
|
||||
```
|
||||
|
||||
This repository stores only the Git base and reusable components. It does not
|
||||
store `WorkloadTemplate` records and does not store user `WorkloadClaim`
|
||||
instances.
|
||||
|
||||
An admin creates or updates a `WorkloadTemplate` in `ocdp-server`. That template
|
||||
can point at one of these Git paths:
|
||||
|
||||
```yaml
|
||||
templateType: kustomize
|
||||
source:
|
||||
type: gitKustomize
|
||||
repositoryUrl: https://gitea.example.com/ocdp/ocdp-workload-manifests.git
|
||||
ref: code-server-v0.1.0
|
||||
path: apps/code-server/base
|
||||
```
|
||||
|
||||
After the template is stored in PostgreSQL and assigned to users or groups, users
|
||||
call `ocdp-server` to create a claim by submitting `templateId`, `workspaceId`,
|
||||
and values. `ocdp-server` resolves the template from PostgreSQL, reads the Git
|
||||
base, generates any temporary source files or runtime specs outside this repo,
|
||||
then writes a `WorkloadClaim` CR into the target Kubernetes cluster. The final
|
||||
user-created claim lives in Kubernetes, not in PostgreSQL.
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
apps/
|
||||
code-server/
|
||||
base/
|
||||
components/
|
||||
vllm-server/
|
||||
base/
|
||||
components/
|
||||
litellm/
|
||||
base/
|
||||
components/
|
||||
tests/
|
||||
kustomize/
|
||||
```
|
||||
|
||||
## Server Usage
|
||||
|
||||
`ocdp-server` should store a normal `gitKustomize` reference:
|
||||
|
||||
```yaml
|
||||
source:
|
||||
type: gitKustomize
|
||||
repositoryUrl: https://gitea.example.com/ocdp/ocdp-workload-manifests.git
|
||||
ref: code-server-v0.1.0
|
||||
path: apps/code-server/base
|
||||
```
|
||||
|
||||
For one deployment, `ocdp-server` should generate temporary source files outside
|
||||
this repository. Those files can point at the Git base and add generated
|
||||
Secrets, ConfigMaps, components, and patches.
|
||||
|
||||
## Exposure
|
||||
|
||||
Base services are `ClusterIP`. User-facing exposure choices such as
|
||||
`clusterip`, `nodeport`, and `loadbalancer` belong in the WorkloadTemplate
|
||||
values schema. `ocdp-server` can translate that value into runtime Service
|
||||
configuration or a generated patch.
|
||||
|
||||
Reusable components are still useful implementation building blocks:
|
||||
|
||||
- `components/ingress`: keep the app Service internal and route through an
|
||||
ingress controller.
|
||||
- `components/service-loadbalancer`: change the app Service to `LoadBalancer`.
|
||||
- `components/service-nodeport`: change the app Service to `NodePort`.
|
||||
|
||||
When `nodeport` is selected by the user, `ocdp-server` may generate an instance
|
||||
patch or runtime Service field for an explicit port:
|
||||
|
||||
```yaml
|
||||
- op: add
|
||||
path: /spec/ports/0/nodePort
|
||||
value: 30080
|
||||
```
|
||||
|
||||
Do not hard-code shared NodePort values in app bases or reusable components.
|
||||
|
||||
Environment overlays are different from user choices. They are selected by the
|
||||
platform from cluster, workspace, or customer profile information and can carry
|
||||
things like StorageClass, IngressClass, GPU runtime class, registry prefix,
|
||||
pull-secret wiring, node selectors, tolerations, and site-specific labels.
|
||||
|
||||
## Validate
|
||||
|
||||
```bash
|
||||
make validate
|
||||
```
|
||||
|
||||
`make validate` runs `kubectl kustomize` against non-user test overlays under
|
||||
`tests/kustomize`.
|
||||
7
apps/code-server/README.md
Normal file
7
apps/code-server/README.md
Normal file
@ -0,0 +1,7 @@
|
||||
# code-server
|
||||
|
||||
The base deploys code-server with a PVC, Service, and password Secret reference.
|
||||
|
||||
The Secret is generated by the instance overlay. Do not commit real passwords or
|
||||
tokens to this catalog.
|
||||
|
||||
65
apps/code-server/base/deployment.yaml
Normal file
65
apps/code-server/base/deployment.yaml
Normal file
@ -0,0 +1,65 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: code-server
|
||||
labels:
|
||||
app.kubernetes.io/name: code-server
|
||||
app.kubernetes.io/component: ide
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
annotations: {}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: code-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: code-server
|
||||
app.kubernetes.io/component: ide
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
securityContext:
|
||||
fsGroup: 1000
|
||||
containers:
|
||||
- name: code-server
|
||||
image: codercom/code-server:latest
|
||||
imagePullPolicy: IfNotPresent
|
||||
args:
|
||||
- --bind-addr
|
||||
- 0.0.0.0:8080
|
||||
- --auth
|
||||
- password
|
||||
- /home/coder/project
|
||||
env:
|
||||
- name: PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: code-server-auth
|
||||
key: password
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 8080
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: http
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: http
|
||||
resources:
|
||||
requests:
|
||||
cpu: "500m"
|
||||
memory: 1Gi
|
||||
limits:
|
||||
cpu: "2"
|
||||
memory: 4Gi
|
||||
volumeMounts:
|
||||
- name: workspace
|
||||
mountPath: /home/coder/project
|
||||
volumes:
|
||||
- name: workspace
|
||||
persistentVolumeClaim:
|
||||
claimName: code-server-data
|
||||
|
||||
7
apps/code-server/base/kustomization.yaml
Normal file
7
apps/code-server/base/kustomization.yaml
Normal file
@ -0,0 +1,7 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- pvc.yaml
|
||||
- deployment.yaml
|
||||
- service.yaml
|
||||
|
||||
15
apps/code-server/base/pvc.yaml
Normal file
15
apps/code-server/base/pvc.yaml
Normal file
@ -0,0 +1,15 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: code-server-data
|
||||
labels:
|
||||
app.kubernetes.io/name: code-server
|
||||
app.kubernetes.io/component: workspace-storage
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 20Gi
|
||||
|
||||
16
apps/code-server/base/service.yaml
Normal file
16
apps/code-server/base/service.yaml
Normal file
@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: code-server
|
||||
labels:
|
||||
app.kubernetes.io/name: code-server
|
||||
app.kubernetes.io/component: ide
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
selector:
|
||||
app.kubernetes.io/name: code-server
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: http
|
||||
|
||||
21
apps/code-server/components/ingress/ingress.yaml
Normal file
21
apps/code-server/components/ingress/ingress.yaml
Normal file
@ -0,0 +1,21 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: code-server
|
||||
labels:
|
||||
app.kubernetes.io/name: code-server
|
||||
app.kubernetes.io/component: ingress
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
rules:
|
||||
- host: code-server.example.local
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: code-server
|
||||
port:
|
||||
name: http
|
||||
|
||||
5
apps/code-server/components/ingress/kustomization.yaml
Normal file
5
apps/code-server/components/ingress/kustomization.yaml
Normal file
@ -0,0 +1,5 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
resources:
|
||||
- ingress.yaml
|
||||
|
||||
8
apps/code-server/components/nfs/kustomization.yaml
Normal file
8
apps/code-server/components/nfs/kustomization.yaml
Normal file
@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
patches:
|
||||
- path: pvc-rwx-patch.yaml
|
||||
target:
|
||||
kind: PersistentVolumeClaim
|
||||
name: code-server-data
|
||||
|
||||
4
apps/code-server/components/nfs/pvc-rwx-patch.yaml
Normal file
4
apps/code-server/components/nfs/pvc-rwx-patch.yaml
Normal file
@ -0,0 +1,4 @@
|
||||
- op: replace
|
||||
path: /spec/accessModes/0
|
||||
value: ReadWriteMany
|
||||
|
||||
@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
patches:
|
||||
- path: service-loadbalancer-patch.yaml
|
||||
target:
|
||||
kind: Service
|
||||
name: code-server
|
||||
|
||||
@ -0,0 +1,4 @@
|
||||
- op: add
|
||||
path: /spec/type
|
||||
value: LoadBalancer
|
||||
|
||||
@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
patches:
|
||||
- path: service-nodeport-patch.yaml
|
||||
target:
|
||||
kind: Service
|
||||
name: code-server
|
||||
|
||||
@ -0,0 +1,4 @@
|
||||
- op: add
|
||||
path: /spec/type
|
||||
value: NodePort
|
||||
|
||||
8
apps/litellm/README.md
Normal file
8
apps/litellm/README.md
Normal file
@ -0,0 +1,8 @@
|
||||
# litellm
|
||||
|
||||
LiteLLM gateway with ConfigMap-backed model routing and Secret-backed runtime
|
||||
keys.
|
||||
|
||||
The instance overlay should generate `litellm-secrets` and patch
|
||||
`config.yaml` for the selected upstream model.
|
||||
|
||||
20
apps/litellm/base/configmap.yaml
Normal file
20
apps/litellm/base/configmap.yaml
Normal file
@ -0,0 +1,20 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: litellm-config
|
||||
labels:
|
||||
app.kubernetes.io/name: litellm
|
||||
app.kubernetes.io/component: config
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
data:
|
||||
config.yaml: |
|
||||
model_list:
|
||||
- model_name: default
|
||||
litellm_params:
|
||||
model: openai/default
|
||||
api_base: os.environ/LITELLM_UPSTREAM_API_BASE
|
||||
api_key: os.environ/LITELLM_UPSTREAM_API_KEY
|
||||
litellm_settings:
|
||||
drop_params: true
|
||||
request_timeout: 600
|
||||
|
||||
80
apps/litellm/base/deployment.yaml
Normal file
80
apps/litellm/base/deployment.yaml
Normal file
@ -0,0 +1,80 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: litellm
|
||||
labels:
|
||||
app.kubernetes.io/name: litellm
|
||||
app.kubernetes.io/component: gateway
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
annotations: {}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: litellm
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: litellm
|
||||
app.kubernetes.io/component: gateway
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
containers:
|
||||
- name: litellm
|
||||
image: docker.litellm.ai/berriai/litellm:main-stable
|
||||
imagePullPolicy: IfNotPresent
|
||||
args:
|
||||
- --config
|
||||
- /app/config.yaml
|
||||
- --port
|
||||
- "4000"
|
||||
- --num_workers
|
||||
- "2"
|
||||
env:
|
||||
- name: LITELLM_MASTER_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: litellm-secrets
|
||||
key: masterKey
|
||||
- name: LITELLM_SALT_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: litellm-secrets
|
||||
key: saltKey
|
||||
- name: LITELLM_UPSTREAM_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: litellm-secrets
|
||||
key: upstreamApiKey
|
||||
- name: LITELLM_UPSTREAM_API_BASE
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: litellm-secrets
|
||||
key: upstreamApiBase
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 4000
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health/readiness
|
||||
port: http
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health/liveliness
|
||||
port: http
|
||||
resources:
|
||||
requests:
|
||||
cpu: "250m"
|
||||
memory: 512Mi
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: 1Gi
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /app/config.yaml
|
||||
subPath: config.yaml
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: litellm-config
|
||||
|
||||
7
apps/litellm/base/kustomization.yaml
Normal file
7
apps/litellm/base/kustomization.yaml
Normal file
@ -0,0 +1,7 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- configmap.yaml
|
||||
- deployment.yaml
|
||||
- service.yaml
|
||||
|
||||
16
apps/litellm/base/service.yaml
Normal file
16
apps/litellm/base/service.yaml
Normal file
@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: litellm
|
||||
labels:
|
||||
app.kubernetes.io/name: litellm
|
||||
app.kubernetes.io/component: gateway
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
selector:
|
||||
app.kubernetes.io/name: litellm
|
||||
ports:
|
||||
- name: http
|
||||
port: 4000
|
||||
targetPort: http
|
||||
|
||||
21
apps/litellm/components/ingress/ingress.yaml
Normal file
21
apps/litellm/components/ingress/ingress.yaml
Normal file
@ -0,0 +1,21 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: litellm
|
||||
labels:
|
||||
app.kubernetes.io/name: litellm
|
||||
app.kubernetes.io/component: ingress
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
rules:
|
||||
- host: litellm.example.local
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: litellm
|
||||
port:
|
||||
name: http
|
||||
|
||||
5
apps/litellm/components/ingress/kustomization.yaml
Normal file
5
apps/litellm/components/ingress/kustomization.yaml
Normal file
@ -0,0 +1,5 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
resources:
|
||||
- ingress.yaml
|
||||
|
||||
@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
patches:
|
||||
- path: service-loadbalancer-patch.yaml
|
||||
target:
|
||||
kind: Service
|
||||
name: litellm
|
||||
|
||||
@ -0,0 +1,4 @@
|
||||
- op: add
|
||||
path: /spec/type
|
||||
value: LoadBalancer
|
||||
|
||||
@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
patches:
|
||||
- path: service-nodeport-patch.yaml
|
||||
target:
|
||||
kind: Service
|
||||
name: litellm
|
||||
|
||||
@ -0,0 +1,4 @@
|
||||
- op: add
|
||||
path: /spec/type
|
||||
value: NodePort
|
||||
|
||||
8
apps/vllm-server/README.md
Normal file
8
apps/vllm-server/README.md
Normal file
@ -0,0 +1,8 @@
|
||||
# vllm-server
|
||||
|
||||
OpenAI-compatible model serving with vLLM.
|
||||
|
||||
The base is CPU-safe YAML. Add `components/gpu-nvidia` in environments that
|
||||
provide NVIDIA GPUs, and let the instance overlay patch model name, resources,
|
||||
and cache size.
|
||||
|
||||
58
apps/vllm-server/base/deployment.yaml
Normal file
58
apps/vllm-server/base/deployment.yaml
Normal file
@ -0,0 +1,58 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: vllm-server
|
||||
labels:
|
||||
app.kubernetes.io/name: vllm-server
|
||||
app.kubernetes.io/component: model-server
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
annotations: {}
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: vllm-server
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: vllm-server
|
||||
app.kubernetes.io/component: model-server
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
containers:
|
||||
- name: vllm
|
||||
image: vllm/vllm-openai:latest
|
||||
imagePullPolicy: IfNotPresent
|
||||
args:
|
||||
- --host
|
||||
- 0.0.0.0
|
||||
- --port
|
||||
- "8000"
|
||||
- --model
|
||||
- Qwen/Qwen2.5-7B-Instruct
|
||||
- --served-model-name
|
||||
- default
|
||||
env:
|
||||
- name: HF_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: vllm-secrets
|
||||
key: hfToken
|
||||
optional: true
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 8000
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: http
|
||||
initialDelaySeconds: 20
|
||||
periodSeconds: 10
|
||||
resources:
|
||||
requests:
|
||||
cpu: "2"
|
||||
memory: 12Gi
|
||||
limits:
|
||||
cpu: "4"
|
||||
memory: 24Gi
|
||||
|
||||
6
apps/vllm-server/base/kustomization.yaml
Normal file
6
apps/vllm-server/base/kustomization.yaml
Normal file
@ -0,0 +1,6 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
resources:
|
||||
- deployment.yaml
|
||||
- service.yaml
|
||||
|
||||
16
apps/vllm-server/base/service.yaml
Normal file
16
apps/vllm-server/base/service.yaml
Normal file
@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: vllm-server
|
||||
labels:
|
||||
app.kubernetes.io/name: vllm-server
|
||||
app.kubernetes.io/component: model-server
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
selector:
|
||||
app.kubernetes.io/name: vllm-server
|
||||
ports:
|
||||
- name: http
|
||||
port: 8000
|
||||
targetPort: http
|
||||
|
||||
7
apps/vllm-server/components/gpu-nvidia/gpu-patch.yaml
Normal file
7
apps/vllm-server/components/gpu-nvidia/gpu-patch.yaml
Normal file
@ -0,0 +1,7 @@
|
||||
- op: add
|
||||
path: /spec/template/spec/runtimeClassName
|
||||
value: nvidia
|
||||
- op: add
|
||||
path: /spec/template/spec/containers/0/resources/limits/nvidia.com~1gpu
|
||||
value: 1
|
||||
|
||||
10
apps/vllm-server/components/gpu-nvidia/kustomization.yaml
Normal file
10
apps/vllm-server/components/gpu-nvidia/kustomization.yaml
Normal file
@ -0,0 +1,10 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
patches:
|
||||
- path: gpu-patch.yaml
|
||||
target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: vllm-server
|
||||
|
||||
21
apps/vllm-server/components/ingress/ingress.yaml
Normal file
21
apps/vllm-server/components/ingress/ingress.yaml
Normal file
@ -0,0 +1,21 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: vllm-server
|
||||
labels:
|
||||
app.kubernetes.io/name: vllm-server
|
||||
app.kubernetes.io/component: ingress
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
rules:
|
||||
- host: vllm.example.local
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: vllm-server
|
||||
port:
|
||||
name: http
|
||||
|
||||
5
apps/vllm-server/components/ingress/kustomization.yaml
Normal file
5
apps/vllm-server/components/ingress/kustomization.yaml
Normal file
@ -0,0 +1,5 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
resources:
|
||||
- ingress.yaml
|
||||
|
||||
@ -0,0 +1,17 @@
|
||||
- op: add
|
||||
path: /spec/template/spec/containers/0/env/-
|
||||
value:
|
||||
name: HF_HOME
|
||||
value: /cache/huggingface
|
||||
- op: add
|
||||
path: /spec/template/spec/containers/0/volumeMounts
|
||||
value:
|
||||
- name: model-cache
|
||||
mountPath: /cache
|
||||
- op: add
|
||||
path: /spec/template/spec/volumes
|
||||
value:
|
||||
- name: model-cache
|
||||
persistentVolumeClaim:
|
||||
claimName: vllm-cache
|
||||
|
||||
12
apps/vllm-server/components/pvc-cache/kustomization.yaml
Normal file
12
apps/vllm-server/components/pvc-cache/kustomization.yaml
Normal file
@ -0,0 +1,12 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
resources:
|
||||
- pvc.yaml
|
||||
patches:
|
||||
- path: deployment-cache-patch.yaml
|
||||
target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: vllm-server
|
||||
|
||||
15
apps/vllm-server/components/pvc-cache/pvc.yaml
Normal file
15
apps/vllm-server/components/pvc-cache/pvc.yaml
Normal file
@ -0,0 +1,15 @@
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: vllm-cache
|
||||
labels:
|
||||
app.kubernetes.io/name: vllm-server
|
||||
app.kubernetes.io/component: model-cache
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 100Gi
|
||||
|
||||
@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
patches:
|
||||
- path: service-loadbalancer-patch.yaml
|
||||
target:
|
||||
kind: Service
|
||||
name: vllm-server
|
||||
|
||||
@ -0,0 +1,4 @@
|
||||
- op: add
|
||||
path: /spec/type
|
||||
value: LoadBalancer
|
||||
|
||||
@ -0,0 +1,8 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1alpha1
|
||||
kind: Component
|
||||
patches:
|
||||
- path: service-nodeport-patch.yaml
|
||||
target:
|
||||
kind: Service
|
||||
name: vllm-server
|
||||
|
||||
@ -0,0 +1,4 @@
|
||||
- op: add
|
||||
path: /spec/type
|
||||
value: NodePort
|
||||
|
||||
28
tests/kustomize/code-server-nodeport/deployment-patch.yaml
Normal file
28
tests/kustomize/code-server-nodeport/deployment-patch.yaml
Normal file
@ -0,0 +1,28 @@
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/requests/cpu
|
||||
value: "1"
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/requests/memory
|
||||
value: 2Gi
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/limits/cpu
|
||||
value: "2"
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/limits/memory
|
||||
value: 4Gi
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/args/4
|
||||
value: /home/coder/workspace
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/volumeMounts/0/mountPath
|
||||
value: /home/coder/workspace
|
||||
- op: add
|
||||
path: /spec/template/spec/containers/0/env/-
|
||||
value:
|
||||
name: OCDP_USERNAME
|
||||
value: test-user
|
||||
- op: add
|
||||
path: /spec/template/spec/containers/0/env/-
|
||||
value:
|
||||
name: OCDP_GIT_REPO
|
||||
value: https://example.com/ocdp/test-workspace.git
|
||||
11
tests/kustomize/code-server-nodeport/generated-secret.yaml
Normal file
11
tests/kustomize/code-server-nodeport/generated-secret.yaml
Normal file
@ -0,0 +1,11 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: code-server-auth
|
||||
labels:
|
||||
app.kubernetes.io/name: code-server
|
||||
app.kubernetes.io/component: auth
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
type: Opaque
|
||||
stringData:
|
||||
password: test-password
|
||||
25
tests/kustomize/code-server-nodeport/kustomization.yaml
Normal file
25
tests/kustomize/code-server-nodeport/kustomization.yaml
Normal file
@ -0,0 +1,25 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: ocdp-workload-test
|
||||
nameSuffix: -test
|
||||
resources:
|
||||
- ../../../apps/code-server/base
|
||||
- generated-secret.yaml
|
||||
components:
|
||||
- ../../../apps/code-server/components/nfs
|
||||
- ../../../apps/code-server/components/service-nodeport
|
||||
patches:
|
||||
- path: deployment-patch.yaml
|
||||
target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: code-server
|
||||
- path: pvc-patch.yaml
|
||||
target:
|
||||
kind: PersistentVolumeClaim
|
||||
name: code-server-data
|
||||
- path: service-nodeport-patch.yaml
|
||||
target:
|
||||
kind: Service
|
||||
name: code-server
|
||||
4
tests/kustomize/code-server-nodeport/pvc-patch.yaml
Normal file
4
tests/kustomize/code-server-nodeport/pvc-patch.yaml
Normal file
@ -0,0 +1,4 @@
|
||||
- op: replace
|
||||
path: /spec/resources/requests/storage
|
||||
value: 10Gi
|
||||
|
||||
@ -0,0 +1,4 @@
|
||||
- op: add
|
||||
path: /spec/ports/0/nodePort
|
||||
value: 30080
|
||||
|
||||
13
tests/kustomize/litellm-ingress/configmap-patch.yaml
Normal file
13
tests/kustomize/litellm-ingress/configmap-patch.yaml
Normal file
@ -0,0 +1,13 @@
|
||||
- op: replace
|
||||
path: /data/config.yaml
|
||||
value: |
|
||||
model_list:
|
||||
- model_name: qwen
|
||||
litellm_params:
|
||||
model: openai/qwen2-5-7b
|
||||
api_base: os.environ/LITELLM_UPSTREAM_API_BASE
|
||||
api_key: os.environ/LITELLM_UPSTREAM_API_KEY
|
||||
litellm_settings:
|
||||
drop_params: true
|
||||
request_timeout: 600
|
||||
|
||||
13
tests/kustomize/litellm-ingress/deployment-patch.yaml
Normal file
13
tests/kustomize/litellm-ingress/deployment-patch.yaml
Normal file
@ -0,0 +1,13 @@
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/requests/cpu
|
||||
value: "500m"
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/requests/memory
|
||||
value: 1Gi
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/limits/cpu
|
||||
value: "2"
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/limits/memory
|
||||
value: 2Gi
|
||||
|
||||
14
tests/kustomize/litellm-ingress/generated-secret.yaml
Normal file
14
tests/kustomize/litellm-ingress/generated-secret.yaml
Normal file
@ -0,0 +1,14 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: litellm-secrets
|
||||
labels:
|
||||
app.kubernetes.io/name: litellm
|
||||
app.kubernetes.io/component: auth
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
type: Opaque
|
||||
stringData:
|
||||
masterKey: sk-test-master
|
||||
saltKey: sk-test-salt
|
||||
upstreamApiKey: sk-no-key-required
|
||||
upstreamApiBase: http://vllm-server-test:8000/v1
|
||||
3
tests/kustomize/litellm-ingress/ingress-patch.yaml
Normal file
3
tests/kustomize/litellm-ingress/ingress-patch.yaml
Normal file
@ -0,0 +1,3 @@
|
||||
- op: replace
|
||||
path: /spec/rules/0/host
|
||||
value: litellm.test.local
|
||||
24
tests/kustomize/litellm-ingress/kustomization.yaml
Normal file
24
tests/kustomize/litellm-ingress/kustomization.yaml
Normal file
@ -0,0 +1,24 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: ocdp-workload-test
|
||||
nameSuffix: -test
|
||||
resources:
|
||||
- ../../../apps/litellm/base
|
||||
- generated-secret.yaml
|
||||
components:
|
||||
- ../../../apps/litellm/components/ingress
|
||||
patches:
|
||||
- path: configmap-patch.yaml
|
||||
target:
|
||||
kind: ConfigMap
|
||||
name: litellm-config
|
||||
- path: deployment-patch.yaml
|
||||
target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: litellm
|
||||
- path: ingress-patch.yaml
|
||||
target:
|
||||
kind: Ingress
|
||||
name: litellm
|
||||
@ -0,0 +1,19 @@
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/args/5
|
||||
value: Qwen/Qwen2.5-7B-Instruct
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/args/7
|
||||
value: qwen2-5-7b
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/requests/cpu
|
||||
value: "4"
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/requests/memory
|
||||
value: 24Gi
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/limits/cpu
|
||||
value: "8"
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources/limits/memory
|
||||
value: 48Gi
|
||||
|
||||
@ -0,0 +1,12 @@
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: vllm-secrets
|
||||
labels:
|
||||
app.kubernetes.io/name: vllm-server
|
||||
app.kubernetes.io/component: auth
|
||||
app.kubernetes.io/part-of: ocdp-workload
|
||||
type: Opaque
|
||||
stringData:
|
||||
hfToken: ""
|
||||
|
||||
22
tests/kustomize/vllm-server-loadbalancer/kustomization.yaml
Normal file
22
tests/kustomize/vllm-server-loadbalancer/kustomization.yaml
Normal file
@ -0,0 +1,22 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
namespace: ocdp-workload-test
|
||||
nameSuffix: -test
|
||||
resources:
|
||||
- ../../../apps/vllm-server/base
|
||||
- generated-secret.yaml
|
||||
components:
|
||||
- ../../../apps/vllm-server/components/gpu-nvidia
|
||||
- ../../../apps/vllm-server/components/pvc-cache
|
||||
- ../../../apps/vllm-server/components/service-loadbalancer
|
||||
patches:
|
||||
- path: deployment-patch.yaml
|
||||
target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: vllm-server
|
||||
- path: pvc-patch.yaml
|
||||
target:
|
||||
kind: PersistentVolumeClaim
|
||||
name: vllm-cache
|
||||
4
tests/kustomize/vllm-server-loadbalancer/pvc-patch.yaml
Normal file
4
tests/kustomize/vllm-server-loadbalancer/pvc-patch.yaml
Normal file
@ -0,0 +1,4 @@
|
||||
- op: replace
|
||||
path: /spec/resources/requests/storage
|
||||
value: 200Gi
|
||||
|
||||
Reference in New Issue
Block a user