BHK Cloud documentation

Managed Kubernetes GPU onboarding

This guide takes you from a BHK Cloud account to a CUDA workload on the managed RTX 3090 pool. The service uses standard Kubernetes APIs and an isolated namespace for each customer workspace.

1. Prerequisites

• A BHK Cloud customer account.
• kubectl 1.29 or newer on your workstation.
• A container image compatible with Linux amd64 and NVIDIA CUDA 12.x for GPU workloads.

Verify kubectl:

kubectl version --client

2. Provision a workspace

1. Sign in at ai.bhkcloud.com/login.html.
2. Open Kubernetes in the dashboard navigation.
3. Select Create Workspace.
4. Enter a lowercase name such as ml-prod and submit.

Provisioning creates a namespace, quota, container defaults, network policy, service account, Role, and RoleBinding. The workspace becomes Active after all resources are committed.

3. Download and use kubeconfig

Choose Download kubeconfig beside the active workspace. The bearer token expires after 24 hours; download a fresh file when it expires.

export KUBECONFIG="$HOME/Downloads/bhk-ml-prod-kubeconfig.yaml"
kubectl config current-context
kubectl get pods
kubectl auth can-i create jobs.batch
kubectl auth can-i create namespaces  # expected: no

Treat the kubeconfig as a secret. Do not commit it to Git, paste it into tickets, or embed it in a container image.

4. Run a GPU job

Create gpu-smoke.yaml:

apiVersion: batch/v1
kind: Job
metadata:
  name: gpu-smoke
spec:
  backoffLimit: 1
  template:
    spec:
      restartPolicy: Never
      runtimeClassName: nvidia
      nodeSelector:
        bhkcloud.com/pool: gpu-rtx3090
      containers:
        - name: cuda
          image: nvidia/cuda:12.4.1-base-ubuntu22.04
          command:
            - nvidia-smi
            - --query-gpu=name,memory.total,driver_version
            - --format=csv,noheader
          resources:
            requests:
              cpu: "1"
              memory: 1Gi
              nvidia.com/gpu: "1"
            limits:
              cpu: "2"
              memory: 2Gi
              nvidia.com/gpu: "1"

Apply it and read the result:

kubectl apply -f gpu-smoke.yaml
kubectl get pods -w
kubectl logs job/gpu-smoke
kubectl delete job gpu-smoke

Expected log output includes NVIDIA GeForce RTX 3090, 24576 MiB. Delete completed GPU workloads when no longer needed.

5. Pull private images

kubectl create secret docker-registry registry-credentials \
  --docker-server=registry.example.com \
  --docker-username="$REGISTRY_USER" \
  --docker-password="$REGISTRY_PASSWORD"

# Add to a Pod template:
# imagePullSecrets:
#   - name: registry-credentials

Secrets are restricted to your workspace namespace. They are not accessible to other customer service accounts.

6. Workspace limits and isolation

A default ingress network policy allows traffic from pods in the same workspace. Cross-workspace ingress is denied. Your Role permits normal workload resources but not nodes, namespaces, cluster roles, resource quotas, or other tenants.

7. Access model

The downloaded credential is bound to the workspace-admin service account in your namespace. It can manage Pods, Logs, Exec, Services, ConfigMaps, Secrets, PVCs, Deployments, StatefulSets, DaemonSets, Jobs, CronJobs, HPAs, Ingresses, and NetworkPolicies only inside that namespace.

For CI, download a fresh kubeconfig and store it in the CI provider's encrypted secret store. The 24-hour token limits the lifetime of a leaked credential.

8. Delete a workspace

In the dashboard, choose Delete and confirm. This deletes the Kubernetes namespace and all namespaced workloads, Secrets, Services, and persistent volume claims, then closes the workspace record.

Troubleshooting

Pod remains Pending

kubectl describe pod POD_NAME
kubectl get events --sort-by=.lastTimestamp

If the event says Insufficient nvidia.com/gpu, another GPU workload is active. Your pod starts when capacity is available.

Exceeded quota

Lower CPU, memory, storage, or pod requests. Contact support@bhkcloud.com for a custom quota or reserved node.

Unauthorized

The 24-hour token likely expired. Sign in to the dashboard and download a fresh kubeconfig.

ImagePullBackOff

Confirm the image name and architecture. For private registries, recreate the registry Secret and reference it with imagePullSecrets.