Nutanix Enterprise AI Manual: Deployment and Installation Guide

Nutanix Docker Hub access tokens, Helm chart configuration parameters for nai-operators and nai-core, connected deployment on Nutanix Kubernetes Platform (NKP), air-gapped NKP deployment with private registries, deployment on Amazon EKS, Azure AKS, Google GKE, self-managed PostgreSQL configurations, dashboard IP access, and Docker registry credentials rotation.

Deploy NAI on NKP in an air#gapped environment

Deploying Nutanix Enterprise AI on a Nutanix Kubernetes Platform Cluster in

For more information, see

Air-Gapped Environments

on page 50.

Deploy NAI on Amazon Elastic Kubernetes Service (EKS). For more information, see

Deploy Nutanix

Enterprise AI on Amazon Elastic Kubernetes Service

on page 67.

Deploy NAI on Azure Kubernetes Service (AKS)

For more information, see

Deploying Nutanix Enterprise AI on Azure Kubernetes Service

on page 76.

Deploy NAI on Google Kubernetes Engine (GKE)

For more information, see

Deploy Nutanix Enterprise AI on Google Kubernetes Engine

on page 85.

Generating Nutanix Docker Hub Access Tokens

Generate tokens that you can use to access the Nutanix Docker Hub private repository.

Before you begin

If your environment is air-gapped, you cannot use this procedure. For more information on the procedure for an air-gapped environment, see

Downloading Product Files

.

About this task

To generate Nutanix Docker Hub Access Tokens, follow these steps:

Procedure

1. Log in to the Nutanix Support portal.

2. In the upper-left corner of the Nutanix Support Portal, click the

menu

Downloads

icon, then click

.

3. Click

Nutanix Enterprise AI

.

4. Click

Generate Access Token

.

You can generate a maximum of two tokens.

Manage Docker Hub Access Token

The

dialog box appears and generates a token. Once generated, this

token remains valid for all Nutanix Enterprise AI (NAI) releases. You only need to generate a replacement if the existing token is manually deleted or revoked.

What to do next

For information on how to use your tokens to download and install Nutanix Enterprise AI, see

Deploy

Nutanix Enterprise AI

on page 27.

Nutanix Enterprise AI Configuration Parameters for the

Helm Chart

nai-operators

The following tables list the configurable parameters of the

helm chart and their default
nai-operators

values.

Table 15: Global Parameters

Table 16: Global Storage Parameters

Table 17: AI Gateway Parameters

Table 18: Valkey Parameters

Table 19: Clickhouse Operators Parameters

Table 20: NAI Database Parameters

Table 21: NAI Jobs Parameters

The following tables lists the configurable parameters of the

Helm chart and their default values.

nai-core

Table 22: Global Parameters

Table 23: Clickhouse keeper

Table 24: Clickhouse Server

Table 25: Storage Parameters

Table 26: NAI IEP Operator Parameters

Table 27: NAI Inference UI Parameters

Table 28: NAI API Parameters

Table 29: NAI Database Parameters

[!NOTE] Note:

The

chart owns the database credentials Secrets.

contains the non-secret

nai-operators
nai-core

connection topology used to render service configuration.

For an internal CNPG deployment, the

and

databases are co-located in the

iep
iam
nai-db-iep

CNPG cluster by default.

For an external PostgreSQL deployment, configure

consistently in both

naiDatabase.external

charts. Configure the corresponding database connection details in both charts as described earlier.

The

chart automatically detects legacy PostgreSQL imports using Helm lookup. Use

nai-operators

the

setting only for advanced or break-glass scenarios.

importOverride

Component-specific

values are intentionally empty by default so that they inherit

storageClass

.

global.storage.storageClassName

Table 30: NAI Monitoring Parameters

Table 31: NAI Labs

[!NOTE] Note: Only one-way TLS is supported (client verifies server certificate)

naiLabs.vectorDb.milvus.milvusTLSSecretName

K8s secret containing the CA certificate for SSL connection to be created in the nai-system namespac

nai-milvus-tls-certs

naiLabs.vectorDb.milvus.milvusTLSCACertName

CA certificate key name within the secret . For example, “ca.pem”.

Leave empty if your Milvus uses a publicly trusted CA (e.g., Let’s Encrypt, Zilliz Cloud)

naiLabs.vectorDb.milvus.milvusTLSServerName

Hostname for TLS certificate verification (optional).

If left blank, defaults to the host value above for hostname verification

Set this if the certificate’s CN/SAN doesn’t match the host value.

For example, host is an IP address (10.111.48.98) but certificate is issued for a hostname (milvus.example.com)

naiLabs.enabled

Flag to deploy Chat and Talk to Data app

false

Table 32: AI Gateway

Table 33: NAI Agent

Install or upgrade Nutanix Enterprise AI (NAI) on Nutanix Kubernetes Platform (NKP).

To install or upgrade NAI on NKP, follow these high-level steps:

1. Set up an NKP cluster. For more information, see

Setting up a Nutanix Kubernetes Platform Cluster

on

page 41.

2. Perform preflight checks. For more information, see

Performing Preflight Checks Before Deploying Nutanix

Enterprise AI on Nutanix Kubernetes Platform

on page 42.

Installing Prerequisite Components on a Nutanix

3. Install prerequisite components. For more information, see

Kubernetes Platform Cluster

on page 43.

4. Deploy NAI on NKP. For more information, see

Deploying Nutanix Enterprise AI on Nutanix Kubernetes

Platform

on page 46.

Setting up a Nutanix Kubernetes Platform Cluster Set up a Nutanix Kubernetes Platform (NKP) cluster before you deploy Nutanix Enterprise AI (NAI).

About this task

To set up your NKP cluster, follow these steps:

Procedure

1. Create an NKP cluster.

Ensure that the Kubernetes version is 1.35and the nodes are Ubuntu based images. For more information, see the

Nutanix Kubernetes® Platform Guide

.

2. Add the NKP Pro or Ultimate license to your NKP Cluster.

Add an NKP License

For more information, see

in the Nutanix Kubernetes® Platform Guide .

3. Ensure that the NKP cluster meets all the requirements listed in

Nutanix Enterprise AI - Private Inference and

Agent Gateway Requirements

on page 10.

4. Create a Nutanix Files server with sufficient storage capacity to save the models.

Creating a File Server

For more information, see

in the Nutanix Files User Guide .

For information on the size of the models, see

Table 49: Pre-validated Models

5. Create a Network File System (NFS v4) export in the Nutanix Files server to import a custom model or to import

a model manually.

Creating an NFS Export

For more information, see

in the Nutanix Files User Guide .

6. (Optional) To override the default configuration with a custom CNI plugin, ensure that NetworkPolicy

enforcement is enabled.

By default, NKP)installs the Cilium add-on with the default configuration while creating a cluster.

What to do next

Perform preflight checks. For more information, see

Performing Preflight Checks Before Deploying Nutanix

Enterprise AI on Nutanix Kubernetes Platform

on page 42.

Performing Preflight Checks Before Deploying Nutanix Enterprise AI on Nutanix Kubernetes Platform Perform preflight checks before installing or upgrading Nutanix Enterprise AI (NAI) on Nutanix Kubernetes Platform (NKP). Preflight checks ensure the Kubernetes environment meets all the technical requirements for a successful NAI installation.

About this task

To perform preflight checks, follow these steps:

Procedure

1. Verify that the Kubernetes version is 1.33 or 1.34:

kubectl get nodes \
--selector='!node-role.kubernetes.io/control-plane,!node-role.kubernetes.io/master'
\
-o custom-
columns=NODE:.metadata.name,KUBELET_VERSION:.status.nodeInfo.kubeletVersion

The expected output is that the Kubernetes version must be 1.33 or 1.34.

2. Verify if the number of CSI pods matches the number of worker nodes:

[ $(kubectl get nodes --no-headers | wc -l) -eq $(kubectl get pods -n ntnx-system --
no-headers | grep csi-node | wc -l) ] && echo "# CSI pods = node count" || echo "#
Mismatch: CSI pods != node count"

The expected output is CSI pods = node count.

3. Verify that the

class binds volumes immediately:

nai-nfs-storage
kubectl get storageclass nai-nfs-storage -o jsonpath='{.volumeBindingMode}{"\n"}'

The expected output is

.

immediate

4. Verify if the

storage class has ReadWriteMany access:

nai-nfs-storage
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-rwx-pvc
namespace: default
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
storageClassName: nai-nfs-storage
EOF
kubectl get pvc test-rwx-pvc -n default

The expected output is that the storage class

has

set to

.

nai-nfs-storage
Access Modes
RWX
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS
MODES   STORAGECLASS      VOLUMEATTRIBUTESCLASS   AGE
test-rwx-pvc            Bound    pvc-0df5145a-590c-4f28-9f9d-cefefc55266c   1Gi
RWX            nai-nfs-storage   <unset>                 5s
kubectl delete pvc test-rwx-pvc

Installing Prerequisite Components on a Nutanix Kubernetes Platform Cluster Install prerequisite components on a Nutanix Kubernetes Platform (NKP) cluster.

Before you begin

Complete the following:

1. Set up an NKP cluster.

Setting up a Nutanix Kubernetes Platform Cluster

For more information, see

on page 41. cert-manager is

deployed when you set up an NKP cluster.

2. Perform preflight checks.

For more information, see

Performing Preflight Checks Before Deploying Nutanix Enterprise AI on Nutanix

Kubernetes Platform

on page 42.

About this task

To install components on an NKP 2.18 cluster, follow these steps:

Procedure

1. Install or upgrade Envoy Gateway:

a. Install Envoy Gateway and Gateway API CRDs:

helm template eg oci://docker.io/envoyproxy/gateway-crds-helm --version v1.8.1 \
--set crds.gatewayAPI.enabled=true \
--set crds.envoyGateway.enabled=true \
| kubectl apply --server-side --force-conflicts -f -

b. Create

with the following configuration:

envoy-gateway-config.yaml
config:
envoyGateway:
gateway:
controllerName: "gateway.envoyproxy.io/gatewayclass-controller"
logging:
level:
default: "info"
provider:
kubernetes:
rateLimitDeployment:
container:
image: "docker.io/envoyproxy/ratelimit:1e50889b"
patch:
type: "StrategicMerge"
value:
spec:
template:
spec:
containers:
- imagePullPolicy: "IfNotPresent"
name: "envoy-ratelimit"
image: "docker.io/envoyproxy/ratelimit:1e50889b"
env:
- name: REDIS_TYPE
value: "sentinel"
- name: REDIS_PIPELINE_WINDOW
value: "150us"
type: "Kubernetes"
extensionApis:
enableEnvoyPatchPolicy: true
enableBackend: true
extensionManager:
maxMessageSize: 11Mi
backendResources:
- group: inference.networking.k8s.io
kind: InferencePool
version: v1
hooks:
xdsTranslator:
translation:
listener:
includeAll: true
route:
includeAll: true
cluster:
includeAll: true
secret:
includeAll: true
post:
- "Translation"
- "Cluster"
- "Route"
service:
fqdn:
hostname: "ai-gateway-controller.nai-system.svc.cluster.local"
port: 1063
rateLimit:
backend:
type: "Redis"
redis:
url: "mymaster,nai-valkey-sentinel.nai-system.svc.cluster.local:26379"

[!NOTE] Note: The rate-limit backend uses Valkey Sentinel with master name

mymaster

and the

nai-valkey-

service in

.

sentinel
nai-system

c. Install or Upgrade Envoy Gateway:

helm upgrade --install eg oci://docker.io/envoyproxy/gateway-helm --version v1.8.1
\
-n envoy-gateway-system --create-namespace --skip-crds \
-f "./envoy-gateway-config.yaml"

2. Install or Upgrade KServe:

The required version is KSERVE_VERSION=v0.19.0.

a. Install or upgrade the KServe CRDs:

helm upgrade --install kserve-crd oci://ghcr.io/kserve/charts/kserve-crd \
--version $KSERVE_VERSION -n kserve --create-namespace --wait

b. Install or upgrade the KServe resources:

helm upgrade --install kserve oci://ghcr.io/kserve/charts/kserve-resources \
--version $KSERVE_VERSION -n kserve --create-namespace --wait \
--set kserve.controller.deploymentMode=RawDeployment \
--set kserve.controller.gateway.disableIngressCreation=true

c. Install or upgrade the KServe LLMInferenceService CRD:

helm upgrade --install kserve-llmisvc-crd oci://ghcr.io/kserve/charts/kserve-
llmisvc-crd \
--version $KSERVE_VERSION -n kserve --create-namespace --wait

d. Install or upgrade the KServe LLMInferenceService resources:

helm upgrade --install kserve-llmisvc-resources oci://ghcr.io/kserve/charts/
kserve-llmisvc-resources \
--version $KSERVE_VERSION -n kserve --create-namespace --wait \
--set kserve.createSharedResources=false \
--set kserve.llmisvc.createGIECRDs=false

3. Install CloudNativePG Operator from NKP platform applications.

4. Install LeaderWorkerSet:

helm install lws oci://registry.k8s.io/lws/charts/lws \
--version 0.8.0 -n lws-system --create-namespace --wait

5. Install or upgrade OpenTelemetry Operator

helm upgrade --install opentelemetry-operator opentelemetry-operator \
--repo https://open-telemetry.github.io/opentelemetry-helm-charts \
--version=0.114.1 -n opentelemetry --create-namespace --wait

6. Install Prometheus Monitoring from NKP platform applications:

To optimize resource utilization on the workload cluster, configure Prometheus Monitoring with the following minimum installation settings when you enable the application:

alertmanager:
enabled: false
grafana:
enabled: false
prometheus:
enabled: false
kubeStateMetrics:
enabled: false
kubernetesServiceMonitors:
enabled: false
prometheus-node-exporter.kubeRBACProxy:
kubeRBACProxy:
enabled: true

Pro: Enabling an Application Using the UI

For more information, see

.

7. Install NVIDIA GPU Operator from NKP platform applications.

If the GPU nodes do not have precompiled NVIDIA drivers installed, enable driver installation in the NVIDIA GPU Operator configuration. This setting ensures that the NVIDIA drivers are installed on the GPU nodes. When enabling the NVIDIA GPU Operator, add the following cluster override:

driver:
enabled: true

Pro: Enabling an Application Using the UI

For more information, see

.

What to do next

1. Verify that the Envoy Gateway CRDs and controller are installed and ready.
2. Verify that the KServe CRDs and controller resources are ready.
3. Verify that the CloudNativePG operator is ready.
4. Verify that the LeaderWorkerSet controller is ready.
5. Verify that the OpenTelemetry Operator is ready.
6. Verify that Prometheus monitoring is ready.
7. Verify that the NVIDIA GPU Operator is ready.

Deploying Nutanix Enterprise AI on Nutanix Kubernetes Platform

on page 46

Deploying Nutanix Enterprise AI on Nutanix Kubernetes Platform Install or upgrade Nutanix Enterprise AI on Nutanix Kubernetes Platform (NKP).

Before you begin

Ensure to complete the following:

Installing Prerequisite Components

Install components on the Kubernetes cluster. For more information, see

on a Nutanix Kubernetes Platform Cluster

on page 43.

Ensure that the required RWO/RWX storage classes exist.

Ensure that the registry secret is available for NAI images.

Upgrades from version 2.7.0 to 2.8.0 must be executed during planned downtime. User logins will be unavailable during the upgrade, and full functionality will resume automatically once the upgrade is complete.

About this task

To install or upgrade Nutanix Enterprise AI on NKP, follow these steps:

Procedure

1. Choose a profile:

Profile-based deployment allows you to select a predefined configuration based on your environment and availability requirements. The default profile uses single replica for components and is intended for baseline deployment. The other profiles are for higher capacity usage.

Table 34: Profile and Capacity

2. Pull and untar both the 2.8.0 charts

helm pull ntnx-charts/nai-operators --version 2.8.0 --untar=true
helm pull ntnx-charts/nai-core --version 2.8.0 --untar=true

The extracted charts contain profile files such as:

./nai-operators/profiles/c1k_k200.yaml
./nai-operators/profiles/c5k_k1k.yaml
./nai-core/profiles/c1k_k200.yaml
./nai-core/profiles/c5k_k1k.yaml

3. Deploy for c1k_k200 profile

a. Deploy NAI Operators for c1k_k200 profile

export NAI_DEFAULT_RWO_STORAGECLASS=<RWO storageclass>
helm upgrade --install nai-operators ntnx-charts/nai-operators --version 2.8.0 \
-n nai-system --create-namespace --wait --timeout 15m \
--set "global.storage.storageClassName=${NAI_DEFAULT_RWO_STORAGECLASS}" \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}" \
-f ./nai-operators/profiles/c1k_k200.yaml

b. Deploy NAI Core for c1k_k200 profile

export NAI_API_RWX_STORAGECLASS=<NFS Storageclass i.e nai-nfs-storage>
export NAI_DEFAULT_RWO_STORAGECLASS=<RWO default storageclass>
helm upgrade --install nai-core ntnx-charts/nai-core --version=2.8.0 \
-n nai-system --create-namespace --wait --timeout 15m \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}" \
--set "global.storage.storageClassNameRWX=${NAI_API_RWX_STORAGECLASS}" \
--set "global.storage.storageClassName=${NAI_DEFAULT_RWO_STORAGECLASS}" \
-f ./nai-core/profiles/c1k_k200.yaml

4. Set up the NAI Helm repository. a. Add and update the Nutanix Helm repository, which contains the

and

helm chart:
nai-core
nai-operators
helm repo add ntnx-charts https://nutanix.github.io/helm-releases && helm repo
update ntnx-charts

b. Search for the version of the

and

helm chart available for installation in the
nai-core
nai-operators

Nutanix helm repository:

helm search repo ntnx-charts/nai-operators --versions
helm search repo ntnx-charts/nai-core --versions

5. Create Docker Registry Secrets:

Create the

namespace and the

secret in both

and

nai-system
docker-registry
nai-system
envoy-

namespaces.

gateway-system

The

is already present on the cluster.

envoy-gateway-system namespace
export REGISTRY_SECRET_NAME=nai-regcred
export DOCKER_SERVER=https://index.docker.io/v1/
export DOCKER_USERNAME=<docker-username>
export DOCKER_PASSWORD=<docker-password>
export DOCKER_EMAIL=<docker-email>
kubectl create namespace nai-system --dry-run=client -o yaml | kubectl apply -f -
kubectl -n nai-system create secret docker-registry ${REGISTRY_SECRET_NAME} \
--docker-server=${DOCKER_SERVER} \
--docker-username=${DOCKER_USERNAME} \
--docker-password=${DOCKER_PASSWORD} \
--docker-email=${DOCKER_EMAIL} \
--dry-run=client -o yaml | kubectl apply -f -
kubectl -n envoy-gateway-system create secret docker-registry ${REGISTRY_SECRET_NAME}
\
--docker-server=${DOCKER_SERVER} \
--docker-username=${DOCKER_USERNAME} \
--docker-password=${DOCKER_PASSWORD} \
--docker-email=${DOCKER_EMAIL} \
--dry-run=client -o yaml | kubectl apply -f -

6. Deploy NAI Operators:

helm upgrade --install nai-operators ntnx-charts/nai-operators --version 2.8.0  -n
nai-system --create-namespace --take-ownership --wait \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}"

[!NOTE] Note: Ensure the REGISTRY_SECRET_NAME environment variable is set before running this command.

7. Deploy NAI Core on NKP:

# Set the environment variable
export NAI_API_RWX_STORAGECLASS=<NFS Storageclass i.e nai-nfs-storage>
export NAI_DEFAULT_RWO_STORAGECLASS=<default storageclass i.e nutanix-volume>
export NKP_WORKSPACE_NAMESPACE=kommander-default-workspace # update this env as per
your workspace
export REGISTRY_SECRET_NAME=<secret created>
helm upgrade --install nai-core ntnx-charts/nai-core --version 2.8.0 -n nai-system --
create-namespace --force-conflicts --wait \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}" \
--set "global.storage.storageClassNameRWX=${NAI_API_RWX_STORAGECLASS}" \
--set "global.storage.storageClassName=${NAI_DEFAULT_RWO_STORAGECLASS}" \
--set
"naiMonitoring.nodeExporter.serviceMonitor.namespaceSelector.matchNames[0]=
${NKP_WORKSPACE_NAMESPACE}" \
--set
"naiMonitoring.dcgmExporter.serviceMonitor.namespaceSelector.matchNames[0]=
${NKP_WORKSPACE_NAMESPACE}"

[!NOTE] Note: You can append optional Helm overrides to the nai-core installation command to customize the deployment

Enable the Chat and Talk to My Data application. These applications are disabled by default. To enable it, add the following flag to the

Helm install command:

nai-core
--set "naiLabs.enabled=true"

Enable HTTPS with a self-signed certificate. By default, NAI does not provision a TLS certificate for the ingress gateway. To quickly enable HTTPS with a self-signed certificate (recommended for dev/test environments), add the following flag to the

Helm install command:

nai-core
--set "gateway.certManager.selfSigned=true"

This requires

to be installed on the cluster. For production TLS options, including

cert-manager

TLS Encryption on Nutanix

using your own certificate or a cert-manager ClusterIssuer, see

Enterprise AI

.

Scale out the ingress gateway and NAI API. To increase replicas for the ingress gateway and the NAI API, add the following flags to the

Helm install command:

nai-core
--set "gateway.replicaCount=<Number_of_replicas>"
--set "naiApi.replicaCount=<Number_of_replicas>"

The default is 1 replica each. Increase based on your scale requirements.

Configure PostgreSQL database connections. To adjust the maximum number of concurrent PostgreSQL connections, add the following flag to the

Helm install command:

nai-core
--set "naiDatabase.postgresConfig.maxConnections=<Number_of_Connections>"

The default value is 1000. Increase this value if you expect a higher number of concurrent clients.

8. Configure the TLS certificate.

For more information, see

TLS Encryption on Nutanix Enterprise AI

on page 119.

What to do next

Verify that the

and

Helm releases have

.

nai-operators
nai-core
STATUS=deployed

Verify that all expected pods are Ready.

Verify that persistent volumes are Bound and use the intended storage classes.

Verify that the selected profile produced the expected replica counts for critical components.

Verify that NAI services are reachable.

Pending

Failed

If you had endpoints in

status before the upgrade displaying the status as

with the message

Unable to pull runtime image with provided credentials, hibernate and resume the endpoints.

Access NAI Dashboard IP:

kubectl get svc -n envoy-gateway-system -l "gateway.envoyproxy.io/owning-gateway-
name=nai-ingress-gateway,gateway.envoyproxy.io/owning-gateway-namespace=nai-system" -
o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}'

For information on viewing the

Dashboard

, see

Log in to Nutanix Enterprise AI

on page 167.

Deploying Nutanix Enterprise AI on a Nutanix Kubernetes Platform Cluster in Air- Gapped Environments

Install Nutanix Enterprise AI (NAI) on a Nutanix Kubernetes Platform (NKP) cluster in an air-gapped environment.

Before you begin

NAI supports air-gapped installation only on NKP.

Ensure that you meet requirements listed in

Prerequisites for Deploying Nutanix Enterprise AI 2.8.0 on a

Nutanix Kubernetes® Platform Cluster in Air-Gapped Environments

on page 52.

About this task

To install NAI on an NKP cluster in an air-gapped environment, follow these high-level steps:

Procedure

Download the NAI 2.8.0 release bundles from the Nutanix portal.

Downloading Nutanix Enterprise AI Air-Gap Release Bundles

For more information, see

on page 52.

Push Container Images to a private registry.

From this step onwards, execute the all commands from within your air-gapped environment typically a jumpbox or bastion host that has access to both your container registry and your Kubernetes cluster. Ensure that you have Docker and kubectl available on this machine before proceeding.

For more information, see

Pushing Container Images to Private Registry in Air-Gapped Environments

on

page 53

Extract the NAI Helm charts bundle to your working directory:

tar -xvf nai-helm-charts-2.8.0.tar

The extraction produces the following Helm chart archives:

gateway-crds-helm-v1.8.1.tgz
gateway-helm-v1.8.1.tgz
kserve-crd-v0.19.0.tgz
kserve-llmisvc-crd-v0.19.0.tgz
kserve-llmisvc-resources-v0.19.0.tgz
kserve-resources-v0.19.0.tgz
lws-0.8.0.tgz
nai-core-2.8.0.tgz
nai-operators-2.8.0.tgz
opentelemetry-operator-0.114.1.tgz

(Optional) Publish Helm Charts to Private Registry

To install Helm charts directly from your OCI-compatible registry instead of local files, push the charts using the following commands:

# Authenticate to your registry
helm registry login -u <username> -p <password> https://<registry>
# Push each chart to the registry
helm push gateway-crds-helm-v1.8.1.tgz oci://<registry>
helm push gateway-helm-v1.8.1.tgz oci://<registry>
helm push kserve-crd-v0.19.0.tgz oci://<registry>
helm push kserve-resources-v0.19.0.tgz oci://<registry>
helm push kserve-llmisvc-crd-v0.19.0.tgz oci://<registry>
helm push kserve-llmisvc-resources-v0.19.0.tgz oci://<registry>
helm push opentelemetry-operator-0.114.1.tgz oci://<registry>
helm push lws-0.8.0.tgz oci://<registry>
helm push nai-core-2.8.0.tgz oci://<registry>
helm push nai-operators-2.8.0.tgz oci://<registry>

The remaining steps assume installation from local Helm chart files.

Configure Registry Credentials

For more information, see

Configuring Docker Registry Credentials for Dependencies

on page 58.

Install Prometheus Monitoring from the Nutanix Kubernetes Platform (NKP) platform applications catalog. Prometheus Monitoring is not included in the Nutanix Enterprise AI air-gap image tar or the helm-charts tar. On an NKP cluster in an air-gapped environment, install Prometheus Monitoring from the NKP platform applications catalog before you install Nutanix Enterprise AI components. Enable the Prometheus Monitoring platform application on your NKP workload cluster.

To optimize resource utilization on the workload cluster, configure Prometheus Monitoring with the following minimum installation settings when you enable the application:

alertmanager:
enabled: false
grafana:
enabled: false
prometheus:
enabled: false
kubeStateMetrics:
enabled: false
kubernetesServiceMonitors:
enabled: false
prometheus-node-exporter.kubeRBACProxy:
kubeRBACProxy:
enabled: true

Pro: Enabling an Application Using the UI

For more information, see

.

Install NVIDIA GPU Operator from NKP platform applications.

If the GPU nodes do not have precompiled NVIDIA drivers installed, enable driver installation in the NVIDIA GPU Operator configuration. This setting ensures that the NVIDIA drivers are installed on the GPU nodes. When enabling the NVIDIA GPU Operator, add the following cluster override:

driver:
enabled: true

Pro: Enabling an Application Using the UI

For more information, see

.

Install CloudNativePG Operator:

helm install cnpg cloudnative-pg \
--repo https://cloudnative-pg.github.io/charts \
--version 0.28.0 -n cnpg-system --create-namespace --wait

Install LeaderWorkerSet:

helm install lws oci://registry.k8s.io/lws/charts/lws \
--version 0.8.0 -n lws-system --create-namespace --wait

10. Install Envoy Gateway.

For more information, see

Installing Envoy Gateway in an Air-gapped Environment

on page 59.

11. Install KServe.

Installing KServe

For more information, see

on page 61.

12. Deploy the OpenTelemetry Operator.

For more information, see

Deploying the OpenTelemetry Operator

on page 61.

13. Install NAI components.

Deploying Nutanix Enterprise AI Components in Air-Gapped Environments

For more information, see

on

page 62.

What to do next

1. Verify NAI operators installation. Check the status of NAI components:

# Check all pods in nai-system namespace
kubectl get pods -n nai-system
# Check NAI Operators and Core Helm release
helm list -n nai-system
# Check persistent volume claims
kubectl get pvc -n nai-system

2. Verify if NAI services are accessible:

# List all services in nai-system
kubectl get svc -n nai-system
# Check NAI API service
kubectl get svc -n nai-system nai-api
# Check NAI UI service
kubectl get svc -n nai-system nai-inference-ui

3. Troubleshoot common issues that can occur when you install NAI 2.8 in air-gapped environments.

For more information, see

Troubleshooting Deployment of Nutanix Enterprise AI 2.8.0 in Air-Gapped

Environments

on page 66.

Prerequisites for Deploying Nutanix Enterprise AI 2.8.0 on a Nutanix Kubernetes® Platform Cluster in Air-Gapped Environments Before proceeding with the installation, ensure the following requirements are met:

Nutanix Kubernetes Platform (NKP) cluster running Kubernetes 1.35.

kubectl CLI v1.33+ configured with cluster access

Helm CLI v4.0.5

Docker or compatible container runtime (for loading images)

Access to a private container registry

Registry credentials with appropriate permissions

Sufficient disk space for loading images (~100GB)

Downloading Nutanix Enterprise AI Air-Gap Release Bundles Download the required Nutanix Enterprise AI 2.8.0 release bundles from the Nutanix Portal.

About this task

To download the required NAI 2.8.0 release bundles from the Nutanix Portal, follow these steps:

Procedure

1. Navigate to the Nutanix Enterprise AI page on the Nutanix Support Portal

2. Select NAI Version 2.8.0 from the available releases

3. Download the following two bundles:

NAI Air-Gap Bundle (

)

nai-v2.8.0.tar

NAI Helm Charts Bundle (

)

nai-helm-charts-2.8.0.tar

Table 35: Air-Gap Download Bundles

Contains all NAI container images and dependencies

Required for air-gapped deployments

NAI Helm Charts Bundle (nai- helm-charts-2.8.0.tar)

~5 MB

Contains NAI Helm charts and all dependency charts

Includes Envoy Gateway, KServe, LeaderWorkerSet and OpenTelemetry charts

4. Transfer both bundles to your air-gapped environment using approved methods such as USB drive, secure file

transfer, and so on.

Pushing Container Images to Private Registry in Air-Gapped Environments Before installing NAI, you must load all container images and push them to your private registry.

Before you begin

Execute all the commands from within your air-gapped environment typically a jumpbox or bastion host that has access to both your container registry and your Kubernetes cluster. Ensure you have Docker and kubectl available on this machine before proceeding.

About this task

To push container images to a private registry, follow these steps:

Procedure

1. Login to your private container registry:

docker login <registry-url>

For example,

docker login registry.example.com

The system displays a prompt to enter your registry credentials.

2. Enter your registry credentials when prompted.
3. Create the Image Push Script.

The Image Push Script pushes container images to your private registry.

a. Create a project/repository with the name

in your container registry, where all NAI images are

nutanix

stored. For example, in Harbor this would be a project named

, resulting in image paths like

nutanix

.

registry.example.com/nutanix/<image-name>:<tag>

b. Create a script file named

with the following content:

push-images-to-registry.sh
#!/bin/bash
#
# NAI Images - Load, Retag, and Push to Private Registry
#
# This script loads NAI container images from a tar bundle, retags them for your
# private registry, and pushes them to the registry.
#
# Prerequisites:
#   - Docker installed and running
#   - Docker logged into the target registry (docker login)
#   - NAI images tar bundle file
#
# Usage:
#   ./push-images-to-registry.sh <registry-url> <project> <tar-file>
#
# Example:
#   ./push-images-to-registry.sh registry.example.com nutanix nai-images-2.8.0.tar
#
set -uo pipefail
# ============================================================================
# Helper Functions
# ============================================================================
print_header() {
echo ""
echo "========================================"
echo "$1"
echo "========================================"
}
print_success() {
echo "# $1"
}
print_error() {
echo "# ERROR: $1" >&2
}
print_info() {
echo "# $1"
}
# ============================================================================
# Validate Arguments
# ============================================================================
if [ $# -ne 3 ]; then
echo "Usage: $0 <registry-url> <project> <tar-file>"
echo ""
echo "Arguments:"
echo "  registry-url    Your private registry URL (e.g.,
registry.example.com)"
echo "  project         Project/repository name in the registry (e.g.,
nutanix)"
echo "  tar-file        Path to the NAI images tar bundle"
echo ""
echo "Example:"
echo "  $0 registry.example.com nutanix nai-images-2.8.0.tar"
echo ""
exit 1
fi
REGISTRY="$1"
PROJECT="$2"
TAR_FILE="$3"
# Validate tar file exists
if [ ! -f "$TAR_FILE" ]; then
print_error "Tar file not found: $TAR_FILE"
exit 1
fi
# ============================================================================
# Configuration
# ============================================================================
print_header "NAI Images - Load, Retag & Push"
echo "Registry:  $REGISTRY"
echo "Project:   $PROJECT"
echo "Tar File:  $TAR_FILE"
echo "Date:      $(date)"
# Arrays to track images
LOADED_IMAGES=()
FAILED_IMAGES=()
# ============================================================================
# Step 1: Load Images from Tar Bundle
# ============================================================================
print_header "Step 1: Loading Images from Tar Bundle"
print_info "Loading images from $TAR_FILE..."
LOAD_OUTPUT=$(docker load -i "$TAR_FILE" 2>&1)
# Extract loaded image names
while IFS= read -r line; do
if [[ "$line" =~ Loaded\ image:\ (.+)$ ]]; then
LOADED_IMAGES+=("${BASH_REMATCH[1]}")
fi
done <<< "$LOAD_OUTPUT"
if [ ${#LOADED_IMAGES[@]} -eq 0 ]; then
print_error "No images were loaded from the tar file"
exit 1
fi
print_success "Loaded ${#LOADED_IMAGES[@]} images"
# ============================================================================
# Step 2: Retag and Push Images
# ============================================================================
print_header "Step 2: Retagging and Pushing Images"
PUSHED_COUNT=0
TOTAL_IMAGES=${#LOADED_IMAGES[@]}
for source_image in "${LOADED_IMAGES[@]}"; do
echo ""
print_info "[$((PUSHED_COUNT + 1))/$TOTAL_IMAGES] Processing: $source_image"
# Retag image for target registry
# Format: nutanix/nai-api:v2.8.0 # registry.example.com/<project>/nai-
api:v2.8.0
if [[ "$source_image" =~ ^nutanix/(.+)$ ]]; then
image_path="${BASH_REMATCH[1]}"
target_image="${REGISTRY}/${PROJECT}/${image_path}"
print_info "Tagging as: $target_image"
if ! docker tag "$source_image" "$target_image"; then
print_error "Failed to tag image"
FAILED_IMAGES+=("$source_image")
continue
fi
print_info "Pushing to registry..."
if docker push "$target_image"; then
print_success "Pushed successfully"
((PUSHED_COUNT++))
else
print_error "Failed to push image"
FAILED_IMAGES+=("$target_image")
fi
else
print_info "Skipping (not in nutanix/* format)"
fi
done
# ============================================================================
# Summary
# ============================================================================
print_header "Summary"
echo "Total images loaded:    $TOTAL_IMAGES"
echo "Successfully pushed:    $PUSHED_COUNT"
echo "Failed:                 ${#FAILED_IMAGES[@]}"
if [ ${#FAILED_IMAGES[@]} -gt 0 ]; then
echo ""
print_error "The following images failed:"
for img in "${FAILED_IMAGES[@]}"; do
echo "  - $img"
done
echo ""
exit 1
fi
echo ""
print_success "All images successfully pushed to $REGISTRY/$PROJECT"
echo ""
exit 0

c. Make the script executable:

chmod +x push-images-to-registry.sh

d. Execute the script to load, retag, and push all NAI images.

./push-images-to-registry.sh <registry-url> <project> nai-v2.8.0.tar

Example:

./push-images-to-registry.sh registry.example.com nutanix nai-v2.8.0.tar

Expected Output:

========================================
NAI Images - Load, Retag & Push
========================================
Registry:  registry.example.com
Project:   nutanix
Tar File:  ./nai-v2.8.0.tar
Date:      Tue Aug 18 05:16:33 PM UTC 2026
========================================
Step 1: Loading Images from Tar Bundle
========================================
# Loading images from ./nai-v2.8.0.tar...
# Loaded 41 images
========================================
Step 2: Retagging and Pushing Images
========================================
# [1/41] Processing: nutanix/nai-iam-proxy-control-plane:v2.8.0
# Tagging as: registry.example.com/nutanix/nai-iam-proxy-control-plane:v2.8.0
# Pushing to registry...
The push refers to repository [registry.example.com/nutanix/nai-iam-proxy-control-
plane]
054a97ddb80b: Pushed
5228eaa6af5b: Pushed
256f393e029f: Mounted from nutanix/nai-inference-ui
v2.8.0: digest:
sha256:587189a6559b7af653769a21f4755c66af14fe5135eaf45728c84a2eb2f3c808 size: 951
# Pushed successfully
# [2/41] Processing: nutanix/nai-iam-ui:v2.8.0
# Tagging as: registry.example.com/nutanix/nai-iam-ui:v2.8.0
# Pushing to registry...
The push refers to repository [registry.example.com/nutanix/nai-iam-ui]
7673a750ed47: Pushed
187de06a3fb0: Pushed
[... continues for all images ...]
========================================
Summary
========================================
Total images loaded:    41
Successfully pushed:    41
Failed:                 0
# All images successfully pushed to registry.example.com/nutanix

The image push process typically takes 30-60 minutes depending on your network speed and registry performance.

All images are retagged with your registry URL while preserving the original image path and tag

Original format: nutanix/nai-api:v2.8.0

Retagged format: /nutanix/nai-api:v2.8.0

The script reports failures and continues processing the remaining images.

You can safely re-run the script if it fails partway through.

Configuring Docker Registry Credentials for Dependencies Configure registry credentials.

About this task

To configure registry credentials, follow these steps:

Procedure

1. Set environment Variables.

Export the following environment variables with your private registry credentials:

export REGISTRY=<registry-url-without-https>
export REGISTRY_USERNAME='<registry-username>'
export REGISTRY_PASSWORD='<registry-password>'
export REGISTRY_EMAIL='<registry-email>'
export IMAGE_PULL_SECRET=nai-docker-regcred
export PROJECT=nutanix # set the registry project name

2. Replace the placeholder values with your actual registry information.

The

must not include the

protocol prefix.

REGISTRY
https://

3. Create Image Pull Secrets.

Create Kubernetes namespaces and docker-registry secrets for Envoy Gateway System :

kubectl create namespace envoy-gateway-system --dry-run=client -o yaml | kubectl
apply -f -
kubectl create secret docker-registry ${IMAGE_PULL_SECRET} \
--docker-server=${REGISTRY} \
--docker-username=${REGISTRY_USERNAME} \
--docker-password=${REGISTRY_PASSWORD} \
--docker-email=${REGISTRY_EMAIL} \
-n envoy-gateway-system \
--dry-run=client -o yaml | kubectl apply -f -

4. Create Kubernetes namespaces and

secrets for KServe:

docker-registry
kubectl create namespace kserve --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret docker-registry ${IMAGE_PULL_SECRET} \
--docker-server=${REGISTRY} \
--docker-username=${REGISTRY_USERNAME} \
--docker-password=${REGISTRY_PASSWORD} \
--docker-email=${REGISTRY_EMAIL} \
-n kserve \
--dry-run=client -o yaml | kubectl apply -f -

5. Create Kubernetes namespaces and docker-registry secrets for OpenTelemetry:

kubectl create namespace opentelemetry --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret docker-registry ${IMAGE_PULL_SECRET} \
--docker-server=${REGISTRY} \
--docker-username=${REGISTRY_USERNAME} \
--docker-password=${REGISTRY_PASSWORD} \
--docker-email=${REGISTRY_EMAIL} \
-n opentelemetry \
--dry-run=client -o yaml | kubectl apply -f -

6. Create Kubernetes namespaces and docker-registry secrets for LeaderWorkerSet:

kubectl create namespace lws-system --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret docker-registry ${IMAGE_PULL_SECRET} \
--docker-server=${REGISTRY} \
--docker-username=${REGISTRY_USERNAME} \
--docker-password=${REGISTRY_PASSWORD} \
--docker-email=${REGISTRY_EMAIL} \
-n lws-system \
--dry-run=client -o yaml | kubectl apply -f -

7. Create Kubernetes namespaces and docker-registry secrets for nai-system:

kubectl create namespace nai-system --dry-run=client -o yaml | kubectl apply -f -
kubectl create secret docker-registry ${IMAGE_PULL_SECRET} \
--docker-server=${REGISTRY} \
--docker-username=${REGISTRY_USERNAME} \
--docker-password=${REGISTRY_PASSWORD} \
--docker-email=${REGISTRY_EMAIL} \
-n nai-system \
--dry-run=client -o yaml | kubectl apply -f -

Installing Envoy Gateway in an Air-gapped Environment Install Envoy Gateway.

Before you begin

Ensure that you meet requirements listed in

Prerequisites for Deploying Nutanix Enterprise AI 2.8.0 on a

Nutanix Kubernetes® Platform Cluster in Air-Gapped Environments

on page 52.

About this task

To install Envoy Gateway, follow these steps:

Procedure

1. Install the Envoy Gateway and Gateway API Custom Resource Definitions (CRDs):

helm template eg ./gateway-crds-helm-v1.8.1.tgz \
--set crds.gatewayAPI.enabled=true \
--set crds.envoyGateway.enabled=true \
| kubectl apply --server-side --force-conflicts -f -

This command also installs the necessary Gateway API CRDs required for Envoy Gateway.

2. Create the configuration template file

:

eg-config-for-gateway-mode.yaml.template
# This file configures Envoy Gateway for AI Gateway mode with rate limiting
config:
envoyGateway:
gateway:
controllerName: "gateway.envoyproxy.io/gatewayclass-controller"
logging:
level:
default: "info"
provider:
kubernetes:
rateLimitDeployment:
patch:
type: "StrategicMerge"
value:
spec:
template:
spec:
containers:
- imagePullPolicy: "IfNotPresent"
name: "envoy-ratelimit"
env:
- name: REDIS_TYPE
value: "sentinel"
- name: REDIS_PIPELINE_WINDOW
value: "150us"
type: "Kubernetes"
extensionApis:
enableEnvoyPatchPolicy: true
enableBackend: true
extensionManager:
maxMessageSize: 11Mi
backendResources:
- group: inference.networking.k8s.io
kind: InferencePool
version: v1
hooks:
xdsTranslator:
translation:
listener:
includeAll: true
route:
includeAll: true
cluster:
includeAll: true
secret:
includeAll: true
post:
- "Translation"
- "Cluster"
- "Route"
service:
fqdn:
hostname: "ai-gateway-controller.nai-system.svc.cluster.local"
port: 1063
rateLimit:
backend:
type: "Redis"
redis:
url: "mymaster,nai-valkey-sentinel.nai-system.svc.cluster.local:26379"

3. Ensure the REGISTRY environment variable is configured.

4. Deploy Envoy Gateway:

helm upgrade --install eg ./gateway-helm-v1.8.1.tgz \
-n envoy-gateway-system --create-namespace --wait \
--set global.images.envoyGateway.image=${REGISTRY}/${PROJECT}/nai-gateway:v1.8.1 \
--set global.images.ratelimit.image=${REGISTRY}/${PROJECT}/nai-ratelimit:1e50889b \
--set "global.imagePullSecrets[0].name=${IMAGE_PULL_SECRET}" \
-f ./eg-config-for-gateway-mode.yaml

The configuration file now uses your private registry for the

images through the

ratelimit
${REGISTRY}

variable substitution.

Installing KServe Install KServe.

About this task

To install KServe, follow these steps:

Procedure

1. Install the KServe Custom Resource Definitions (CRDs:)

helm upgrade --install kserve-crd ./kserve-crd-v0.19.0.tgz -n kserve --create-
namespace --wait

2. Deploy the KServe controller with RawDeployment mode:

helm upgrade --install kserve ./kserve-resources-v0.19.0.tgz \
-n kserve --wait \
--set kserve.controller.deploymentMode=RawDeployment \
--set kserve.controller.gateway.disableIngressCreation=true \
--set kserve.controller.image=${REGISTRY}/${PROJECT}/nai-kserve-controller \
--set kserve.controller.rbacProxyImage=${REGISTRY}/${PROJECT}/nai-kube-rbac-
proxy:v0.18.0 \
--set "kserve.controller.imagePullSecrets[0].name=${IMAGE_PULL_SECRET}"

3. Deploy the KServe llmisvc crds:

helm upgrade --install kserve-llmisvc-crd ./kserve-llmisvc-crd-v0.19.0.tgz -n kserve
--create-namespace --wait

4. Deploy the KServe llmisvc controller:

helm upgrade --install kserve-llmisvc-resources ./kserve-llmisvc-resources-
v0.19.0.tgz \
-n kserve --create-namespace --wait --set kserve.createSharedResources=false --set
kserve.llmisvc.createGIECRDs=false \
--set kserve.llmisvc.controller.image=${REGISTRY}/${PROJECT}/nai-llmisvc-controller
\
--set "kserve.llmisvc.controller.imagePullSecrets[0]=${IMAGE_PULL_SECRET}"

Deploying the OpenTelemetry Operator Deploy the OpenTelemetry Operator for observability and telemetry collection.

About this task

To deploy the OpenTelemetry Operator, run the following command:

Procedure

Deploy the OpenTelemetry Operator for observability and telemetry collection:

helm upgrade --install opentelemetry-operator ./opentelemetry-operator-0.114.1.tgz \
-n opentelemetry --create-namespace --wait \
--set manager.image.repository=${REGISTRY}/${PROJECT}/nai-opentelemetry-operator \
--set manager.collectorImage.repository=${REGISTRY}/${PROJECT}/nai-opentelemetry-
collector-contrib \
--set "imagePullSecrets[0].name=${IMAGE_PULL_SECRET}"

Deploying Nutanix Enterprise AI Components in Air-Gapped Environments Deploy Nutanix Enterprise AI components in air-gapped environments.

Before you begin

Upgrades from version 2.7.0 to 2.8.0 must be executed during planned downtime. User logins will be unavailable during the upgrade, and full functionality will resume automatically once the upgrade is complete.

About this task

To deploy Nutanix Enterprise AI components in air-gapped environments, follow these steps:

Procedure

using the

1. Create a values override file for NAI Operators named

darksite-nai-operators.yaml.template

following template:

This file configures all operator images to use your private registry.

global:
imagePullSecrets:
- name: ${IMAGE_PULL_SECRET}
storage:
storageClassName: ${NAI_DEFAULT_RWO_STORAGECLASS}
naiValkey:
image:
name: ${REGISTRY}/${PROJECT}/nai-valkey
naiJobs:
naiJobsImage:
image: ${REGISTRY}/${PROJECT}/nai-jobs
nai-clickhouse-operator:
operator:
image:
registry: ${REGISTRY}
repository: ${PROJECT}/nai-clickhouse-operator
metrics:
image:
registry: ${REGISTRY}
repository: ${PROJECT}/nai-clickhouse-metrics-exporter
ai-gateway-helm:
extProc:
image:
repository: ${REGISTRY}/${PROJECT}/nai-ai-gateway-extproc
controller:
image:
repository: ${REGISTRY}/${PROJECT}/nai-ai-gateway-controller
naiDatabase:
image: ${REGISTRY}/${PROJECT}/nai-postgresql:17.10-standard-trixie

2. Generate the actual values file:

Use

to replace environment variables and create the final values file:

envsubst
envsubst < darksite-nai-operators.yaml.template > darksite-nai-operators.yaml

The

command substitutes

,

and

with the actual

envsubst
${REGISTRY}
${PROJECT}
${IMAGE_PULL_SECRET}

values from your environment variables.

3. Install NAI Operators:

helm upgrade --install nai-operators ./nai-operators-2.8.0.tgz \
-n nai-system --create-namespace --wait --timeout 15m -f ./darksite-nai-
operators.yaml

4. Prepare NAI Core Values Override file:

Create a values override file for NAI Core using the provided template. This configures all NAI core component images.

a. Create the template file named

with the following content:

darksite-nai-core.yaml.template
global:
imagePullSecrets:
- name: ${IMAGE_PULL_SECRET}
storage:
storageClassName: ${NAI_DEFAULT_RWO_STORAGECLASS}
storageClassNameRWX: ${NAI_API_RWX_STORAGECLASS}
gateway:
envoyDeployment:
container:
image: ${REGISTRY}/${PROJECT}/nai-envoy:distroless-v1.38.0
naiIepOperator:
iepOperatorImage:
image: ${REGISTRY}/${PROJECT}/nai-iep-operator
modelProcessorImage:
image: ${REGISTRY}/${PROJECT}/nai-python-processor
dataSourceProcessorImage:
image: ${REGISTRY}/${PROJECT}/nai-python-processor
batchInferenceProcessor:
containers:
processor:
image: ${REGISTRY}/${PROJECT}/nai-go-processor
statusProvider:
image: ${REGISTRY}/${PROJECT}/nai-go-processor
finetuneProcessor:
containers:
processor:
image: ${REGISTRY}/${PROJECT}/nai-finetuning
statusProvider:
image: ${REGISTRY}/${PROJECT}/nai-go-processor
naiInferenceUi:
naiUiImage:
image: ${REGISTRY}/${PROJECT}/nai-inference-ui
naiJobs:
naiJobsImage:
image: ${REGISTRY}/${PROJECT}/nai-jobs
naiApi:
naiApiImage:
image: ${REGISTRY}/${PROJECT}/nai-api
supportedTGIImage: ${REGISTRY}/${PROJECT}/nai-tgi
supportedKserveRuntimeImage: ${REGISTRY}/${PROJECT}/nai-kserve-huggingfaceserver
eppImage: ${REGISTRY}/${PROJECT}/nai-epp-inference-scheduler
supportedVLLMImage: ${REGISTRY}/${PROJECT}/nai-vllm
supportedKserveCustomModelServerRuntimeImage: ${REGISTRY}/${PROJECT}/nai-kserve-
custom-model-server
naiDatabase:
clientImage: ${REGISTRY}/${PROJECT}/nai-postgresql:17.10-standard-trixie
naiIam:
iamProxy:
image: ${REGISTRY}/${PROJECT}/nai-iam-proxy
iamProxyControlPlane:
image: ${REGISTRY}/${PROJECT}/nai-iam-proxy-control-plane
iamUi:
image: ${REGISTRY}/${PROJECT}/nai-iam-ui
iamUserAuthn:
image: ${REGISTRY}/${PROJECT}/nai-iam-user-authn
iamThemis:
image: ${REGISTRY}/${PROJECT}/nai-iam-themis
iamThemisBootstrap:
image: ${REGISTRY}/${PROJECT}/nai-iam-bootstrap
naiAgent:
agentImage:
image: ${REGISTRY}/${PROJECT}/nai-agent-app
naiLabs:
labsImage:
image: ${REGISTRY}/${PROJECT}/nai-rag-app
nai-clickhouse-keeper:
clickhouseKeeper:
image:
registry: ${REGISTRY}
repository: ${PROJECT}/nai-clickhouse-keeper
oauth2-proxy:
image:
repository: ${REGISTRY}/${PROJECT}/nai-oauth2-proxy
nai-clickhouse-server:
clickhouse:
image:
registry: ${REGISTRY}
repository: ${PROJECT}/nai-clickhouse-server
initContainers:
addUdf:
image:
registry: ${REGISTRY}
repository: ${PROJECT}/nai-clickhouse-udf
waitForKeeper:
image:
registry: ${REGISTRY}
repository: ${PROJECT}/nai-jobs
nai-clickhouse-schemas:
image:
registry: ${REGISTRY}
repository: ${PROJECT}/nai-clickhouse-schemas
naiMonitoring:
opentelemetry:
collectorImage: ${REGISTRY}/${PROJECT}/nai-opentelemetry-collector-
contrib:0.152.0
targetAllocator:
image:
repository: ${REGISTRY}/${PROJECT}/nai-target-allocator
nodeExporter:
serviceMonitor:
namespaceSelector:
matchNames:
- prometheus
- kommander
- kommander-default-workspace
- ${NKP_WORKSPACE_NAMESPACE}
dcgmExporter:
serviceMonitor:
namespaceSelector:
matchNames:
- prometheus
- kommander
- kommander-default-workspace
- ${NKP_WORKSPACE_NAMESPACE}

b. Ensure that

,

,

,

REGISTRY
IMAGE_PULL_SECRET
NAI_API_RWX_STORAGECLASS

, and

environment variables are set before

NAI_DEFAULT_RWO_STORAGECLASS
NKP_WORKSPACE_NAMESPACE

running the next command.

c. Generate the actual values file:

Use

to replace environment variables and create the final values file:

envsubst
envsubst < darksite-nai-core.yaml.template > darksite-nai-core.yaml

5. Install NAI Core:

helm upgrade --install nai-core ./nai-core-2.8.0.tgz -n nai-system --create-namespace
--wait --timeout 15m \
-f ./darksite-nai-core.yaml

All configuration including registry URLs, storage classes, and monitoring namespaces are in the darksite-nai- core.yaml values file, making the install command much simpler.

The NAI Core installation may take 10 to15 minutes depending on your cluster resources and network speed.

[!NOTE] Note: You can append optional Helm overrides to the nai-core installation command to customize the deployment

The Chat and Talk to My Data applications are disabled by default. To enable it, add the following flag to the

Helm install command:

nai-core
--set "naiLabs.enabled=true"

By default, NAI does not provision a TLS certificate for the ingress gateway. To quickly enable HTTPS with a self-signed certificate (recommended for dev/test environments), add the following flag to the

Helm install command:

nai-core
--set "gateway.certManager.selfSigned=true"

This requires

to be installed on the cluster. For production TLS options, including

cert-manager

using your own certificate or a cert-manager ClusterIssuer, see

TLS Encryption on Nutanix

Enterprise AI

.

Scale out: To increase replicas for the ingress gateway and the NAI API, add the following flags to the

Helm install command:

nai-core
--set "gateway.replicaCount=<Number_of_replicas>"
--set "naiApi.replicaCount=<Number_of_replicas>"

The default is 1 replica each. Increase based on your scale requirements.

Database Connections: To adjust the maximum number of concurrent PostgreSQL connections, add the following flag to the

Helm install command:

nai-core
--set "naiDatabase.postgresConfig.maxConnections=<Number_of_Connections>"

The default is 1000. Increase if you expect a higher number of concurrent clients.

6. Configure the TLS certificate.

TLS Encryption on Nutanix Enterprise AI

For more information, see

on page 119.

Troubleshooting Deployment of Nutanix Enterprise AI 2.8.0 in Air-Gapped Environments The following troubleshooting tips can help you resolve a few common issues that can occur when you are installing

Image Pull Errors

Verify that:

Registry credentials are correct

Image pull secrets exist in the correct namespaces

Registry URL is accessible from the cluster

Image paths match your registry structure

All required images are present in your private registry

CRD Installation Failures

Ensure you have sufficient permissions to create cluster-scoped resources.

Helm Installation Timeouts

Increase the timeout value using –timeout flag (e.g., –timeout 20m).

Storage Class Issues

Verify the storage class exists: kubectl get storageclass

Ensure the storage class supports the required access mode (RWX for NAI API, RWO for others)

Check PVC status: kubectl get pvc -n nai-system

Pod Startup Failures

Check pod logs: kubectl logs -n nai-system

Describe pod for events: kubectl describe pod -n nai-system

Verify resource limits if running on resource-constrained clusters

Dependency Issues

Ensure all dependencies (Envoy Gateway, KServe, OpenTelemetry) are installed and running before installing NAI Core.

Deploy Nutanix Enterprise AI on Amazon Elastic Kubernetes Service

Install or upgrade Nutanix Enterprise AI (NAI) on Amazon Elastic Kubernetes Service (EKS).

To install or upgrade NAI on EKS, follow these high-level steps:

1. Set up an EKS cluster. For more information, see

Setting up an Amazon Elastic Kubernetes Service Cluster

on page 67.

2. Perform preflight checks. For more information, see

Performing Preflight Checks Before Installing NAI on

Amazon EKS

on page 68.

Installing Prerequisite Components on an EKS

3. Install prerequisite components. For more information, see

Cluster

on page 69.

4. Deploy NAI on EKS. For more information, see

Deploying Nutanix Enterprise AI on Amazon Elastic

Kubernetes Service

on page 72.

Setting up an Amazon Elastic Kubernetes Service Cluster Set up your Amazon Elastic Kubernetes Service (EKS) cluster for Nutanix Enterprise AI. Do not follow this procedure for Nutanix Kubernetes Platform (NKP) managed or attached EKS clusters.

Before you begin

Ensure that your cluster meets all the requirements mentioned in

Nutanix Enterprise AI - Private Inference

and Agent Gateway Requirements

on page 10.

About this task

To set up your EKS cluster before you deploy Nutanix Enterprise AI, follow these steps:

Procedure

Configure the AWS CLI.

AWS documentation

For more information, see

.

Configure an Amazon virtual private cloud (VPC) with public and private subnets for EKS using CloudFormation.

For more information, see

AWS documentation

.

Create an Amazon EKS cluster with Kubernetes version 1.33 or later.

For more information, see

AWS documentation

.

Create a kubeconfig file to connect kubectl to the EKS cluster.

AWS documentation

For more information, see

.

Create an EKS role for the EKS cluster.

For more information, see

AWS documentation

.

Create an Elastic Compute Cloud (EC2) role for the EKS cluster.

AWS documentation

For more information, see

.

Create a default node group. Make sure the nodes have accelerator AVX2 or newer.

The recommended instance type is from instance family C5.

Create a GPU node group on the EKS Cluster.

For more information, see

AWS documentation

.

Configure the Amazon EBS CSI driver and the RWO storageclass.

AWS documentation

For more information, see

.

10. Configure the Amazon EFS CSI driver and the RWX storageclass.

For more information, see

AWS documentation

.

11. Configure the Amazon VPC CNI plugin to enable network policy enforcement.

AWS documentation

For more information, see

.

Performing Preflight Checks Before Installing NAI on Amazon EKS Perform preflight checks before installing NAI on Amazon Elastic Kubernetes Service (EKS). Preflight checks ensure the Kubernetes environment meets all the technical requirements for a successful NAI installation.

About this task

To perform preflight checks for installing NAI on EKS, follow these steps:

Procedure

1. Make sure the Kubernetes version on your worker nodes is compatible:

kubectl get nodes --selector='!node-role.kubernetes.io/control-plane' \
-o custom-
columns=NAME:.metadata.name,KUBELET_VERSION:.status.nodeInfo.kubeletVersion \
--no-headers

The expected output is that the Kubernetes version must be 1.33 or later.

2. Verify if the number of EFS CSI pods match the number of worker nodes:

[ $(kubectl get nodes --no-headers | wc -l) -eq $(kubectl get pods -n kube-system --
no-headers | grep efs-csi-node | wc -l) ] && echo "# efs-csi-node pods = node count"
|| echo "# Mismatch: efs-csi-node pods != node count"

The expected output is

.

efs-csi-node pods= node count

3. Confirm that the

class binds volumes immediately:

nai-nfs-storage
kubectl get storageclass nai-nfs-storage -o jsonpath='{.volumeBindingMode}{"\n"}'

The expected output is

.

immediate

4. Check if the

storage class has

access:

nai-nfs-storage
ReadWriteMany
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-rwx-pvc
namespace: default
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
storageClassName: nai-nfs-storage
EOF
kubectl get pvc test-rwx-pvc -n default

The expected output is that the storage class

has

set to

.

nai-nfs-storage
Access Modes
RWX
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS
MODES   STORAGECLASS      VOLUMEATTRIBUTESCLASS   AGE
test-rwx-pvc            Bound    pvc-0df5145a-590c-4f28-9f9d-cefefc55266c   1Gi
RWX            nai-nfs-storage   <unset>                 5s
kubectl delete pvc test-rwx-pvc

Installing Prerequisite Components on an EKS Cluster Install components on an EKS cluster.

About this task

To install components on an EKS cluster, follow these steps:

Procedure

1. Install cert-manager:

helm upgrade --install cert-manager cert-manager --repo https://charts.jetstack.io --
version v1.19.3 --set installCRDs=true -n cert-manager --create-namespace --wait

2. Install or upgrade Envoy Gateway:

a. Install Envoy Gateway and Gateway API CRDs:

helm template eg oci://docker.io/envoyproxy/gateway-crds-helm --version v1.8.1 \
--set crds.gatewayAPI.enabled=true \
--set crds.envoyGateway.enabled=true \
| kubectl apply --server-side --force-conflicts -f -

b. Create

with the following configuration:

envoy-gateway-config.yaml
config:
envoyGateway:
gateway:
controllerName: "gateway.envoyproxy.io/gatewayclass-controller"
logging:
level:
default: "info"
provider:
kubernetes:
rateLimitDeployment:
container:
image: "docker.io/envoyproxy/ratelimit:1e50889b"
patch:
type: "StrategicMerge"
value:
spec:
template:
spec:
containers:
- imagePullPolicy: "IfNotPresent"
name: "envoy-ratelimit"
image: "docker.io/envoyproxy/ratelimit:1e50889b"
env:
- name: REDIS_TYPE
value: "sentinel"
- name: REDIS_PIPELINE_WINDOW
value: "150us"
type: "Kubernetes"
extensionApis:
enableEnvoyPatchPolicy: true
enableBackend: true
extensionManager:
maxMessageSize: 11Mi
backendResources:
- group: inference.networking.k8s.io
kind: InferencePool
version: v1
hooks:
xdsTranslator:
translation:
listener:
includeAll: true
route:
includeAll: true
cluster:
includeAll: true
secret:
includeAll: true
post:
- "Translation"
- "Cluster"
- "Route"
service:
fqdn:
hostname: "ai-gateway-controller.nai-system.svc.cluster.local"
port: 1063
rateLimit:
backend:
type: "Redis"
redis:
url: "mymaster,nai-valkey-sentinel.nai-system.svc.cluster.local:26379"

[!NOTE] Note: The rate-limit backend uses Valkey Sentinel with master name

mymaster

and the

nai-valkey-

service in

.

sentinel
nai-system

c. Install or Upgrade Envoy Gateway:

helm upgrade --install eg oci://docker.io/envoyproxy/gateway-helm --version v1.8.1
\
-n envoy-gateway-system --create-namespace --skip-crds \
-f "./envoy-gateway-config.yaml"

3. Install or Upgrade KServe:

The required version is KSERVE_VERSION=v0.19.0.

a. Install or upgrade the KServe CRDs:

helm upgrade --install kserve-crd oci://ghcr.io/kserve/charts/kserve-crd \
--version $KSERVE_VERSION -n kserve --create-namespace --wait

b. Install or upgrade the KServe resources:

helm upgrade --install kserve oci://ghcr.io/kserve/charts/kserve-resources \
--version $KSERVE_VERSION -n kserve --create-namespace --wait \
--set kserve.controller.deploymentMode=RawDeployment \
--set kserve.controller.gateway.disableIngressCreation=true

c. Install or upgrade the KServe LLMInferenceService CRD:

helm upgrade --install kserve-llmisvc-crd oci://ghcr.io/kserve/charts/kserve-
llmisvc-crd \
--version $KSERVE_VERSION -n kserve --create-namespace --wait

d. Install or upgrade the KServe LLMInferenceService resources:

helm upgrade --install kserve-llmisvc-resources oci://ghcr.io/kserve/charts/
kserve-llmisvc-resources \
--version $KSERVE_VERSION -n kserve --create-namespace --wait \
--set kserve.createSharedResources=false \
--set kserve.llmisvc.createGIECRDs=false

4. Install CloudNativePG Operator:

helm install cnpg cloudnative-pg \
--repo https://cloudnative-pg.github.io/charts \
--version 0.28.0 -n cnpg-system --create-namespace --wait

5. Install LeaderWorkerSet:

helm install lws oci://registry.k8s.io/lws/charts/lws \
--version 0.8.0 -n lws-system --create-namespace --wait

6. Install or upgrade OpenTelemetry Operator

helm upgrade --install opentelemetry-operator opentelemetry-operator \
--repo https://open-telemetry.github.io/opentelemetry-helm-charts \
--version=0.114.1 -n opentelemetry --create-namespace --wait

7. Install or upgrade Prometheus Monitoring:

helm upgrade --install prometheus kube-prometheus-stack \
--repo https://prometheus-community.github.io/helm-charts \
--version=82.13.6 -n prometheus --create-namespace --wait \
--set grafana.enabled=false \
--set prometheus.enabled=false \
--set kubeStateMetrics.enabled=false \
--set alertManager.enabled=false \
--set kubernetesServiceMonitors.enabled=false \
--set prometheus-node-exporter.kubeRBACProxy.enabled=true

8. Install or upgrade NVIDIA GPU Operator :

helm upgrade --install --wait gpu-operator gpu-operator \
--repo https://helm.ngc.nvidia.com/nvidia \
-n gpu-operator --create-namespace --version=v26.3.0

What to do next

1. Verify that the Envoy Gateway CRDs and controller are installed and ready.
2. Verify that the KServe CRDs and controller resources are ready.
3. Verify that the CloudNativePG operator is ready.
4. Verify that the LeaderWorkerSet controller is ready.
5. Verify that the OpenTelemetry Operator is ready.
6. Verify that Prometheus monitoring is ready.
7. Verify that the NVIDIA GPU Operator is ready.

Deploying Nutanix Enterprise AI on Amazon Elastic Kubernetes Service Install or upgrade Nutanix Enterprise AI on Amazon Elastic Kubernetes Service (EKS).

Before you begin

Ensure to complete the following:

Installing Prerequisite Components

Install components on the Kubernetes cluster. For more information, see

on an EKS Cluster

on page 69.

Upgrades from version 2.7.0 to 2.8.0 must be executed during planned downtime. User logins will be unavailable during the upgrade, and full functionality will resume automatically once the upgrade is complete.

About this task

To install or upgrade Nutanix Enterprise AI on EKS, follow these steps:

Procedure

1. Choose a profile:

Profile-based deployment allows you to select a predefined configuration based on your environment and availability requirements. The default profile uses single replica for components and is intended for baseline deployment. The other profiles are for higher capacity usage.

Table 36: Profile and Capacity

2. Pull and untar both the 2.8.0 charts

helm pull ntnx-charts/nai-operators --version 2.8.0 --untar=true
helm pull ntnx-charts/nai-core --version 2.8.0 --untar=true

The extracted charts contain profile files such as:

./nai-operators/profiles/c1k_k200.yaml
./nai-operators/profiles/c5k_k1k.yaml
./nai-core/profiles/c1k_k200.yaml
./nai-core/profiles/c5k_k1k.yaml

3. Deploy for c1k_k200 profile

a. Deploy NAI Operators for c1k_k200 profile

helm upgrade --install nai-operators ntnx-charts/nai-operators --version 2.8.0 \
-n nai-system --create-namespace --wait --timeout 15m \
--set "global.storage.storageClassName=${NAI_DEFAULT_RWO_STORAGECLASS}" \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}" \
-f ./nai-operators/profiles/c1k_k200.yaml \
-f ./nai-operators/eks-values.yaml

b. Deploy NAI Core for c1k_k200 profile

export NAI_API_RWX_STORAGECLASS=<RWX/NFS storageclass>
export NAI_DEFAULT_RWO_STORAGECLASS=<RWO default storageclass>
helm upgrade --install nai-core ntnx-charts/nai-core --version=2.8.0 \
-n nai-system --create-namespace --wait --timeout 15m \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}" \
--set "global.storage.storageClassNameRWX=${NAI_API_RWX_STORAGECLASS}" \
--set "global.storage.storageClassName=${NAI_DEFAULT_RWO_STORAGECLASS}" \
-f ./nai-core/profiles/c1k_k200.yaml \
-f ./nai-core/eks-values.yaml

4. Set up the NAI Helm repository. a. Add and update the Nutanix Helm repository, which contains the

and

helm chart:
nai-core
nai-operators
helm repo add ntnx-charts https://nutanix.github.io/helm-releases && helm repo
update ntnx-charts

b. Search for the version of the

and

helm chart available for installation in the
nai-core
nai-operators

Nutanix helm repository:

helm search repo ntnx-charts/nai-operators --versions
helm search repo ntnx-charts/nai-core --versions

5. Create Docker Registry Secrets:

Create the

namespace and the

secret in both

and

nai-system
docker-registry
nai-system
envoy-

namespaces.

gateway-system

The

is already present on the cluster.

envoy-gateway-system namespace
export REGISTRY_SECRET_NAME=nai-regcred
export DOCKER_SERVER=https://index.docker.io/v1/
export DOCKER_USERNAME=<docker-username>
export DOCKER_PASSWORD=<docker-password>
export DOCKER_EMAIL=<docker-email>
kubectl create namespace nai-system --dry-run=client -o yaml | kubectl apply -f -
kubectl -n nai-system create secret docker-registry ${REGISTRY_SECRET_NAME} \
--docker-server=${DOCKER_SERVER} \
--docker-username=${DOCKER_USERNAME} \
--docker-password=${DOCKER_PASSWORD} \
--docker-email=${DOCKER_EMAIL} \
--dry-run=client -o yaml | kubectl apply -f -
kubectl -n envoy-gateway-system create secret docker-registry ${REGISTRY_SECRET_NAME}
\
--docker-server=${DOCKER_SERVER} \
--docker-username=${DOCKER_USERNAME} \
--docker-password=${DOCKER_PASSWORD} \
--docker-email=${DOCKER_EMAIL} \
--dry-run=client -o yaml | kubectl apply -f -

6. Deploy NAI Operators:

helm upgrade --install nai-operators ntnx-charts/nai-operators --version 2.8.0  -n
nai-system --create-namespace --take-ownership --wait \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}"

[!NOTE] Note: Ensure the REGISTRY_SECRET_NAME environment variable is set before running this command.

helm chart and extract it to your local repository:

7. Pull the

nai-core
helm pull ntnx-charts/nai-core --version 2.8.0 --untar=true

8. Install or upgrade NAI Core on EKS:

# Set the environment variable
NAI_API_RWX_STORAGECLASS=<NFS Storageclass i.e nai-nfs-storage>
NAI_DEFAULT_RWO_STORAGECLASS=<default storageclass>
REGISTRY_SECRET_NAME=<secret created>
helm upgrade --install nai-core ntnx-charts/nai-core --version=2.8.0 -n nai-system --
create-namespace --wait \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}" \
--set "global.storage.storageClassNameRWX=${NAI_API_RWX_STORAGECLASS}" \
--set "global.storage.storageClassName=${NAI_DEFAULT_RWO_STORAGECLASS}" \
-f ./nai-core/eks-values.yaml

[!NOTE] Note: You can append optional Helm overrides to the nai-core installation command to customize the deployment

Enable the Chat and Talk to My Data application. These applications are disabled by default. To enable it, add the following flag to the

Helm install command:

nai-core
--set "naiLabs.enabled=true"

Enable HTTPS with a self-signed certificate. By default, NAI does not provision a TLS certificate for the ingress gateway. To quickly enable HTTPS with a self-signed certificate (recommended for dev/test environments), add the following flag to the

Helm install command:

nai-core
--set "gateway.certManager.selfSigned=true"

This requires

to be installed on the cluster. For production TLS options, including

cert-manager

TLS Encryption on Nutanix

using your own certificate or a cert-manager ClusterIssuer, see

Enterprise AI

.

Scale out the ingress gateway and NAI API. To increase replicas for the ingress gateway and the NAI API, add the following flags to the

Helm install command:

nai-core
--set "gateway.replicaCount=<Number_of_replicas>"
--set "naiApi.replicaCount=<Number_of_replicas>"

The default is 1 replica each. Increase based on your scale requirements.

Configure PostgreSQL database connections. To adjust the maximum number of concurrent PostgreSQL connections, add the following flag to the

Helm install command:

nai-core
--set "naiDatabase.postgresConfig.maxConnections=<Number_of_Connections>"

The default value is 1000. Increase this value if you expect a higher number of concurrent clients.

9. Configure the TLS certificate.

For more information, see

TLS Encryption on Nutanix Enterprise AI

on page 119.

What to do next

Ensure that all the pods are successfully deployed in the

namespace and are in the ready state by:

nai-system
kubectl get pods -n nai-system

The following is a sample output of the command. Match the pod names by ignoring the auto-generated suffix added by Kubernetes and its status.

[!NOTE] Note:

The number of

should be equal to the number of worker

nai-otel-collector-collector pods

nodes in the NAI Kubernetes cluster.

NAME                                                            READY   STATUS
RESTARTS   AGE
chi-nai-clickhouse-server-chcluster1-0-0-0                      1/1     Running     0
16h
chk-nai-clickhouse-keeper-chkeeper-0-0-0                        1/1     Running     0
16h
iam-database-bootstrap-b8etj-hk9rz                              0/1     Completed   0
16h
iam-proxy-68f9459885-zcwgs                                      1/1     Running     0
16h
iam-proxy-control-plane-6897669d64-rvglt                        1/1     Running     0
16h
iam-themis-749b7b56f8-pmclb                                     1/1     Running     0
16h
iam-themis-bootstrap-qgczx-p7bl6                                0/1     Completed   0
16h
iam-ui-6697d94478-fftl5                                         1/1     Running     0
16h
iam-user-authn-5b4dcfdfb7-jhllq                                 1/1     Running     0
16h
nai-api-784fb7b99-8tch9                                         1/1     Running     0
16h
nai-api-db-migrate-mgnba-cl7gw                                  0/1     Completed   0
16h
nai-clickhouse-schema-job-1771350985-bd4sq                      0/1     Completed   0
16h
nai-db-0                                                        1/1     Running     0
16h
nai-iep-model-controller-5cd8bcd5f-9h9cl                        1/1     Running     0
16h
nai-labs-86589cc95d-gk87q                                       1/1     Running     0
16h
nai-oauth2-proxy-5746ccc8b7-65jmf                               1/1     Running     0
16h
nai-oidc-client-registration-lvrrv-pbdsn                        0/1     Completed   0
16h
nai-otel-collector-collector-4pxtl                              1/1     Running     0
16h
nai-otel-collector-collector-8p4w8                              1/1     Running     0
16h
nai-otel-collector-collector-bwstg                              1/1     Running     0
16h
nai-otel-collector-collector-drddv                              1/1     Running     0
16h
nai-otel-collector-collector-k457n                              1/1     Running     0
16h
nai-otel-collector-collector-nmwjw                              1/1     Running     0
16h
nai-otel-collector-targetallocator-c9dcc6544-5wbrx              1/1     Running     0
16h
nai-pulse-job-29522885-6w7pn                                    0/1     Completed   0
10h
nai-ui-6d9cc89b87-b4npf                                         1/1     Running     0
16h
nutanix-ai-operators-nai-clickhouse-operator-7f9965dbdd-trwp5   2/2     Running     0
16h
redis-standalone-6df56bc96d-hp86q                               2/2     Running     0
16h

Access NAI Dashboard IP:

kubectl get svc -n envoy-gateway-system -l "gateway.envoyproxy.io/owning-gateway-
name=nai-ingress-gateway,gateway.envoyproxy.io/owning-gateway-namespace=nai-system" -
o jsonpath='{.items[0].status.loadBalancer.ingress[0].hostname}'

Log in to Nutanix Enterprise AI

For information on viewing the Dashboard, see

on page 167.

If you had endpoints in

Pending

status before the upgrade displaying the status as

Failed

with the message

Unable to pull runtime image with provided credentials, hibernate and resume the endpoints.

Deploying Nutanix Enterprise AI on Azure Kubernetes Service

Install or upgrade Nutanix Enterprise AI (NAI) on Azure Kubernetes Service (AKS).

To install or upgrade NAI on AKS, follow these step:

1. Set up an AKS cluster. For more information, see

Setting up an Azure Kubernetes Service Cluster

on

page 76.

2. Perform preflight checks. For more information, see

Performing Preflight Checks Before Installing Nutanix

Enterprise AI on Azure AKS

on page 77.

3. Install prerequisite components. For more information, see

Deploying Nutanix Enterprise AI on Azure

Kubernetes Service

on page 82.

4. Deploy NAI on AKS. For more information, see

Deploying Nutanix Enterprise AI on Azure Kubernetes

Service

on page 82.

Setting up an Azure Kubernetes Service Cluster Set up your Azure Kubernetes Service (AKS) cluster for Nutanix Enterprise AI (NAI). This procedure is not applicable for Nutanix Kubernetes Platform (NKP) managed or attached AKS clusters.

Before you begin

Ensure that your cluster meets all the requirements mentioned in

Nutanix Enterprise AI - Private Inference

and Agent Gateway Requirements

on page 10.

About this task

To set up your AKS cluster before you deploy NAI, follow these high-level steps:

Procedure

1. Configure Azure CLI.

For more information, see

Azure documentation

.

2. Create an Azure resource group.

Azure documentation

For more information, see

.

3. Create an AKS cluster with Kubernetes version 1.33 or 1.34.

Azure documentation

For more information, see

.

4. Configure the AKS cluster with a supported network policy engine enabled for enforcement.

For more information, see

Azure documentation

.

5. Configure the AKS preview extension.

Azure documentation

For more information, see

.

6. Create a default node group. Make sure the nodes have accelerator AVX2 or newer.
7. Add a GPU node pool to the AKS cluster using Azure CLI.

Ensure that you skip installing the GPU driver when adding the GPU node pool to the cluster. For more information, see

Azure documentation

Azure documentation

. For more information, see

.

Performing Preflight Checks Before Installing Nutanix Enterprise AI on Azure AKS Perform preflight checks before installing Nutanix Enterprise AI on Azure AKS. Preflight checks ensure the Kubernetes environment meets all the technical requirements for a successful NAI installation.

About this task

To perform preflight checks for installing Nutanix Enterprise AI on Azure AKS, follow these steps:

Procedure

1. Choose a profile:

Profile-based deployment allows you to select a predefined configuration based on your environment and availability requirements. The default profile uses single replica for components and is intended for baseline deployment. The other profiles are for higher capacity usage.

Table 37: Profile and Capacity

2. Pull and untar both the 2.8.0 charts

helm pull ntnx-charts/nai-operators --version 2.8.0 --untar=true
helm pull ntnx-charts/nai-core --version 2.8.0 --untar=true

The extracted charts contain profile files such as:

./nai-operators/profiles/c1k_k200.yaml
./nai-operators/profiles/c5k_k1k.yaml
./nai-core/profiles/c1k_k200.yaml
./nai-core/profiles/c5k_k1k.yaml

3. Deploy for small profile:

a. Deploy NAI Operators for small profile

helm upgrade --install nai-operators ntnx-charts/nai-operators --version 2.8.0 \
-n nai-system --create-namespace --wait --timeout 15m \
--set "global.storage.storageClassName=${NAI_DEFAULT_RWO_STORAGECLASS}" \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}" \
-f ./nai-operators/profiles/small.yaml \
-f ./nai-operators/aks-values.yaml

b. Deploy NAI Core for small profile

export NAI_API_RWX_STORAGECLASS=<RWX/NFS storageclass>
export NAI_DEFAULT_RWO_STORAGECLASS=<RWO default storageclass>
helm upgrade --install nai-core ntnx-charts/nai-core --version=2.8.0 \
-n nai-system --create-namespace --wait --timeout 15m \
--set "global.imagePullSecrets[0].name=${REGISTRY_SECRET_NAME}" \
--set "global.storage.storageClassNameRWX=${NAI_API_RWX_STORAGECLASS}" \
--set "global.storage.storageClassName=${NAI_DEFAULT_RWO_STORAGECLASS}" \
-f ./nai-core/profiles/small.yaml \
-f ./nai-core/aks-values.yaml

4. Ensure your Kubernetes version is 1.33 or 1.34:

kubectl get nodes --selector='!node-role.kubernetes.io/control-plane' \
-o custom-
columns=NAME:.metadata.name,KUBELET_VERSION:.status.nodeInfo.kubeletVersion \
--no-headers

The expected output is that the Kubernetes version must be 1.33 or 1.34.

5. Verify if the number of Azure Disk CSI pods (

) match the number of worker nodes:

csi-azuredisk-node
[ $(kubectl get nodes --no-headers | wc -l) -eq $(kubectl get pods -n kube-system --
no-headers | grep csi-azuredisk-node | wc -l) ] && echo "# csi-azuredisk-node pods =
node count" || echo "# Mismatch: csi-azuredisk-node pods != node count"

The expected output is

.

csi-azuredisk-node pods= node count

6. Confirm that the

storage class binds volumes immediately:

azurefile-csi
kubectl get sc azurefile-csi -o jsonpath='{.metadata.name}: {.volumeBindingMode}
{"\n"}'

The expected output is

.

immediate

7. Check if the storage class has

access:

ReadWriteMany
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-rwx-pvc
namespace: default
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
storageClassName: azurefile-csi
EOF
kubectl get pvc test-rwx-pvc -n default

The expected output is that the storage class

has

set to

.

azurefile-csi
Access Modes
RWX
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS
MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
test-rwx-pvc   Bound    pvc-4811d039-5cf4-4925-9181-2b2a3689ea47   1Gi        RWX
azurefile-csi   <unset>                 4s
kubectl delete pvc test-rwx-pvc

8. Verify if a CNI is enabled on the cluster:

az aks show --resource-group <resource-group-name> --name <aks-cluster-name> --query
"networkProfile.networkPolicy"