Configuration Reference

This page documents the CiscoDevice custom resource and the explicitly marked local-configuration variants. Unless a field is labelled local-only, it lives under spec on the CR. The controller reads the CR, strips credentials and controller-resolved references, and materializes the non-secret runtime configuration into a ConfigMap that the VK pod reads — you do not edit the ConfigMap directly. Credentials and private keys are never copied into that ConfigMap. The public-CA option described below copies only validated public certificates; other reference fields state how their Secrets are projected.

For device-side prerequisites (IOS-XE CLI config, DHCP pools, VLANs, etc.) and per-platform networking examples, see:

Catalyst 8000V — VirtualPortGroup install
Catalyst 9000 — AppGigabitEthernet install (access and trunk)

Minimal spec

apiVersion: cisco.vk/v1alpha1
kind: CiscoDevice
metadata:
  name: cat9000-1
  namespace: default
spec:
  driver: XE
  address: "192.168.1.100"
  port: 443
  username: admin
  credentialSecretRef:
    name: cat9000-1-creds       # Secret with key: password
  tls:
    enabled: true
    insecureSkipVerify: false
  # allowUnsignedApps: true      # enable when running unsigned packages
                                  # (your own builds, or devices without
                                  # signed-verification enforcement)
  xe:
    networking:
      interface:
        type: AppGigabitEthernet
        appGigabitEthernet:
          mode: trunk
          vlanIf:
            dhcp: true
            vlan: 200
            guestInterface: 0

This Catalyst 9000 example assumes the VLAN and DHCP setup in the platform guide. It requires a device certificate trusted by the worker's system CA store and matching spec.address. For a private CA, see Shared TLS below. It leaves gNOI in backward-compatible auto mode. Configure explicit verified spec.gnoi before invoking any gNOI operation.

CLI flags & environment variables

Runtime settings are generally passed as flags or environment variables. Node name and log level also accept device-configuration fallbacks, as noted below. See the CLI reference for the full list.

Flag	Env var	Default	Description
`--nodename`	`CISCO_VK_NODE_NAME`, then `VKUBELET_NODE_NAME`	`device.nodeName`, then `cisco-vk-<device-address>`	Resolution: flag → listed environment variables → `device.nodeName` → address-derived name; falls back to `cisco-virtual-kubelet` without an address.
`--config` / `-c`	—	`/etc/virtual-kubelet/config.yaml`	Device config path
`--kubeconfig`	`KUBECONFIG`	in-cluster	kubeconfig path
`--log-level`	`LOG_LEVEL`	`device.logLevel`, then `info`	Resolution: flag → environment → device setting → `info`; accepts `debug`, `info`, `warn`/`warning`, or `error`.
`--tls-cert-file`	—	`/etc/virtual-kubelet/tls/tls.crt`	Kubelet HTTPS listener certificate; if both files are absent, generate a self-signed pair under `/var/lib/virtual-kubelet/tls/`.
`--tls-key-file`	—	`/etc/virtual-kubelet/tls/tls.key`	Kubelet HTTPS listener key; exactly one file being present is an error.
—	`VK_DEVICE_PASSWORD`	—	Device password. Overrides `device.password` from the config file. Set by the controller when `credentialSecretRef` is used.

Configuration fields

Core

Field	Type	Required	Default	Notes
`driver`	enum	yes	—	`XE`, `NXOS`, `XR`, `OPENCONFIG`, `FAKE`. `XE` is the broadest production driver. `NXOS` supports NX-API CLI app-hosting plus the initial NX-API REST/DME `NXOSConfig` Network as Code slice (`system`, `feature`, `feature_set`, `vlan`, `interface_ethernet`). `XR` and `OPENCONFIG` remain reserved placeholders.
`address`	string	yes	—	Management IP or hostname
`port`	int (1–65535)	no	443 (TLS) / 80	Device API port. IOS-XE uses RESTCONF/NETCONF/gNMI depending on transport; NX-OS uses NX-API CLI and NX-API REST/DME over HTTP(S).
`username`	string	yes	—	Device username
`password`	string	no	—	Device password. Do not set in controller mode — use `credentialSecretRef` instead.
`credentialSecretRef`	LocalObjectReference	no	—	Reference to a `Secret` in the same namespace containing key `password`. See Security.

Shared TLS

The file paths in this block are directly usable in local configuration or a custom Deployment that mounts them. For controller-managed device HTTPS, spec.tls.caSecretRef projects a public CA bundle into both per-device worker planes. Arbitrary local paths are not automatically mounted. Dedicated gNOI trust remains separate: use spec.gnoi.tls.secretRef or IOS XE provisioning.

Managed public-CA Secret introduced in v2026.11.1

spec.tls.caSecretRef requires the matching November CRD, admission contract, manager and worker image. Upgrade those components together. For installations staying on October v2026.10.0, use a system-trusted device certificate or an explicitly mounted CA file in a custom Deployment. The local caFile field alone does not arrange that mount in a controller-managed worker.

tls:
  enabled: true
  insecureSkipVerify: false
  certFile: /path/to/client.crt
  keyFile: /path/to/client.key
  caFile:  /path/to/ca.crt

Field	Type	Default	Notes
`tls.enabled`	bool	`false`	Enable HTTPS to the device
`tls.insecureSkipVerify`	bool	`false`	Skip certificate verification
`tls.certFile`	string	—	Local/custom-Deployment client certificate path; must be configured together with `tls.keyFile`.
`tls.keyFile`	string	—	Local/custom-Deployment client key path; must be configured together with `tls.certFile`.
`tls.caFile`	string	—	Local/custom-Deployment CA bundle path.
`tls.caSecretRef.name`	string	—	Same-namespace Secret containing `ca.crt`; per-device workers only. Requires verified TLS and excludes `caFile`. Only the public CA key is projected.

An incomplete shared client-certificate pair now fails startup consistently for configdriver, telemetry, gNOI, and NX-API consumers. Before upgrading, remove any obsolete lone certFile or keyFile, or configure both paths.

For a CiscoDevice managed by the controller:

spec:
  tls:
    enabled: true
    insecureSkipVerify: false
    caSecretRef:
      name: switch-https-public-ca

Create that Secret from the independently trusted CA bundle, with key ca.crt. The HTTPS certificate must match the device address (IP SAN for an IP address). Prefer a dedicated public-CA-only Secret. The controller validates its complete ca.crt as public CA certificates and copies those public bytes into device-ca.crt in the existing managed ConfigMap. Workers mount only that validated snapshot, not the mutable source Secret. A later malformed or key-containing Secret update therefore cannot be refreshed directly into a running worker by kubelet. No tls.key, ca.key, unrelated key or invalid bundle is copied. Public certificates in this ConfigMap are intentionally readable to principals with ConfigMap-read access. This does not change existing Kubernetes Secret-read permissions or replace RBAC isolation. It does not provision/replace a certificate on the switch.

The controller binds the CA revision and SHA-256 into each worker template. Workers verify the actual mounted bytes before opening clients. Rotation, deletion or recreation triggers a fenced Recreate rollout; missing/malformed CA material removes the public snapshot and leaves replacement workers unready rather than falling back to insecure TLS. Revision/claim fencing precedes replacement; convergence is asynchronous, not instantaneous connection revocation. Managed software claims additionally re-read the current Secret revision, and preparation receipts bind that revision so later trust changes block reuse. Keep old receipts for audit; do not rewrite them after CA rotation.

Upgrade the CRD, native admission contract, manager and workers before enabling this opt-in field. The manager attests the native public-CA-only projection policy (including rejection of direct source-Secret projections); an older manager must fail that changed contract at startup. The controller-owned startup flag also makes older worker binaries reject the configuration before startup. No arbitrary rollback is supported while the reference remains enabled. The aggregator and standalone file configuration must continue to use an explicitly mounted caFile; they do not resolve Kubernetes CA references. Existing configurations without the new field retain their current behavior.

gNOI

Field	Type	Default	Notes
`gnoi.transportSecurity`	enum	`auto`	`tls` forces TLS; `auto` follows `tls.enabled` and preserves legacy inference.
`gnoi.port`	int (1–65535)	mode-dependent	Overrides only the gNOI/gNXI endpoint port. Explicit TLS defaults to `9339`. Legacy `auto` inference uses `9339` for TLS, `50052` for plaintext, or preserves an existing nonstandard device port.
`gnoi.tls.secretRef.name`	string	—	Kubernetes only. Same-namespace Secret containing required `ca.crt` and, optionally, both `tls.crt` and `tls.key`. Requires `gnoi.transportSecurity: tls`.
`gnoi.tls.caFile`	string	—	Local config only. Required CA bundle path for dedicated verified gNOI TLS; rejected in a `CiscoDevice`.
`gnoi.tls.certFile`, `gnoi.tls.keyFile`	string	—	Local config only. Optional client certificate pair; configure both or neither. Rejected in a `CiscoDevice`.
`xe.gnoi.certificateProvisioning.certificateID`	string	—	IOS-XE only. Required for opt-in provisioning; 1–64 letters, digits, `.`, `_`, or `-`.
`xe.gnoi.certificateProvisioning.secretRef.name`	string	—	IOS-XE only. Required same-namespace Secret. `tls.crt` and `ca.crt` are required; `ca.key` enables the local `ProvisionCertificate` signer/action and should be removed after provisioning; `bootstrap.crt` is an optional exact leaf pin.
`xe.gnoi.certificateProvisioning.replaceTargetCABundle`	bool	—	IOS-XE only. Must be `true`; confirms that `ca.crt` is the complete desired replacement for IOS-XE's shared gNXI/gNMI CA bundle.

For IOS-XE, explicit spec.gnoi.transportSecurity: tls selects its TLS-only username/password metadata adapter. Verified trust comes from exactly one of: the system root pool, verified shared spec.tls, dedicated spec.gnoi.tls, or IOS-XE provisioning. Dedicated gNOI TLS has no enabled or insecureSkipVerify controls. In a CiscoDevice, it must reference a Secret; the controller validates and mounts only its fixed keys read-only, then restarts an enabled per-device worker when valid material changes. With global gNOI disablement, the existing worker omits the trust projection and does not roll for Secret changes. Aggregated config-only topology has no per-device worker.

spec:
  gnoi:
    transportSecurity: tls
    port: 9339
    tls:
      secretRef:
        name: cat9000-1-gnoi-tls

For a direct local cisco-vk run --config file, use paths instead of a Secret reference:

device:
  gnoi:
    transportSecurity: tls
    tls:
      caFile: /run/cvk/gnoi/ca.crt
      # certFile and keyFile are an optional pair for mutual TLS.

IOS-XE certificate provisioning is intentionally scoped under spec.xe.gnoi. Its Secret automatically supplies isolated bootstrap and steady-state gNOI trust, so spec.gnoi.tls and spec.xe.gnoi.certificateProvisioning are mutually exclusive. Top-level spec.tls.insecureSkipVerify: true is accepted for an unrelated transport only when gNOI uses one of those dedicated verified trust sources. Provisioning is performed only by the gated ProvisionCertificate action; read-only GNOIOSVerify never installs a certificate. See the secure authentication and provisioning workflow and its security guidance. Environment overrides remain documented in the CLI reference.

Node and pod topology

Field	Type	Default	Notes
`podCIDR`	string	—	IOS-XE static-address input when DHCP is off. The current implementation gives only the first container a fixed address and does not allocate unique addresses across Pods; see the limitation below.
`labels`	map[string]string	`{}`	Legacy extra Node labels. In managed topology, non-topology values remain compatible, but topology values must agree with protected `metadata.labels` and the administrator projection allowlist.
`taints`	[]v1.Taint	`[]`	Extra Node taints. In managed topology the manager, not the worker, owns the projected/static taint set.
`maxPods`	int32	`16`	Pod capacity advertised to Kubernetes; not discovered device app-hosting capacity. Managed topology requires `1–110`. Choose a value that fits the platform's actual capacity.
`region`	string	—	Legacy region source. In managed topology, authoritative `metadata.labels["topology.kubernetes.io/region"]` must be present and this value may only agree with it.
`zone`	string	—	Legacy zone source. In managed topology, authoritative `metadata.labels["topology.kubernetes.io/zone"]` must be present and this value may only agree with it.

For scheduling and fleet rollouts, use protected CiscoDevice.metadata.labels and the opt-in administrator policy described in Managed topology and topology-aware IOS-XE rollouts. Only region, zone, and allowlisted topology.cisco.vk/* keys reach Nodes; operations.cisco.vk/* labels remain rollout-only risk data.

Static addressing is not IP address management

With DHCP disabled, IOS-XE derives the first container's IPv4 address from the podCIDR network address by adding 10 to its last octet. It does not reserve addresses across Pods, so separate Pods reuse that address; subsequent containers receive no static address from this calculation. Use DHCP for multiple independent workloads.

Resource limits

These fields configure device app profiles. Despite their default names, configured defaultCPU and defaultMemory override the corresponding container requests; IOS-XE also applies defaultStorage this way. They do not set Node capacity or allocatable resources. Node capacity comes from the driver's capacity reporting and maxPods.

resourceLimits:
  defaultCPU: "500m"
  defaultMemory: "512Mi"
  defaultStorage: "500Mi"  # IOS-XE only; omit for NX-OS

Field	Type	Notes
`resourceLimits.defaultCPU`	string	Device-wide app-profile CPU override; replaces the container CPU request when set.
`resourceLimits.defaultMemory`	string	Device-wide app-profile memory override; replaces the container memory request when set.
`resourceLimits.defaultStorage`	string	IOS-XE app-profile disk override; replaces the ephemeral-storage request when set. NX-OS rejects this setting and explicit ephemeral-storage requests.
`resourceLimits.maxCPU`	string	Reserved field; currently not enforced by the drivers.
`resourceLimits.maxMemory`	string	Reserved field; currently not enforced by the drivers.
`resourceLimits.maxStorage`	string	Reserved field; currently not enforced by the drivers.
`resourceLimits.others`	map[string]string	Driver-specific settings. For NX-OS, `maxApps` overrides the conservative app-hosting slot guard. These are not automatically advertised as Kubernetes extended resources.

Without a request or a device override, both app-profile implementations start with 1000m CPU, 512Mi memory, 1024Mi disk and one vCPU. IOS-XE rounds CPU limits up to a vCPU count; NX-OS rejects CPU limits above 1000m. Memory limits are not directly read by the profile conversion. Kubernetes admission may populate requests before the Pod reaches CVK. Do not use the reserved max* fields as resource-enforcement controls.

NX-OS maps each Kubernetes container to one NX-OS app-hosting app. CVK defaults the NX-OS app-slot guard to 4, matching the observed Nexus 9300v lab ceiling, and rejects oversized pods before sending config. Raise resourceLimits.others.maxApps only after validating a higher platform-specific limit. NX-OS app-resource profile values are bounded to the platform's 16-bit CLI fields; CPU milli-units or memory MiB above 65535 are rejected before any app-hosting configuration is sent.

Logging

logLevel: info   # debug | info | warn | error

The logLevel field is passed to the VK pod as the --log-level flag on the container.

App packaging

Field	Type	Default	Notes
`allowUnsignedApps`	bool	`false`	When `true`, CVK performs two actions: (1) on connect it writes the persistent IOS-XE `sign-verification: false` control and invokes the app-hosting runtime verification-disable RPC; (2) the reconciler treats `iox-pkg-policy-invalid` during `INSTALLING` as a transient signal rather than a fatal error. Both device operations must be accepted before unsigned installs can work. Enable only for unsigned custom or test packages; production packages should remain signed. If the platform refuses the runtime change, CVK logs a warning and unsigned installs remain blocked even when the configuration datastore reads `false`. See Troubleshooting → PackagePolicyInvalid.

OpenTelemetry topology

otel:
  enabled: true
  endpoint: "otel-collector.observability.svc:4317"
  insecure: true
  serviceName: "cisco-network"
  intervalSecs: 60

Field	Type	Default	Notes
`otel.enabled`	bool	`false`	Toggle topology trace emission
`otel.endpoint`	string	—	OTLP gRPC collector endpoint. Required when `enabled: true`.
`otel.insecure`	bool	`true`	Skip TLS on the gRPC connection
`otel.serviceName`	string	`"cisco-network"`	Base service name. The device hostname is appended → `"cisco-network.<hostname>"`.
`otel.intervalSecs`	int (min 10)	`60`	Interval between trace emissions

OTEL export only happens when the driver implements TopologyProvider (the IOS-XE driver does). See Observability → OpenTelemetry.

XEConfig — IOS-XE networking

Required when driver: XE. The type field selects which mode is in use; exactly one of the mode-specific blocks must be set to match.

xe:
  networking:
    interface:
      type: VirtualPortGroup      # | AppGigabitEthernet | Management
      virtualPortGroup: { ... }
      appGigabitEthernet: { ... }
      management: { ... }

`type`	Typical platform	Details
`VirtualPortGroup`	Catalyst 8000V	configuration-cat8000v.md
`AppGigabitEthernet`	Catalyst 9000	configuration-cat9000.md
`Management`	Either (containers on management network)	Below

For AppGigabitEthernet, top-level appGigabitEthernet.dhcp applies only to access mode with VLAN 0. Other access and trunk configurations use appGigabitEthernet.vlanIf.dhcp.

Management interface (both platforms)

The Management interface mode places containers on the device's management network rather than on a dedicated App Hosting interface. It is supported on both Catalyst 8000V and Catalyst 9000.

xe:
  networking:
    interface:
      type: Management
      management:
        dhcp: true
        guestInterface: 0

Field	Type	Default	Notes
`dhcp`	bool	`false`	DHCP on the management interface. If `false`, use the limited static `podCIDR` calculation described above.
`guestInterface`	uint8 (0–3)	`0`	Container-side interface index (0 = eth0).

Note that putting containers on the management network exposes them directly to management traffic and any ACLs in place there. For workload isolation, prefer VirtualPortGroup (on Catalyst 8000V) or AppGigabitEthernet (on Catalyst 9000).

Example pod manifest

apiVersion: v1
kind: Pod
metadata:
  name: hello-app
  namespace: default
spec:
  nodeName: cat9000-1
  tolerations:
  - key: virtual-kubelet.io/provider
    operator: Exists
  containers:
  - name: hello
    image: flash:/hello-app.iosxe.tar
    resources:
      requests:
        memory: "256Mi"
        cpu: "500m"
      limits:
        memory: "512Mi"
        cpu: "1000m"

Note

The container image reference can be either:

A flash path (flash:/..., bootflash:/...) — image tar must be pre-loaded on the device.
An HTTP(S) URL — the VK attempts a device-native pull first; if the platform cannot pull, it downloads the image itself and installs from flash (see Image delivery below).

A registry reference such as nginx:latest is not accepted. To serve registry images to devices, see Image Distribution.

Package policy (signed vs unsigned) can be managed via spec.allowUnsignedApps: true in the CiscoDevice spec — CVK configures the device and adjusts reconciler behaviour automatically. The equivalent IOS-XE CLI is no app-hosting signed-verification.

Image delivery

By default the VK passes the container image URL directly to the IOS-XE install RPC and waits for the app to reach RUNNING. On platforms that support device-native HTTP pulls, no extra steps are needed.

To deploy images from Docker Hub, GHCR, or a private registry, see Image Distribution.

On platforms that cannot pull from HTTP URLs, the VK automatically falls back to a copy-then-install path: it downloads the image to device flash via the copy RESTCONF RPC, then reinstalls from the local flash path. This fallback can take several minutes (the copy RPC is synchronous and blocks until the device finishes downloading).

`imagePullPolicy`

Behaves analogously to the Kubernetes imagePullPolicy field, with one important platform constraint:

IOS-XE App Hosting does not cache images on flash when using the device-native pull path. When the device installs an app directly from an HTTP URL (app-hosting install appid ... package <url>), the image is fetched and loaded into the container runtime without leaving a copy on flash. This means IfNotPresent and Always are functionally identical on the device-native pull path — the image is always re-downloaded from the URL on each deploy.

The IfNotPresent flash-cache optimization only applies to the copy fallback path: if the VK previously copied a tar to flash (because the device-native pull timed out), that cached copy is reused on subsequent deploys.

Value	Behaviour
`Always`	Always attempt device-native pull; if that times out, always re-download via copy RPC. The flash cache is never reused.
`IfNotPresent` (default)	Always attempt device-native pull first. If that times out, try installing from a previously cached flash tar before re-downloading. If the device-native pull succeeds, no flash cache is created and no difference from `Always` is observable.
`Never`	Image must already exist on flash. HTTP URLs are rejected immediately — no install RPC is issued.

To reliably benefit from IfNotPresent caching, deliberately use the copy path: either configure a short imagePullPolicy timeout so the device-native pull always times out quickly (triggering copy fallback), or pre-stage the image using imagePullPolicy: Never with a flash path after a first manual copy.

`imagePullSecrets`

When the image is served from a registry that requires authentication, provide credentials via a Kubernetes Secret and reference it from spec.imagePullSecrets. The VK embeds the credentials as HTTP basic auth in the copy RPC URL.

spec:
  imagePullSecrets:
  - name: registry-credentials

The referenced Secret should be of type kubernetes.io/dockerconfigjson or contain a token key. The VK tries the following fields in priority order: token key → .dockerconfigjson identitytoken → .dockerconfigjson registrytoken → .dockerconfigjson username/password → .dockerconfigjson auth (base64 user:pass).

Note

Authentication is applied only to the copy RPC (device-initiated download). The device-native pull path does not support credential injection via this mechanism.

Pod annotations for the copy path

Two optional annotations tune the copy fallback behaviour:

Annotation	Default	Description
`cisco.io/apphost-package-dest`	`flash:/virtual-kubelet/<app-id>.tar`	On-device flash path where the image is copied. Must use an IOS-XE filesystem prefix (`flash:`, `bootflash:`, `harddisk:`, `usb:`, `nvram:`); each path segment may contain only ASCII letters, digits, `-`, `.`, `_`, or `~`.
`cisco.io/apphost-package-timeout`	`180s` (3 min)	Per-status wait limit for each app-hosting lifecycle transition (for example, `DEPLOYED`, `ACTIVATED`, then `RUNNING`); a multi-stage install can therefore take more than this value overall. Accepts Go duration strings (`3m`, `300s`) or bare seconds (`180`). Clamped to [10s, 30m].

metadata:
  annotations:
    cisco.io/apphost-package-dest:    "flash:/virtual-kubelet/my-app.tar"
    cisco.io/apphost-package-timeout: "5m"

Kubernetes pod events

During image delivery the VK emits pod events visible in kubectl describe pod:

Event	Reason	When
Normal	`Pulling`	Device-native HTTP pull started
Warning	`ImagePullFallback`	Device-native pull timed out; copy fallback begins
Normal	`Copying`	Copy RPC started (shows source URL and destination path)
Normal	`Pulled`	Image successfully copied to device flash
Normal	`Started`	App reached RUNNING via copy fallback