Skip to content

Welcome to Cisco Virtual Kubelet

A Virtual Kubelet provider that lets Kubernetes schedule container workloads directly onto Cisco Catalyst series switches and IOS-XE devices with App-Hosting capabilities.

Make your network infrastructure a first-class Kubernetes citizen.

Concepts at a glance

Four ideas you'll see referenced throughout the docs:

  • Virtual Kubelet - an open-source project that lets any system impersonate a Kubernetes node. Instead of running kubelet on a real VM or bare-metal host, a Virtual Kubelet provider registers a virtual node in your cluster and handles pod lifecycle however it likes. This project is a provider for Cisco devices.
  • IOx / App-Hosting - Cisco's on-device container runtime, available on Catalyst 8000V, Catalyst 9000, IR1100 Series, and IE3500 Series platforms. It runs OCI-like container packages (.tar files) directly on the device alongside normal network functions.
  • Network as Code CRDs - Kubernetes resources such as IOSXEConfig, IOSXEConfigBundle, IOSXETelemetry, DeviceOperation, and IOSXESoftwareUpgrade that express device configuration, telemetry, diagnostics, and operations as Kubernetes API objects.
  • RESTCONF, NETCONF, gNMI, and gNOI - IOS-XE management protocols used by the provider. App-hosting lifecycle still uses RESTCONF; declarative config can use RESTCONF, NETCONF, or gNMI; telemetry and software operations use gNMI/gNOI.

Put those together: each Cisco device becomes a virtual node in your cluster. Pods scheduled to that node run as App-Hosting containers on the device, while configuration and operational workflows stay Kubernetes-native.

Kubernetes API
  -> CiscoDevice
  -> per-device cisco-vk pod
  -> virtual node
  -> pods as IOS-XE app-hosting containers

Kubernetes API
  -> config.cisco.vk and ops.cisco.vk CRDs
  -> config, telemetry, operation, and software lifecycle reconcilers
  -> IOS-XE through RESTCONF, NETCONF, gNMI, and gNOI

What it does

  • Native Kubernetes integration - deploy to Cisco devices with standard kubectl apply. No separate lifecycle is required for app-hosted pods.
  • Driver-based architecture - extensible driver pattern with IOS-XE (Catalyst 8000V, Catalyst 9000, IR1100 Series, and IE3500 Series) available today.
  • Full pod lifecycle - create, update, recover, and delete containers via RESTCONF, with automatic state reconciliation and pod recovery.
  • Network as Code - declarative IOS-XE configuration CRDs with defaults, group targeting, templates, bundles, revisions, drift detection, and apply logs.
  • Operations and upgrades - read-only diagnostics, gNOI probes, write-class operational actions, and multi-phase IOS-XE software upgrades behind explicit RBAC and runtime gates.
  • Observability built in - Prometheus metrics for device CPU, memory, storage, and interfaces; OpenTelemetry topology traces with CDP, OSPF, and hosted-app context; node annotations carrying router ID, hostname, and neighbor counts.
  • Secure credentials - device passwords are injected via Kubernetes Secrets and valueFrom.secretKeyRef, never embedded in ConfigMaps.
  • Flexible networking - DHCP or static allocation across VirtualPortGroup, AppGigabitEthernet, and Management interfaces. Pod IP discovery uses device operational data first and ARP as a fallback.

Status

This project is under active development and is published as open source under cisco-open.

  • Releases - official releases are cut monthly and tagged on GitHub. The latest release is the recommended starting point; main may contain unreleased in-flight changes.
  • CRD versions - cisco.vk/v1alpha1, config.cisco.vk/v1alpha1, and ops.cisco.vk/v1alpha1. Breaking changes are still possible as the schemas stabilise.
  • Drivers - XE is production-focused; FAKE is for testing; XR, NXOS, and OPENCONFIG are reserved driver names in the API surface.
  • Images - images are not yet published to a public container registry. Build locally from a release tag or main, then push to a registry your cluster can pull from. See Getting Started.

Feature Maturity

Not all feature areas have the same level of maturity. The table below summarises the current state for the June 2026 release.

Feature area Maturity Notes
Pod lifecycle (App-Hosting create / update / delete) Stable Supported on Catalyst 8000V 17.15+, Catalyst 9000 17.18+, IR1100 Series 17.12+, and IE3500 Series 17.18+.
CiscoDevice and VK deployment lifecycle Stable Controller-managed per-device VK pods.
Network as Code config driver (IOSXEConfig family) Beta Declarative IOS-XE config CRDs with drift detection, bundles, revisions, and apply logs. Schema is v1alpha1; family coverage and wire-format behaviour are still expanding.
Operations (DeviceOperation, IOSXEOperationalAction) Beta Read-only diagnostics and gNOI probes are stable in intent; write-class actions require an explicit runtime gate and carry additional operational risk.
Software Lifecycle (IOSXESoftwareUpgrade) Beta Multi-phase gNOI OS install/activate/verify. Disabled by default; requires --enable-iosxesoftwareupgrade. Tested on limited platforms.
Telemetry (IOSXETelemetry) Beta MDT-over-gNMI subscriptions converted to OpenTelemetry signals. Pipeline architecture is stable; subscription schema is v1alpha1.
Observability (Prometheus metrics, OTEL topology traces) Beta Metrics catalog and trace shapes may change between releases.

Beta features

Features marked Beta are functional and tested but carry v1alpha1 API versions. Breaking schema changes are still possible. They should be evaluated in non-production environments before broader rollout. Runtime gates exist for the highest-risk surfaces (write-class gNOI, software upgrades) and must be opted into explicitly.

Where to next

Glossary

Term Meaning
App-Hosting Cisco's on-device container platform. Runs .tar container packages on IOS-XE devices.
CDP Cisco Discovery Protocol, used for Layer 2 neighbor discovery.
CR / CRD Custom Resource / Custom Resource Definition, Kubernetes' API extension mechanism.
gNMI gRPC Network Management Interface, used for model-driven telemetry and optional config transport.
gNOI gRPC Network Operations Interface, used for read-only probes, file operations, reboot, factory reset, and software upgrade flows.
IOx Cisco's on-device application hosting framework, including App-Hosting.
Network as Code Declarative IOS-XE intent shape consumed by IOSXEConfig and related config CRDs.
OTEL / OpenTelemetry Vendor-neutral observability framework; this project emits OTEL traces and metrics.
RESTCONF HTTP/JSON management API for network devices, defined by RFC 8040, modeled by YANG.
Virtual Kubelet Upstream project letting any system appear as a Kubernetes node.
VK Short for Virtual Kubelet.
VPG / VirtualPortGroup A logical L3 interface on IOS-XE used to bridge app-hosted containers into the device network.
YANG Data modeling language used to describe configuration and state.