Cisco Nxos

---
name: cisco-nxos
description: "Expert coverage of Cisco NX-OS across all versions: Nexus 9000 data center switching, VXLAN/EVPN fabric design, vPC, spine-leaf architecture, NX-API, Nexus Dashboard, and DC security hardening. Use for \"NX-OS\", \"Nexus 9000\", \"vPC\", \"VXLAN EVPN\", \"NX-API\", \"Nexus Dashboard\", \"NDFC\", \"spine-leaf\", \"data center fabric\". Do NOT use for cross-vendor comparison, platform selection, or category-wide architecture -- use the `routing-switching` skill."
license: MIT
---

# Cisco NX-OS

This skill covers Cisco NX-OS across all supported versions (10.x for Nexus 9000/3000). Areas of expertise include:

- Modular microkernel architecture with process isolation and non-stop forwarding
- VXLAN/EVPN fabric design (BGP EVPN control plane, symmetric IRB, multi-site)
- vPC (Virtual Port-Channel) configuration and troubleshooting
- Spine-leaf data center fabric architecture
- NX-API (CLI, REST, JSON-RPC) programmability
- Nexus Dashboard (NDFC, NDI, NDO) management and orchestration
- gRPC/gNMI streaming telemetry
- ACI mode vs standalone NX-OS
- Checkpoint and rollback operations

## How to Approach Tasks

1. **Classify** the request:
   - **Troubleshooting** -- Load `references/diagnostics.md` for show commands, NX-API queries, vPC/VXLAN troubleshooting
   - **Design / Architecture** -- Load `references/architecture.md` for VXLAN/EVPN, ACI vs standalone, NX-API, Nexus Dashboard
   - **Best practices** -- Load `references/best-practices.md` for DC fabric design, vPC, ECMP, security hardening
   - **Configuration** -- Apply NX-OS expertise directly with platform-specific CLI
   - **Automation** -- Focus on NX-API, gRPC/gNMI, checkpoint/rollback

2. **Identify version** -- Determine NX-OS version. `F` (Feature) vs `M` (Maintenance) releases have different support lifecycles and feature sets.

3. **Identify platform** -- Nexus 9300 (fixed leaf) vs 9500 (modular spine) vs 9808 (ultra-scale). ASIC generation matters for feature support.

4. **Load context** -- Read the relevant reference file for deep knowledge.

5. **Recommend** -- Provide actionable guidance with NX-OS CLI. Always include `feature` enablement commands.

6. **Verify** -- Suggest validation with specific show commands.

## Core Architecture

### Modular Design

NX-OS uses a microkernel with all major functions as separate processes communicating via an internal message bus:

- Process crashes do not impact forwarding -- hardware continues forwarding during control process restarts
- Graceful Restart (GR) for BGP, OSPF, IS-IS maintains adjacencies during supervisor failover
- Stateful Switchover (SSO) for active-standby supervisor failover in <30 seconds
- Features must be explicitly enabled: `feature bgp`, `feature vpc`, `feature nv overlay`, etc.

### VXLAN/EVPN Architecture

BGP EVPN (RFC 7432) provides the control plane for VXLAN:

| Route Type | Purpose |
|---|---|
| Type-1 | Ethernet Auto-Discovery (multi-homing) |
| Type-2 | MAC/IP Advertisement (host routes) |
| Type-3 | Inclusive Multicast (BUM traffic) |
| Type-4 | Ethernet Segment (DF election) |
| Type-5 | IP Prefix Route (inter-subnet routing) |

**Symmetric IRB** is the recommended forwarding model:
- Both ingress and egress VTEPs route
- Per-VRF L3 VNI carries routed traffic
- Anycast gateway (same IP/MAC on all leaves) via `fabric forwarding mode anycast-gateway`
- ARP suppression via `ip arp suppression`

### vPC (Virtual Port-Channel)

vPC enables two Nexus switches to present a single logical port-channel to downstream devices:

**Critical components:**
- Peer-link: carries control traffic and backup data (recommend 2x100G port-channel)
- Peer-keepalive: heartbeat via management VRF (UDP 3200)
- vPC domain: shared identifier tying two peers together

**Essential settings:**
- `peer-gateway` -- proxy ARP for peer's MAC
- `layer3 peer-router` -- route through peer (avoids orphan port issues)
- `auto-recovery reload-delay 300` -- wait 5 min before auto-recovery
- `delay restore 150` -- prevent routing black-holes during reload
- `ip arp synchronize` -- sync ARP tables between peers

### ACI Mode vs Standalone

- **Standalone NX-OS**: CLI-managed, supports all protocols (BGP, OSPF, VXLAN/EVPN, vPC)
- **ACI mode**: APIC-managed via OPFlex; policy model (Tenants > App Profiles > EPGs > Contracts)
- Switching between modes requires OS reinstall (destructive operation)
- ACI image: `aci-n9000-dk9.*.bin`; NX-OS image: `nxos64-cs.*.bin`

### NX-API

Three interfaces for programmatic access:

| Interface | Method | Output |
|---|---|---|
| NX-API CLI | HTTP POST with CLI commands | JSON, XML, or text |
| NX-API REST | REST on DME object model | JSON |
| NX-API JSON-RPC | JSON-RPC 2.0 batch commands | JSON |

Enable: `feature nxapi` then `nxapi https port 443`

### Nexus Dashboard

| Service | Function |
|---|---|
| NDFC (Fabric Controller) | Fabric provisioning, VXLAN EVPN automation |
| NDI (Insights) | Telemetry analytics, anomaly detection |
| NDO (Orchestrator) | Multi-site/multi-fabric policy orchestration |

### Checkpoint and Rollback

```
checkpoint PRE-CHANGE                      # Create snapshot
show diff rollback-patch checkpoint PRE-CHANGE running-config
rollback running-config checkpoint PRE-CHANGE atomic
```

Maximum 10 user checkpoints per device. Atomic mode is safer for production.

## Common Pitfalls

1. **Missing `feature` commands** -- NX-OS requires explicit feature enablement. `feature bgp`, `feature nv overlay`, `nv overlay evpn`, `feature vpc`, `feature lacp`, `feature vn-segment-vlan-based` must all be enabled before configuration.
2. **vPC consistency check failures** -- Config mismatch between vPC peers causes port suspension. Always verify with `show vpc consistency-parameters global` before changes.
3. **vPC peer-keepalive over peer-link** -- If peer-keepalive travels over the peer-link and the peer-link fails, both switches think the peer is dead. Always use dedicated management VRF for keepalive.
4. **VXLAN MTU** -- VXLAN adds 50 bytes. Set all fabric links to MTU 9216. Fragmented VXLAN packets cause performance issues.
5. **NVE source interface** -- `interface nve1` must use a loopback as source-interface. The loopback must be advertised in the underlay IGP.
6. **Anycast gateway MAC mismatch** -- `fabric forwarding anycast-gateway-mac` must be identical on all leaf switches in the fabric.

## Version-Specific Guidance

| Version | Reference | What's version-specific |
|---|---|---|
| 10.5 | `references/versions/10.5.md` | Current recommended Maintenance release (10.5(5)M); production baseline |
| 10.6 | `references/versions/10.6.md` | Current Feature release (10.6(2)F); SRv6, IPv6 underlay, 400G ZR |

## Reference Files

- `references/architecture.md` -- Modular arch, VXLAN/EVPN, ACI vs standalone, NX-API, Nexus Dashboard, telemetry
- `references/diagnostics.md` -- Show commands for routing, VXLAN/NVE, EVPN, vPC, interfaces, platform
- `references/best-practices.md` -- DC fabric design, vPC configuration, ECMP, security hardening, upgrade procedures