Vmware Nsx

---
name: vmware-nsx
description: "Expert coverage of VMware NSX across all versions: NSX Manager, transport nodes, Distributed Firewall (DFW), T0/T1 gateways, Geneve overlay, Federation, NSX ALB, and vDefend security. Use for \"VMware NSX\", \"NSX Manager\", \"DFW\", \"distributed firewall\", \"T0 gateway\", \"T1 gateway\", \"Geneve\", \"NSX Federation\", \"vDefend\", \"NSX ALB\". Do NOT use for cross-vendor comparison, platform selection, or category-wide architecture -- use the `dc-fabric` skill."
license: MIT
---

# VMware NSX

This skill covers VMware NSX (now under Broadcom) across all supported versions. Areas of expertise include:

- NSX Manager cluster management and control plane
- Transport nodes (host and Edge) and TEP configuration
- Distributed Firewall (DFW / vDefend) micro-segmentation at the hypervisor kernel
- T0 and T1 logical gateways for north-south and east-west routing
- Geneve overlay networking and segment management
- NSX Federation for multi-site deployments
- NSX ALB (Avi Networks) load balancing
- NSX Intelligence for flow visualization and security recommendations
- VDS-based deployment on vSphere 7+
- REST API, Terraform NSX provider, Ansible
- Broadcom licensing changes and VCF integration

When a question is version-specific, consult the matching `references/versions/` file. When the version is unknown, provide general guidance and note where behavior differs.

## How to Approach Tasks

1. **Classify** the request:
   - **Troubleshooting** -- Load `references/diagnostics.md` for Central CLI, API debugging, flow analysis
   - **Security design** -- Apply DFW/Gateway Firewall guidance below
   - **Architecture** -- Load `references/architecture.md` for Manager cluster, transport nodes, T0/T1, Federation
   - **Routing** -- Apply T0/T1 gateway design guidance below
   - **Automation** -- Apply REST API, Terraform, or Ansible guidance

2. **Identify version** -- Determine which NSX version. If unclear, ask. Version matters for feature availability (VDS mode requires 3.2+, vDefend Turbo mode requires 4.1+, TEP HA with BFD requires 4.2.1+).

3. **Load context** -- Read the relevant reference file for deep knowledge.

4. **Analyze** -- Apply NSX-specific reasoning. NSX operates at the hypervisor level; physical network knowledge is still required for the underlay.

5. **Recommend** -- Provide actionable guidance with GUI paths, REST API calls, or Central CLI commands.

6. **Verify** -- Suggest validation steps (transport node status, DFW rule hit counts, BGP neighbor state).

## Core Architecture

### NSX Manager Cluster

Centralized management and control plane deployed as 3 virtual appliances:
- Each node runs both management and control plane functions
- Cluster state synchronized via Raft consensus
- Virtual IP (VIP) configured for single-endpoint access
- Minimum: 1 node (lab only); Production: 3-node cluster

**Manager responsibilities:**
- REST API endpoint for all configuration
- Policy engine translating intent to data plane configuration
- Certificate management (auto-renewal in 4.2.1+)
- Integration with vCenter, Kubernetes, and cloud platforms
- Upgrade orchestration

### Transport Nodes

Hypervisors and Edge appliances participating in the NSX overlay:

**Host Transport Nodes (ESXi):**
- NSX kernel modules (VIBs) installed on each host
- TEP (Tunnel Endpoint) configured for Geneve tunneling
- DFW kernel module enforces firewall rules at each vNIC
- Segment bridge/overlay processing in kernel for line-rate performance

**Edge Transport Nodes:**
- NSX Edge VMs or bare-metal appliances
- Handle north-south traffic through T0/T1 Service Routers (SR)
- Provide NAT, VPN, load balancing, and gateway firewall services
- Deploy in pairs for HA (active-active or active-standby per T0)

**Transport Zones:**
- Overlay transport zone: groups hosts sharing the same Geneve tunnel mesh
- VLAN transport zone: handles physical uplinks for Edge nodes
- A host can belong to multiple transport zones

### VDS vs N-VDS

**N-VDS (legacy):** Separate NSX virtual switch on each host. Replaced in NSX 3.2+.
**VDS (current):** NSX runs on top of the standard vSphere Distributed Switch. Single switch for all vSphere networking and NSX overlay. Recommended for NSX 4.x on vSphere 7+.

Benefits of VDS mode:
- Single switch management for all networking
- Reduced operational complexity
- Consistent uplink management
- vMotion without NSX-specific TEP migration

## Distributed Firewall (DFW)

### Architecture

DFW is NSX's flagship micro-segmentation capability:
- Kernel module on every ESXi host transport node
- Rules enforced at the vNIC level -- traffic inspected before entering the virtual switch
- Stateful packet inspection per VM
- Scales horizontally -- more hosts = more DFW throughput
- No traffic hairpinning to a central firewall appliance

### Rule Organization

```
Security Policy (priority/sequence number)
  Rule 1: Source Group -> Destination Group : Service : Action (Allow/Drop/Reject)
  Rule 2: ...
  ...
Default Rule: Allow-all or Deny-all (operator configurable)
```

**Applied-To:** Scopes a policy or rule to specific VMs, groups, or segments. Reduces rule evaluation overhead on hosts that do not need the rule.

### Groups

Dynamic, attribute-based membership replacing static IP lists:
- **Static members**: Specific VMs, IPs, MAC addresses, segments
- **Dynamic criteria**: VM tags, OS type, VM name pattern, computer name, security tags
- Groups update automatically as VMs are created, tagged, migrated, or deleted

```
Example: Tag = "Production" AND OS = "Linux"
```

### Context Profiles (L7)

Application-layer matching for DFW rules:
- **App-ID**: Identify applications by traffic signature (not just port)
- **FQDN/URL**: Filter by domain name
- **DNS**: Block or allow DNS queries by domain pattern
- Full L7 IDS/IPS requires vDefend ATP add-on

### Rule Design Best Practices

1. **Emergency rules** at the top: Block known-bad IPs/signatures
2. **Infrastructure rules**: Allow shared services (DNS, NTP, AD, DHCP) for all VMs
3. **Application rules**: Per-application policies between tiers (web -> app -> db)
4. **Environment isolation**: Prevent production-to-dev communication
5. **Default deny**: Set the default rule to drop with logging

### Applied-To Optimization

- Without Applied-To, every rule is pushed to every host -- wastes memory and CPU
- Scope rules to the smallest group possible
- Use Applied-To at the policy level (not per-rule) when all rules in a policy apply to the same scope
- Monitor rule count per host: excessive rules degrade DFW performance

## T0 and T1 Gateways

### Tier-0 Gateway

North-south connectivity between NSX overlay and physical network:
- BGP and/or static routing with upstream physical routers
- ECMP across multiple Edge nodes for load balancing
- VRF-Lite: multiple T0 VRFs on a single T0 gateway for multi-tenancy
- Active-Active: all Edge nodes forward traffic (requires stateless services or ECMP-aware upstream)
- Active-Standby: primary Edge node with BFD-based failover

### Tier-1 Gateway

East-west routing between logical segments:
- Advertises connected subnets up to T0 via Service Router (SR) component
- Distributed Router (DR) in each host kernel for high-performance E-W routing
- Multiple T1 gateways connect to a single T0 (common for tenant isolation)
- Attachment point for NSX segments (logical switches)

### Data Path

```
VM-to-VM (same segment): Switched in hypervisor kernel -- no Edge involvement
VM-to-VM (different segment, same T1): Routed via T1 DR in kernel -- no Edge involvement
VM-to-external: T1 DR (kernel) -> T1 SR (Edge) -> T0 SR (Edge) -> Physical uplink
```

**Key insight**: East-west traffic between VMs on different segments is routed entirely in the hypervisor kernel by the T1 Distributed Router. Only north-south traffic traverses Edge nodes. This is why NSX east-west routing performance scales with host count.

## NSX Federation

Centralized management across geographically distributed NSX deployments:

**Components:**
- **Global Manager**: Centralized policy definition across all sites
- **Local Managers**: Per-site NSX Manager clusters executing policy locally
- **Stretched objects**: Segments, Groups, and Gateway Firewall policies stretched across sites

**Capabilities:**
- Define security policies once, enforce consistently across all sites
- Stretched segments for workload mobility between sites
- Federated gateway firewall for inter-site traffic policy
- VRF configuration at Global Manager level (4.2.1+)

**When to use Federation:**
- Multiple data centers requiring consistent DFW policies
- Workload mobility between sites (vMotion/cold migration)
- Centralized security governance with distributed enforcement

## NSX ALB (Avi Networks)

NSX Advanced Load Balancer for L4-L7 load balancing:

**Architecture:**
- **Avi Controller**: 3-node management cluster; integrates with NSX Manager and vCenter
- **Service Engines (SEs)**: Data plane VMs deployed per application; auto-scale
- RESTful API and Terraform provider for automation

**Features:**
- L4-L7 load balancing (TCP/UDP, HTTP/HTTPS)
- SSL/TLS termination and offload
- Web Application Firewall (WAF) with OWASP rule sets
- Application analytics with end-to-end latency metrics
- Multi-cloud (vSphere, AWS, Azure, GCP)
- Kubernetes ingress via AKO (Avi Kubernetes Operator)

**Note**: NSX ALB replaces the deprecated NSX Edge load balancer in VCF environments.

## Broadcom Licensing Impact

- NSX no longer sold standalone; bundled into VMware Cloud Foundation (VCF) SKUs
- VCF bundles: vSphere + vSAN + NSX + Aria in per-core subscription pricing
- Legacy perpetual NSX-T licenses honored but no longer sold to new customers
- NSX ALB licensing included in specific VCF tiers
- Support via Broadcom portal (not VMware MyPortal)
- Documentation at techdocs.broadcom.com

## Automation

### REST API

```bash
# Authenticate
POST https://<nsx-mgr>/api/session/create
Body: j_username=admin&j_password=<password>

# Get all segments
GET https://<nsx-mgr>/policy/api/v1/infra/segments

# Create a segment
PATCH https://<nsx-mgr>/policy/api/v1/infra/segments/web-segment
Body: {"display_name":"web-segment","subnets":[{"gateway_address":"10.1.1.1/24"}],"transport_zone_path":"/infra/sites/default/enforcement-points/default/transport-zones/<tz-id>"}

# Get DFW rules
GET https://<nsx-mgr>/policy/api/v1/infra/domains/default/security-policies

# Get transport node status
GET https://<nsx-mgr>/api/v1/transport-nodes/<id>/state
```

### Terraform

```hcl
provider "nsxt" {
  host                 = "nsx-mgr.example.com"
  username             = "admin"
  password             = var.nsx_password
  allow_unverified_ssl = true
}

resource "nsxt_policy_segment" "web" {
  display_name        = "web-segment"
  transport_zone_path = data.nsxt_policy_transport_zone.overlay.path
  subnet {
    cidr = "10.1.1.1/24"
  }
}

resource "nsxt_policy_group" "web_servers" {
  display_name = "WebServers"
  criteria {
    condition {
      member_type = "VirtualMachine"
      key         = "Tag"
      operator    = "EQUALS"
      value       = "web"
    }
  }
}
```

## Common Pitfalls

1. **Forgetting physical underlay MTU** -- NSX Geneve overlay adds ~54 bytes. Physical switches carrying TEP traffic must have MTU set to 9000+ or encapsulated frames will be silently dropped. NSX does not manage the physical underlay.

2. **DFW rule explosion without Applied-To** -- Without scoping rules via Applied-To, every rule is pushed to every host. In large environments this exhausts host memory and degrades DFW performance. Always scope policies.

3. **Edge node sizing** -- Edge nodes handle all north-south traffic, NAT, VPN, and gateway firewall. Undersized Edge VMs create a bottleneck. Use bare-metal Edge for high-throughput environments.

4. **T0 Active-Active with stateful services** -- Active-Active T0 requires stateless services or ECMP-aware upstream routers. NAT and stateful firewall on T0 require Active-Standby mode.

5. **VCF licensing confusion** -- NSX is no longer a standalone product. New customers must purchase VCF bundles. Existing perpetual license holders should evaluate renewal costs against VCF subscription pricing.

6. **Federation stretched segment latency** -- Stretched segments across sites introduce WAN latency for BUM traffic. Keep stretched L2 to a minimum; prefer L3 connectivity between sites.

7. **Ignoring NSX Manager cluster health** -- NSX Manager runs both management and control plane. If the cluster loses quorum, new DFW rules and segment changes cannot be deployed even though existing policies continue to be enforced.

## Version-Specific Guidance

| Version | Reference | What's version-specific |
|---|---|---|
| 4.2 | `references/versions/4.2.md` | vDefend Turbo mode, TEP HA with BFD, VRF at Global Manager, VCF integration |

## Reference Files

Load these when you need deep knowledge:

- `references/architecture.md` -- Manager cluster internals, transport node lifecycle, DFW pipeline, T0/T1 data path, Federation architecture, N-VDS vs VDS. Read for "how does X work" questions.
- `references/diagnostics.md` -- Central CLI commands, NSX Manager API debugging, DFW rule statistics, overlay connectivity testing, Edge BGP troubleshooting. Read when troubleshooting.