Dc Fabric

---
name: dc-fabric
description: "Cross-platform comparison and architecture guidance for data center fabric technologies including Cisco ACI, VMware NSX, and open VXLAN/EVPN fabrics: spine-leaf architecture, overlay/underlay design, policy-driven networking, and multi-site DC interconnect. Use for \"data center fabric\", \"spine-leaf\", \"ACI vs NSX\", \"overlay underlay\", \"DC network design\", \"fabric architecture\", \"CLOS topology\", \"DC interconnect\", \"multi-pod\", \"multi-site fabric\". Do NOT use for cisco-aci-, vmware-nsx-, sonic-, dent-, or containerlab-specific configuration -- use that technology's own skill."
license: MIT
---

# DC Fabric

This skill covers data center fabric technologies, providing deep expertise in spine-leaf architectures, overlay/underlay design, policy-driven SDN platforms, and multi-site data center interconnect. Sibling technology skills provide platform implementation details.

## When to Use This Skill vs. a Technology Skill

**Use this skill when the question is cross-platform, comparative, or architectural:**
- "Should I use ACI or NSX for my data center?"
- "Explain overlay vs underlay in a spine-leaf fabric"
- "How do I design a multi-site DC interconnect?"
- "Compare policy models across ACI, NSX, and open EVPN"
- "What are the trade-offs between hardware SDN and software SDN?"
- "How many spines do I need for 200 racks?"

**Route to a technology skill when the question is platform-specific:**
- "Configure an EPG contract in ACI" --> the `cisco-aci` skill
- "Set up DFW rules in NSX" --> the `vmware-nsx` skill
- "APIC 6.1 ESG migration" --> the `cisco-aci` skill's `references/versions/6.1.md`
- "NSX 4.2 TEP HA with BFD" --> the `vmware-nsx` skill's `references/versions/4.2.md`

## How to Approach Tasks

1. **Classify** the request:
   - **Architecture / Design** -- Load `references/concepts.md` for spine-leaf, overlay/underlay, and fabric fundamentals
   - **Platform comparison** -- Compare ACI, NSX, and open fabric, then route for implementation
   - **Troubleshooting** -- Identify the fabric platform, then consult the appropriate technology skill
   - **Migration** -- Assess source/target platforms and overlay technologies
   - **Capacity planning** -- Apply CLOS topology math and oversubscription analysis

2. **Gather context** -- Fabric type (ACI, NSX, open EVPN), scale (leaf count, endpoints, tenants), traffic patterns (east-west vs north-south), multi-site requirements, existing hypervisor platform

3. **Analyze** -- Apply DC fabric principles. Consider oversubscription ratios, failure domain isolation, operational complexity, and vendor lock-in.

4. **Recommend** -- Provide specific, actionable guidance with platform trade-offs

5. **Qualify** -- State assumptions about scale, traffic profile, and operational maturity

## Platform Comparison

### When to Use Each Platform

| Platform | Best For | Key Strengths |
|---|---|---|
| **Cisco ACI** | Large enterprise/SP DCs, Cisco-heavy environments | Policy model (EPG/contract), APIC automation, multi-site NDO, Cloud ACI |
| **VMware NSX** | vSphere-centric DCs, micro-segmentation priority | DFW kernel-level firewall, T0/T1 logical routing, VCF integration |
| **Open EVPN (NX-OS/EOS)** | Multi-vendor fabrics, maximum flexibility | Standards-based, no controller lock-in, CLI/API driven |

### Architecture Comparison

| Aspect | Cisco ACI | VMware NSX | Open EVPN |
|---|---|---|---|
| **Controller** | APIC cluster (3+ physical/virtual) | NSX Manager cluster (3 VMs) | None (distributed control plane) |
| **Overlay protocol** | VXLAN (OpFlex-managed) | Geneve | VXLAN |
| **Control plane** | OpFlex (declarative) | NSX Manager (policy API) | BGP EVPN (RFC 7432) |
| **Policy model** | Tenant/VRF/BD/EPG/Contract | T0/T1/Segment/DFW Groups | VRF/VNI/ACL (manual) |
| **Micro-segmentation** | EPG contracts (whitelist) | DFW rules (kernel-level) | ACLs on leaf (limited) |
| **Multi-site** | NDO (Multi-Pod/Multi-Site) | Federation (Global Manager) | BGW with EVPN re-origination |
| **Underlay** | IS-IS (auto-provisioned) | Physical network (independent) | eBGP or OSPF/IS-IS (manual) |
| **Hardware lock-in** | Nexus 9000 only | Any x86 hypervisor host | Multi-vendor (Cisco, Arista, etc.) |

### Operational Model Comparison

| Aspect | ACI | NSX | Open EVPN |
|---|---|---|---|
| **Day-0** | APIC discovery, fabric bring-up | NSX Manager deploy, VIB install | Switch-by-switch config |
| **Day-1** | GUI/API tenant provisioning | Policy API segment/DFW creation | CLI/automation per leaf |
| **Day-2** | Health scores, ELAM, contract stats | Flow visualization, DFW stats | Traditional show/debug |
| **Learning curve** | Steep (policy model is unique) | Moderate (vSphere familiarity helps) | Low (standard protocols) |
| **Automation** | REST API, Terraform ACI provider, Ansible | REST API, Terraform NSX provider | Ansible, Terraform, Nornir |

## Spine-Leaf Design Guidance

### Sizing a CLOS Fabric

**Two-tier (spine-leaf):**
- Maximum leaf switches = number of spine ports
- Each leaf connects to every spine (full mesh)
- Oversubscription ratio = (total leaf downlink BW) / (total leaf uplink BW)
- Target: 3:1 or better for general workloads; 1:1 for storage/HPC

**Three-tier (super-spine):**
- Required when leaf count exceeds spine port density
- Super-spines interconnect spine pods
- Each spine connects to every super-spine

### Oversubscription Calculation

```
Leaf downlinks: 48 x 10G = 480 Gbps
Leaf uplinks:   6 x 100G = 600 Gbps
Ratio: 480:600 = 0.8:1 (non-blocking)

Leaf downlinks: 48 x 25G = 1200 Gbps
Leaf uplinks:   6 x 100G = 600 Gbps
Ratio: 1200:600 = 2:1 (acceptable for most workloads)
```

### MTU Considerations

VXLAN adds 50 bytes of overhead. All fabric links (leaf-to-spine, spine-to-spine) must support jumbo frames:
- Minimum fabric MTU: 9214 bytes
- ACI: Fabric MTU auto-configured by APIC
- NSX: Physical underlay MTU must be set manually on physical switches
- Open EVPN: Set MTU on all fabric interfaces manually

## Multi-Site Design Patterns

| Pattern | Use Case | Technology |
|---|---|---|
| **Stretched L2** | VM mobility across sites | ACI Multi-Site, NSX Federation stretched segments |
| **L3 DCI** | Independent failure domains | EVPN Type-5 re-origination at border gateways |
| **Active-Active DC** | Maximum availability | GSLB + independent fabrics with L3 interconnect |
| **DR/BC** | Disaster recovery | Async replication + stretched VLANs for failover |

**Best practice**: Prefer L3 DCI over stretched L2 whenever possible. Stretched L2 across sites increases the failure domain and introduces latency-sensitive BUM traffic across the WAN.

## Technology Routing

| Request Pattern | Route To |
|---|---|
| Cisco ACI, APIC, EPG, contracts, OpFlex, policy model | the `cisco-aci` skill |
| ACI 6.1, ESG, endpoint security groups | the `cisco-aci` skill's `references/versions/6.1.md` |
| VMware NSX, DFW, T0/T1, Geneve, NSX Manager | the `vmware-nsx` skill |
| NSX 4.2, vDefend, TEP HA, VCF integration | the `vmware-nsx` skill's `references/versions/4.2.md` |
| SONiC, SAI, SwSS, syncd, ConfigDB, whitebox switch, DASH | the `sonic` skill |
| Containerlab, clab, network lab, container topology, vrnetlab | the `containerlab` skill |
| DENT, DentOS, switchdev, enterprise edge NOS, Just Walk Out | the `dent` skill |
| Open EVPN on NX-OS, Nexus standalone | the `cisco-nxos` skill |
| Open EVPN on Arista EOS, CloudVision | the `arista-eos` skill |

## Common Pitfalls

1. **Choosing fabric technology before defining requirements** -- ACI, NSX, and open EVPN solve different problems. Define segmentation, automation, multi-site, and operational maturity requirements before selecting a platform.

2. **Ignoring underlay MTU** -- VXLAN/Geneve overhead causes silent packet drops if fabric MTU is not set to 9214+. This is the most common Day-1 fabric issue.

3. **Over-engineering multi-site** -- Stretched L2 across data centers should be a deliberate architectural decision, not a convenience shortcut. L3 DCI with independent failure domains is almost always more resilient.

4. **Assuming NSX replaces physical networking** -- NSX overlays run on top of physical switches. The underlay still needs proper design (ECMP, MTU, QoS). NSX does not eliminate physical network complexity.

5. **Mixing ACI mode and standalone NX-OS** -- Nexus 9000 in ACI mode cannot run standard NX-OS CLI. Organizations that want both must maintain separate switch pools.

6. **Ignoring east-west security** -- Traditional perimeter firewalls inspect north-south traffic only. DC fabrics need micro-segmentation (ACI contracts, NSX DFW, or host-based firewalls) for east-west traffic between workloads.

## Reference Files

- `references/concepts.md` -- Spine-leaf topology, VXLAN/EVPN overlay vs underlay, ACI vs NSX vs open fabric comparison. Read for architecture and design questions.