Gke

---
name: gke
description: "Expert coverage of Google Kubernetes Engine (GKE): Autopilot, Standard mode, Config Sync, Policy Controller, GKE Enterprise, multi-cluster, and Node Auto Provisioning. Use for \"GKE\", \"Google Kubernetes Engine\", \"GKE Autopilot\", \"Config Sync\", \"Policy Controller\", \"GKE Enterprise\", \"Autopilot mode\"."
license: MIT
---

# Google GKE

This skill covers Google Kubernetes Engine (GKE), including:

- Autopilot mode (recommended for most workloads)
- Standard mode (full control over nodes)
- Config Sync (GitOps at fleet scale)
- Policy Controller (OPA/Gatekeeper-based policy enforcement)
- GKE Enterprise features (multi-cluster, fleet management)
- Node Auto Provisioning (NAP)
- Networking (VPC-native, Dataplane V2/Cilium, Gateway API)
- Security (Binary Authorization, Workload Identity Federation, GKE Sandbox)
- Storage (Persistent Disk CSI, Filestore CSI)
- Observability (GKE-integrated Cloud Monitoring, Managed Prometheus)

## How to Approach Tasks

1. **Classify** the request:
   - **Architecture** -- Load `references/architecture.md` for Autopilot vs Standard, networking, enterprise features
   - **Mode selection** -- Autopilot vs Standard decision guidance
   - **GitOps/Policy** -- Config Sync, Policy Controller configuration
   - **Multi-cluster** -- Fleet management, MCS, multi-cluster ingress
   - **Troubleshooting** -- Node issues, networking, IAM errors

2. **Identify mode** -- Autopilot or Standard? GKE Enterprise or standalone? Ask if unclear.

3. **Load context** -- Read the reference file for deep technical detail.

4. **Apply** -- Provide gcloud CLI, Terraform, or kubectl examples as appropriate.

5. **Validate** -- Suggest `gcloud container clusters describe`, `kubectl get nodes`, Cloud Monitoring dashboards.

## Autopilot vs Standard

| Dimension | Autopilot | Standard |
|-----------|-----------|---------|
| Node management | Fully managed by Google | User-managed node pools |
| Billing | Per pod (CPU/memory/GPU/storage) | Per node (VM cost) |
| Security | Hardened by default (Restricted PSS) | Configurable |
| Node SSH | No | Yes |
| Custom node pools | No | Yes |
| DaemonSets | Limited (Google-managed only by default) | Full support |
| Privileged pods | No | Yes |
| Max pods/node | Managed | Configurable |
| GPU support | Yes (auto-provisioned) | Yes (dedicated node pools) |
| Cost optimization | Automatic (right-sized pods) | Manual |

**Autopilot is recommended** for most workloads in 2025-2026. Choose Standard when you need: SSH access to nodes, custom node images, privileged containers, specific kernel parameters, or DaemonSets with host access.

```bash
# Create Autopilot cluster
gcloud container clusters create-auto prod-cluster \
  --region=us-central1 \
  --release-channel=regular

# Create Standard cluster
gcloud container clusters create prod-cluster \
  --region=us-central1 \
  --num-nodes=3 \
  --machine-type=e2-standard-4 \
  --enable-autoscaling --min-nodes=1 --max-nodes=10 \
  --release-channel=regular \
  --workload-pool=PROJECT_ID.svc.id.goog
```

## Autopilot Details

### Resource Management

Autopilot enforces resource requests on all pods. If requests are not specified, defaults are applied:
- Default CPU request: 500m
- Default memory request: 2Gi
- Default ephemeral storage request: 1Gi

Pods are billed based on their resource requests (not actual usage). Right-sizing requests is critical for cost control.

### Compute Classes

Autopilot supports compute classes for workload-specific hardware:

```yaml
metadata:
  annotations:
    cloud.google.com/compute-class: "Scale-Out"    # general, Scale-Out, Balanced, Performance
spec:
  nodeSelector:
    cloud.google.com/compute-class: "Scale-Out"
```

| Class | Best For |
|-------|----------|
| General-purpose | Default, most workloads |
| Scale-Out | High-density, cost-sensitive |
| Balanced | Production, balanced CPU/memory |
| Performance | CPU-intensive, low-latency |

### Spot Pods (Autopilot)

```yaml
spec:
  nodeSelector:
    cloud.google.com/gke-spot: "true"
  tolerations:
  - key: cloud.google.com/gke-spot
    operator: Equal
    value: "true"
    effect: NoSchedule
  terminationGracePeriodSeconds: 25   # spot pods get 25s notice
```

## GKE Enterprise

GKE Enterprise is a paid tier that adds fleet-wide management capabilities:

### Config Sync

GitOps engine that synchronizes cluster configuration from Git repositories:

```yaml
apiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
  name: root-sync
  namespace: config-management-system
spec:
  sourceFormat: unstructured
  git:
    repo: https://github.com/myorg/k8s-config
    branch: main
    dir: clusters/production
    auth: ssh
    secretRef:
      name: git-creds
  override:
    reconcileTimeout: 5m
```

```yaml
# RepoSync: namespace-scoped sync (delegated to app teams)
apiVersion: configsync.gke.io/v1beta1
kind: RepoSync
metadata:
  name: repo-sync
  namespace: team-alpha
spec:
  sourceFormat: unstructured
  git:
    repo: https://github.com/myorg/team-alpha-config
    branch: main
    dir: config
    auth: ssh
```

**Config Sync features**:
- Drift detection and auto-remediation
- Hierarchical config (cluster-level + namespace-level)
- OCI artifact support (sync from OCI registries, not just Git)
- Multi-cluster: same RootSync deployed to all fleet clusters

### Policy Controller

OPA/Gatekeeper-based policy enforcement:

```yaml
# Constraint: require labels
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: require-team-label
spec:
  match:
    kinds:
    - apiGroups: [""]
      kinds: ["Namespace"]
  parameters:
    labels:
    - key: team
    - key: cost-center
```

**Policy bundles**: pre-built policy libraries (CIS Benchmark, Pod Security Standards, Cost management).

### Multi-Cluster

| Feature | Description |
|---------|-------------|
| Fleet | Logical grouping of clusters for centralized management |
| Multi-Cluster Services (MCS) | Export services across clusters with unified DNS |
| Multi-Cluster Ingress (MCI) | Global L7 load balancing across regional clusters |
| Fleet RBAC | Centralized role-based access across all fleet clusters |

```yaml
# Export a service to the fleet
apiVersion: networking.gke.io/v1
kind: ServiceExport
metadata:
  name: myapp
  namespace: production
```

Other clusters in the fleet can access this service via `myapp.production.svc.clusterset.local`.

## Node Auto Provisioning (NAP)

GKE's auto-provisioning creates new node pools automatically when no existing pool satisfies pending pod requirements:

```bash
gcloud container clusters update prod-cluster \
  --enable-autoprovisioning \
  --min-cpu=1 --max-cpu=100 \
  --min-memory=1 --max-memory=400 \
  --autoprovisioning-scopes=https://www.googleapis.com/auth/cloud-platform
```

NAP creates ephemeral node pools optimized for the specific workload. Unlike Karpenter, NAP uses Cluster Autoscaler under the hood and creates node pools (not individual nodes).

## Networking

### Dataplane V2 (Cilium)

GKE's default network dataplane uses Cilium/eBPF:
- eBPF-based service load balancing (replaces kube-proxy)
- Native NetworkPolicy enforcement
- Built-in network policy logging
- FQDN-based NetworkPolicy support

```bash
gcloud container clusters create prod-cluster \
  --enable-dataplane-v2
```

### Gateway API

GKE provides a native Gateway controller:

```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: external
  namespace: infra
spec:
  gatewayClassName: gke-l7-global-external-managed
  listeners:
  - name: https
    port: 443
    protocol: HTTPS
    tls:
      certificateRefs:
      - name: my-cert
```

GatewayClasses: `gke-l7-global-external-managed` (Global External), `gke-l7-regional-external-managed` (Regional External), `gke-l7-rilb` (Internal).

## Security

### Workload Identity Federation

```bash
# Enable on cluster
gcloud container clusters update prod-cluster \
  --workload-pool=PROJECT_ID.svc.id.goog

# Bind KSA to GSA
gcloud iam service-accounts add-iam-policy-binding \
  GSA_NAME@PROJECT_ID.iam.gserviceaccount.com \
  --role=roles/iam.workloadIdentityUser \
  --member="serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]"
```

```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: myapp-sa
  namespace: production
  annotations:
    iam.gke.io/gcp-service-account: GSA_NAME@PROJECT_ID.iam.gserviceaccount.com
```

### GKE Sandbox (gVisor)

Runs pods in a gVisor sandbox for defense-in-depth:

```yaml
spec:
  runtimeClassName: gvisor
  containers:
  - name: untrusted-app
    image: untrusted:latest
```

### Binary Authorization

Enforce deploy-time container image policies:

```bash
gcloud container binauthz policy import policy.yaml
```

Only allows images signed by trusted attestors. Prevents deploying unscanned or unsigned images.

## Reference Files

- `references/architecture.md` -- Autopilot internals, Standard mode details, Config Sync architecture, Policy Controller deep dive, multi-cluster networking, storage, security model. Read for architecture and design questions.