Most AKS clusters I get called in to review were created with defaults and grown by accident. One node pool running everything, no availability zones, kubenet networking chosen years ago for reasons nobody remembers, and an upgrade story that amounts to hoping. Designing the cluster properly up front costs one afternoon. This is what that afternoon should produce.

Network Model: Use Azure CNI Overlay

The old tradeoff was kubenet (cheap on IPs, but limited) versus classic Azure CNI (every pod gets a VNet IP, which eats address space alive). Azure CNI Overlay resolved it: nodes get VNet IPs, pods get IPs from a private overlay CIDR that does not consume VNet space, and you keep full network policy support. For new clusters there is very little reason to pick anything else. Pair it with Cilium as the data plane for better performance and richer network policies.

resource "azurerm_kubernetes_cluster" "this" {
  name                      = "aks-payments-prod"
  location                  = "westeurope"
  resource_group_name       = azurerm_resource_group.aks.name
  dns_prefix                = "payments"
  kubernetes_version        = "1.32"
  sku_tier                  = "Standard"
  automatic_upgrade_channel = "patch"

  network_profile {
    network_plugin      = "azure"
    network_plugin_mode = "overlay"
    network_data_plane  = "cilium"
    pod_cidr            = "192.168.0.0/16"
    service_cidr        = "172.16.0.0/16"
    dns_service_ip      = "172.16.0.10"
    outbound_type       = "userDefinedRouting"
  }

  default_node_pool {
    name                 = "system"
    vm_size              = "Standard_D4ds_v5"
    zones                = ["1", "2", "3"]
    auto_scaling_enabled = true
    min_count            = 3
    max_count            = 6
    only_critical_addons_enabled = true
    vnet_subnet_id       = azurerm_subnet.aks_nodes.id
  }

  identity {
    type = "SystemAssigned"
  }
}

Two details matter here. The outbound type is userDefinedRouting so all egress goes through the hub firewall instead of a default load balancer public IP. And the system pool is tainted for critical addons only, which brings us to node pools.

Node Pool Strategy

The minimum sane layout is one system pool (three nodes across three zones, nothing but kube-system) and one or more user pools for workloads. Add dedicated pools when a workload class needs different hardware or isolation: memory optimized for caches, GPU for inference, spot pools for batch. Keep pools homogeneous and let the cluster autoscaler do the sizing.

resource "azurerm_kubernetes_cluster_node_pool" "apps" {
  name                  = "apps"
  kubernetes_cluster_id = azurerm_kubernetes_cluster.this.id
  vm_size               = "Standard_D8ds_v5"
  zones                 = ["1", "2", "3"]
  auto_scaling_enabled  = true
  min_count             = 3
  max_count             = 20
  max_pods              = 100
  vnet_subnet_id        = azurerm_subnet.aks_nodes.id
}

resource "azurerm_kubernetes_cluster_node_pool" "batch_spot" {
  name                  = "batchspot"
  kubernetes_cluster_id = azurerm_kubernetes_cluster.this.id
  vm_size               = "Standard_D8ds_v5"
  priority              = "Spot"
  eviction_policy       = "Delete"
  spot_max_price        = -1
  auto_scaling_enabled  = true
  min_count             = 0
  max_count             = 30
  node_taints           = ["kubernetes.azure.com/scalesetpriority=spot:NoSchedule"]
}

Availability and Upgrades

Zones on every pool, Standard tier for the uptime SLA on the API server, and pod topology spread constraints in your workloads so replicas actually land in different zones. For upgrades, pin the minor version in Terraform and let the patch channel handle CVE fixes automatically inside a maintenance window. Minor upgrades should be a deliberate event: bump the control plane first, then roll pools one at a time with surge capacity, and always test on a staging cluster running the identical configuration. AKS minor versions leave support roughly a year after release, so schedule two upgrade cycles per year in the team calendar rather than being forced into one.

Tomorrow’s post takes this cluster and hardens it: workload identity, pod security, image policy, and Defender for Containers.

Cheers
Osama