Containers get the conference talks, but an enormous amount of real workload still runs on VMs, and Virtual Machine Scale Sets are how VMs stop being pets. A scale set gives you identical instances from a model, spread across fault domains and zones, growing and shrinking on metrics, and upgradeable in place without downtime. The catch is that each of those behaviors has settings that default to something you probably do not want.

Flexible Orchestration and Zones

Use Flexible orchestration mode for anything new; Uniform is the legacy mode. Flexible spreads instances across zones and fault domains with better guarantees, mixes VM sizes and spot with standard instances in one set, and exposes instances as full VMs for the rare cases you need to inspect one.

resource "azurerm_orchestrated_virtual_machine_scale_set" "web" {
  name                        = "vmss-web-prod"
  location                    = "westeurope"
  resource_group_name         = azurerm_resource_group.web.name
  sku_name                    = "Standard_D4ds_v5"
  instances                   = 3
  platform_fault_domain_count = 1
  zones                       = ["1", "2", "3"]
  zone_balance                = true

  os_profile {
    linux_configuration {
      admin_username                  = "azureuser"
      disable_password_authentication = true
      admin_ssh_key {
        username   = "azureuser"
        public_key = tls_private_key.vmss.public_key_openssh
      }
    }
  }

  source_image_id = azurerm_shared_image_version.web.id

  network_interface {
    name    = "nic"
    primary = true
    ip_configuration {
      name      = "ipconf"
      primary   = true
      subnet_id = azurerm_subnet.web.id
      load_balancer_backend_address_pool_ids = [
        azurerm_lb_backend_address_pool.web.id
      ]
    }
  }
}

Note the source image: a versioned image from Azure Compute Gallery, baked by Packer or Image Builder in CI, not a marketplace image plus a twenty minute custom script extension. Immutable images make scale out fast and rollback trivial, because rolling back is deploying the previous image version.

Autoscale Without Flapping

The classic mistake is symmetric thresholds: scale out above 70 percent CPU, in below 65, and the set oscillates all day, each oscillation churning connections. Leave a wide corridor (out at 70, in at 30), make scale in cooldowns long (15 to 30 minutes) while scale out stays fast, and scale out by larger steps than you scale in. Where the workload allows it, schedule based rules beat metric chasing: if traffic reliably triples at 9 AM, be scaled before 9 AM rather than reacting through ten minutes of degraded service. And scale on the metric closest to user experience you have, requests per instance or queue depth from Application Insights, with CPU as the backstop rather than the primary signal.

Rolling Upgrades With Real Health

Rolling upgrade mode replaces instances in batches when the model changes, but it is only as safe as its health signal. Install the Application Health extension and point it at an endpoint that verifies the application actually works, dependencies included, not just that the VM answers TCP. Then configure the rolling policy: batch size around 20 percent, pause between batches, and automatic rollback on health failure. The upgrade then becomes: publish new image version, update the scale set model, and watch batches roll, with the platform halting automatically the moment healthy instance percentage drops below your floor. Combine that with the load balancer draining connections before an instance is removed and deployments become genuinely boring, which is the goal of this entire discipline.

For batch and stateless workloads, add a spot priority mix: a floor of standard instances for baseline with spot instances above it at a fraction of the price, and the scale set replaces evicted spot capacity automatically. Savings of 60 to 80 percent on the flexible portion are routine.

Cheers
Osama