GitOps is the practice of using a Git repository as the single source of truth for cluster state. ArgoCD continuously compares what is in the repository against what is running in Kubernetes and reconciles any drift. On OCI OKE, this means infrastructure (VCNs, node pools) comes from Terraform, and application workloads (Deployments, Services, ConfigMaps) come from a Git-controlled ArgoCD app. This post covers bootstrapping ArgoCD on OKE with Terraform and configuring the app-of-apps pattern for multi-service deployments.

Step 1: OKE Cluster with ArgoCD Namespace

resource "oci_containerengine_cluster" "gitops_cluster" {
  compartment_id     = var.compartment_id
  name               = "gitops-production"
  kubernetes_version = "v1.31.1"
  vcn_id             = var.vcn_id

  endpoint_config {
    is_public_ip_enabled = false
    subnet_id            = var.control_plane_subnet_id
    nsg_ids              = [var.control_plane_nsg_id]
  }

  options {
    service_lb_subnet_ids = [var.lb_subnet_id]
    kubernetes_network_config {
      pods_cidr     = "10.244.0.0/16"
      services_cidr = "10.96.0.0/16"
    }
  }

  defined_tags = { "Operations.Environment" = "production", "Operations.ManagedBy" = "terraform" }
}

resource "oci_containerengine_node_pool" "system_nodes" {
  cluster_id         = oci_containerengine_cluster.gitops_cluster.id
  compartment_id     = var.compartment_id
  name               = "system-node-pool"
  kubernetes_version = "v1.31.1"
  node_shape         = "VM.Standard.E4.Flex"

  node_shape_config {
    ocpus         = 4
    memory_in_gbs = 32
  }

  node_config_details {
    size = 3
    placement_configs {
      availability_domain = data.oci_identity_availability_domains.ads.availability_domains[0].name
      subnet_id           = var.worker_subnet_id
    }
  }

  node_source_details {
    source_type = "IMAGE"
    image_id    = var.oke_worker_image_id
  }

  defined_tags = { "Operations.ManagedBy" = "terraform" }
}

resource "helm_release" "argocd" {
  name             = "argocd"
  repository       = "https://argoproj.github.io/argo-helm"
  chart            = "argo-cd"
  version          = "7.4.0"
  namespace        = "argocd"
  create_namespace = true

  set {
    name  = "server.service.type"
    value = "ClusterIP"
  }

  set {
    name  = "configs.params.server\.insecure"
    value = "true"
  }

  set {
    name  = "repoServer.autoscaling.enabled"
    value = "true"
  }

  depends_on = [oci_containerengine_node_pool.system_nodes]
}

Step 2: App-of-Apps Root Application

resource "kubectl_manifest" "argocd_root_app" {
  yaml_body = yamlencode({
    apiVersion = "argoproj.io/v1alpha1"
    kind       = "Application"
    metadata = {
      name      = "root-app"
      namespace = "argocd"
      finalizers = ["resources-finalizer.argocd.argoproj.io"]
    }
    spec = {
      project = "default"
      source = {
        repoURL        = "https://github.com/yourorg/oke-gitops-config.git"
        targetRevision = "main"
        path           = "apps"
      }
      destination = {
        server    = "https://kubernetes.default.svc"
        namespace = "argocd"
      }
      syncPolicy = {
        automated = {
          prune    = true
          selfHeal = true
        }
      }
    }
  })

  depends_on = [helm_release.argocd]
}

Step 3: ArgoCD Alarm on Out-of-Sync Apps

resource "oci_monitoring_alarm" "argocd_app_out_of_sync" {
  compartment_id        = var.compartment_id
  display_name          = "argocd-application-out-of-sync"
  is_enabled            = true
  metric_compartment_id = var.compartment_id
  namespace             = "oci_oke"
  query                 = "argocd_app_sync_status[5m]{sync_status = 'OutOfSync'}.count() > 0"
  severity              = "WARNING"
  pending_duration      = "PT10M"
  destinations          = [var.ops_notification_topic_id]
  body                  = "One or more ArgoCD applications are out of sync for more than 10 minutes. Check ArgoCD UI for drift detection details and any sync errors."
}

Operational Notes

The app-of-apps pattern lets you manage all ArgoCD applications through ArgoCD itself. The root application points to a directory in your Git repository that contains Application manifests. Adding a new service means committing a new Application manifest to that directory. ArgoCD detects the new file, creates the Application resource, and begins syncing the workload. Removing a service means deleting the Application manifest from the directory, and ArgoCD prunes the associated Kubernetes resources automatically when prune: true is set.

Set selfHeal: true on production applications to automatically revert any manual changes made directly to the cluster with kubectl. Any drift from the Git-controlled desired state is corrected within seconds of detection. This enforces the GitOps contract: the cluster state must match the repository state, and any exception requires a Git commit, not a direct kubectl command.

Regards,
Osama

#OCI #OracleCloud #OKE #GitOps #ArgoCD #Kubernetes #Terraform #IaC #TechBlog #Oracle #DevOps #PlatformEngineering #CloudNative #OracleCloudInfrastructure #AppOfApps #ContinuousDeployment #Helm #CICD #ClusterManagement #DeclarativeDeployment