✦

Technology Geek

18+ YEARS AMONG THE CLOUDS

✦ TRANSMISSION

OCI Resource Manager: Terraform State Management and Stack Automation

Running Terraform locally works until it does not. State files live on someone’s laptop. Two engineers run apply at the same time and corrupt the state. Nobody knows which version of the configuration was last applied. These are not edge cases. They happen on every team that grows past one or two engineers managing the same infrastructure.

OCI Resource Manager is Oracle’s managed Terraform service. It stores state inside OCI, handles locking, provides a full job execution history, detects drift between your configuration and the live infrastructure, and integrates with OCI IAM so infrastructure changes get the same access control applied to everything else. This post covers creating stacks from Git repositories, running plan and apply jobs, detecting drift, using private endpoints, and wiring Resource Manager into a GitLab CI pipeline.

How Resource Manager Works

A Stack is the Resource Manager equivalent of a Terraform workspace. It holds a configuration source, the variable values for that configuration, and the job history showing every plan, apply, and destroy that has run against it. The state file is stored inside OCI, encrypted with your Vault key, and locked during job execution so concurrent applies are impossible.

Step 1: IAM Policy

resource "oci_identity_policy" "resource_manager_policy" {
  compartment_id = var.compartment_id
  name           = "resource-manager-policy"
  description    = "Permissions for OCI Resource Manager"

  statements = [
    "Allow group ${var.platform_group} to manage orm-stacks in compartment id ${var.compartment_id}",
    "Allow group ${var.platform_group} to manage orm-jobs in compartment id ${var.compartment_id}",
    "Allow group ${var.platform_group} to manage orm-config-source-providers in compartment id ${var.compartment_id}",
    "Allow group ${var.dev_group} to read orm-stacks in compartment id ${var.compartment_id}",
    "Allow group ${var.dev_group} to read orm-jobs in compartment id ${var.compartment_id}",
    "Allow service oci-orm to manage all-resources in compartment id ${var.compartment_id}"
  ]
}

Step 2: Connect GitLab as Configuration Source

resource "oci_vault_secret" "gitlab_token" {
  compartment_id = var.compartment_id
  vault_id       = var.vault_id
  key_id         = var.vault_key_id
  secret_name    = "gitlab-resource-manager-token"

  secret_content {
    content_type = "BASE64"
    content      = base64encode(var.gitlab_access_token)
  }
}

resource "oci_resource_manager_configuration_source_provider" "gitlab_source" {
  compartment_id              = var.compartment_id
  display_name                = "gitlab-infrastructure-repo"
  config_source_provider_type = "GITLAB_ACCESS_TOKEN"
  api_endpoint                = "https://gitlab.example.com"
  secret_id                   = oci_vault_secret.gitlab_token.id
}

Step 3: Create a Stack from Git

resource "oci_resource_manager_stack" "production_network" {
  compartment_id = var.compartment_id
  display_name   = "production-network-stack"
  description    = "VCN, subnets, NSGs, and gateways for production"

  config_source {
    config_source_type               = "GIT_CONFIG_SOURCE"
    configuration_source_provider_id = oci_resource_manager_configuration_source_provider.gitlab_source.id
    repository_url                   = "https://gitlab.example.com/platform/oci-network.git"
    branch_name                      = "main"
    working_directory                = "/terraform/network"
  }

  variables = {
    compartment_id     = var.compartment_id
    region             = var.region
    vcn_cidr           = "10.0.0.0/16"
    environment        = "production"
    enable_nat_gateway = "true"
  }

  terraform_version = "1.5.x"

  defined_tags = {
    "Operations.Environment" = "production"
    "Operations.ManagedBy"   = "terraform"
  }
}

output "stack_id" {
  value = oci_resource_manager_stack.production_network.id
}

Step 4: Running Plan and Apply Jobs

# Create a plan job
oci resource-manager job create-plan-job \
  --stack-id ${STACK_ID} \
  --display-name "plan-$(date +%Y%m%d-%H%M%S)"

# Wait for plan completion and check state
oci resource-manager job get \
  --job-id ${PLAN_JOB_ID} \
  --query 'data.{state:"lifecycle-state", result:"lifecycle-details"}'

# Review the plan output
oci resource-manager job get-job-logs \
  --job-id ${PLAN_JOB_ID} \
  --query 'data.items[*].message' \
  --output json | jq -r '.[]'

# Apply using the plan output
oci resource-manager job create-apply-job \
  --stack-id ${STACK_ID} \
  --execution-plan-strategy FROM_JOB_ID \
  --execution-plan-job-id ${PLAN_JOB_ID} \
  --display-name "apply-$(date +%Y%m%d-%H%M%S)"

Step 5: Drift Detection

Drift detection runs a plan against the current live infrastructure and compares it to the last known state. If someone made a manual change through the OCI Console or CLI, drift detection surfaces it so you can decide whether to reconcile it or update your Terraform configuration to reflect the intended change.

# Run a drift detection plan
oci resource-manager job create-plan-job \
  --stack-id ${STACK_ID} \
  --display-name "drift-check-$(date +%Y%m%d)"

# Any resource showing a change is drift from the last apply
oci resource-manager job get-job-logs \
  --job-id ${DRIFT_JOB_ID} \
  --query 'data.items[?contains(message, `will be`) || contains(message, `must be replaced`)].message' \
  --output json | jq -r '.[]'

# Automate weekly drift checks via Events and Functions
resource "oci_events_rule" "weekly_drift_check" {
  compartment_id = var.compartment_id
  display_name   = "weekly-drift-detection"
  is_enabled     = true

  condition = jsonencode({
    eventType = ["com.oraclecloud.scheduler.fired"]
    data = { scheduleName = ["weekly-drift-check"] }
  })

  actions {
    actions {
      action_type = "FAAS"
      is_enabled  = true
      function_id = var.drift_check_function_id
    }
  }
}

Step 6: Private Endpoint for Private Subnet Resources

resource "oci_resource_manager_private_endpoint" "production_pe" {
  compartment_id = var.compartment_id
  display_name   = "resource-manager-private-endpoint"
  vcn_id         = var.vcn_id
  subnet_id      = var.private_subnet_id
  nsg_id_list    = [var.resource_manager_nsg_id]
}

When your Terraform configuration creates or manages resources in private subnets, Resource Manager needs network access to those resources during job execution. The private endpoint allows the Terraform runner inside Resource Manager to reach your VCN without traffic crossing the public internet.

Step 7: GitLab CI Pipeline Integration

# .gitlab-ci.yml
stages:
  - plan
  - apply

variables:
  STACK_ID: "ocid1.ormstack.oc1..your-stack-ocid"
  OCI_REGION: "me-jeddah-1"

.oci_setup: &oci_setup
  before_script:
    - mkdir -p ~/.oci
    - |
      cat > ~/.oci/config < /tmp/oci_key.pem
    - chmod 600 /tmp/oci_key.pem

terraform_plan:
  stage: plan
  image: ghcr.io/oracle/oci-cli:latest
  <> plan.env
      while true; do
        STATE=$(oci resource-manager job get \
          --job-id ${PLAN_JOB} \
          --query 'data."lifecycle-state"' --raw-output)
        [[ "${STATE}" == "SUCCEEDED" ]] && break
        [[ "${STATE}" == "FAILED" ]] && exit 1
        sleep 15
      done
      oci resource-manager job get-job-logs \
        --job-id ${PLAN_JOB} \
        --query 'data.items[*].message' \
        --output json | jq -r '.[]'
  artifacts:
    reports:
      dotenv: plan.env
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

terraform_apply:
  stage: apply
  image: ghcr.io/oracle/oci-cli:latest
  <<: *oci_setup
  needs:
    - job: terraform_plan
      artifacts: true
  script:
    - |
      APPLY_JOB=$(oci resource-manager job create-apply-job \
        --stack-id ${STACK_ID} \
        --execution-plan-strategy FROM_JOB_ID \
        --execution-plan-job-id ${PLAN_JOB_ID} \
        --display-name "gitlab-apply-${CI_PIPELINE_ID}" \
        --query 'data.id' --raw-output)
      while true; do
        STATE=$(oci resource-manager job get \
          --job-id ${APPLY_JOB} \
          --query 'data."lifecycle-state"' --raw-output)
        [[ "${STATE}" == "SUCCEEDED" ]] && break
        [[ "${STATE}" == "FAILED" ]] && exit 1
        sleep 15
      done
  environment: production
  when: manual
  rules:
    - if: $CI_COMMIT_BRANCH == 'main'

The when: manual on the apply stage means a human must explicitly trigger it from the GitLab UI after reviewing the plan output. Plans run automatically on every merge request. This gives you a human review gate without requiring anyone to run Terraform locally or manage credentials on developer machines.

Step 8: Job History and Audit

# List all jobs for a stack
oci resource-manager job list \
  --stack-id ${STACK_ID} \
  --sort-by TIME_CREATED \
  --sort-order DESC \
  --query 'data.items[*].{id:id, name:"display-name", operation:operation, state:"lifecycle-state", created:"time-created", by:"created-by"}' \
  --output table

# List all resources managed by a stack
oci resource-manager stack list-associated-resources \
  --stack-id ${STACK_ID} \
  --query 'data.items[*].{resource-id:"resource-id", type:"resource-type", region:region}' \
  --output table

Operational Notes

Use one stack per environment per module. A single stack covering all your production infrastructure creates a blast radius problem: a failed apply can leave half your infrastructure in an inconsistent state. Separate stacks for networking, compute, and database layers mean a failed database apply does not touch the network stack.

Store sensitive variable values in OCI Vault and reference them as secret references in the stack configuration. Resource Manager reads secret values from Vault at job execution time so credentials never appear in the stack variable store or the job logs.

Run drift detection on a weekly schedule as a minimum. A drift that goes undetected for a month is a month of configuration that does not match your Terraform code. Reconciling it requires careful analysis of what changed deliberately versus accidentally, which gets harder the longer you wait.

Regards,
Osama

#OCI #OracleCloud #ResourceManager #Terraform #IaC #OracleCloudInfrastructure #DevOps #PlatformEngineering #CloudArchitecture #TechBlog #Oracle #GitOps #InfrastructureAsCode #GitLab #CICD #DriftDetection #CloudAutomation #StateManagement #DevSecOps #PlatformTeam

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.