Amazon S3 Advanced: Replication, Lifecycle Policies, and Intelligent Tiering in Production

Most teams use S3 as a bucket you put files in and retrieve later. That works until you need cross-region disaster recovery, until storage costs start compounding on data nobody is reading, or until a versioning incident fills a bucket with millions of delete markers that nobody cleaned up. S3 has a deep feature set that most teams never use, and most of those features pay for themselves quickly.

In this article I will walk through Cross-Region Replication for disaster recovery, lifecycle policies that reduce storage costs automatically, S3 Intelligent Tiering for unpredictable access patterns, and Object Lock for compliance workloads where data must be immutable.

Cross-Region Replication

Cross-Region Replication copies objects to a bucket in a different region automatically after upload. Objects replicate asynchronously, typically within minutes. Use it for disaster recovery, latency reduction for globally distributed reads, and compliance requirements that mandate data copies in specific geographic regions.

resource "aws_s3_bucket" "source" {
  bucket = "production-data-us-east-1"
}

resource "aws_s3_bucket_versioning" "source" {
  bucket = aws_s3_bucket.source.id
  versioning_configuration { status = "Enabled" }
}

resource "aws_s3_bucket" "replica" {
  provider = aws.eu_west_1
  bucket   = "production-data-eu-west-1"
}

resource "aws_s3_bucket_versioning" "replica" {
  provider = aws.eu_west_1
  bucket   = aws_s3_bucket.replica.id
  versioning_configuration { status = "Enabled" }
}

resource "aws_s3_bucket_replication_configuration" "main" {
  bucket = aws_s3_bucket.source.id
  role   = aws_iam_role.s3_replication.arn

  rule {
    id     = "replicate-all"
    status = "Enabled"

    filter {}

    destination {
      bucket        = aws_s3_bucket.replica.arn
      storage_class = "STANDARD_IA"

      encryption_configuration {
        replica_kms_key_id = aws_kms_key.replica.arn
      }

      replication_time {
        status = "Enabled"
        time { minutes = 15 }
      }

      metrics {
        status = "Enabled"
        event_threshold { minutes = 15 }
      }
    }

    delete_marker_replication {
      status = "Enabled"
    }
  }

  depends_on = [aws_s3_bucket_versioning.source]
}

resource "aws_iam_role" "s3_replication" {
  name = "s3-replication-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "s3.amazonaws.com" }
      Action    = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy" "s3_replication" {
  name = "s3-replication-policy"
  role = aws_iam_role.s3_replication.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = ["s3:GetReplicationConfiguration", "s3:ListBucket"]
        Resource = aws_s3_bucket.source.arn
      },
      {
        Effect = "Allow"
        Action = ["s3:GetObjectVersionForReplication", "s3:GetObjectVersionAcl", "s3:GetObjectVersionTagging"]
        Resource = "${aws_s3_bucket.source.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["s3:ReplicateObject", "s3:ReplicateDelete", "s3:ReplicateTags"]
        Resource = "${aws_s3_bucket.replica.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["kms:Decrypt"]
        Resource = aws_kms_key.source.arn
      },
      {
        Effect = "Allow"
        Action = ["kms:GenerateDataKey"]
        Resource = aws_kms_key.replica.arn
      }
    ]
  })
}

Replication Time Control with a 15-minute SLA is the production setting when you need a predictable RPO for your DR strategy. Standard replication has no SLA. With RTC enabled, AWS guarantees 99.99 percent of objects replicate within 15 minutes and publishes CloudWatch metrics for replication latency. Enable it for anything where your RPO is measured in minutes rather than hours.

Replica storage class set to STANDARD_IA reduces DR bucket costs. Replica data is rarely accessed. Reading from a replica is only necessary during a regional outage, which is a low-frequency event. Storing replicas in STANDARD_IA costs about 45 percent less than STANDARD while still providing millisecond retrieval when you need it.

Lifecycle Policies

resource "aws_s3_bucket_lifecycle_configuration" "main" {
  bucket = aws_s3_bucket.source.id

  rule {
    id     = "application-logs"
    status = "Enabled"

    filter {
      prefix = "logs/"
    }

    transition {
      days          = 30
      storage_class = "STANDARD_IA"
    }

    transition {
      days          = 90
      storage_class = "GLACIER_IR"
    }

    expiration {
      days = 365
    }

    noncurrent_version_transition {
      noncurrent_days = 7
      storage_class   = "GLACIER_IR"
    }

    noncurrent_version_expiration {
      noncurrent_days                   = 30
      newer_noncurrent_versions         = 3
    }
  }

  rule {
    id     = "abort-incomplete-multipart"
    status = "Enabled"

    filter {}

    abort_incomplete_multipart_upload {
      days_after_initiation = 7
    }
  }

  rule {
    id     = "delete-markers-cleanup"
    status = "Enabled"

    filter {}

    expiration {
      expired_object_delete_marker = true
    }
  }
}

The abort_incomplete_multipart_upload rule is one most teams forget. When a multipart upload starts but never completes, the parts accumulate in S3 and you are billed for them. Aborting incomplete uploads after 7 days catches failed or abandoned uploads before they become a silent cost leak. For buckets with heavy multipart upload activity, this rule alone can reduce storage costs by a meaningful amount.

noncurrent_version_expiration with newer_noncurrent_versions = 3 keeps the 3 most recent non-current versions and expires older ones. Without this, versioned buckets accumulate unlimited historical versions of every object. A bucket with active writes and no version expiration will eventually hold orders of magnitude more data in old versions than in current objects.

S3 Intelligent Tiering

resource "aws_s3_bucket_intelligent_tiering_configuration" "main" {
  bucket = aws_s3_bucket.source.id
  name   = "entire-bucket"

  tiering {
    access_tier = "DEEP_ARCHIVE_ACCESS"
    days        = 180
  }

  tiering {
    access_tier = "ARCHIVE_ACCESS"
    days        = 90
  }
}

resource "aws_s3_bucket_lifecycle_configuration" "intelligent_tiering" {
  bucket = aws_s3_bucket.source.id

  rule {
    id     = "move-to-intelligent-tiering"
    status = "Enabled"

    filter {
      prefix = "assets/"
    }

    transition {
      days          = 0
      storage_class = "INTELLIGENT_TIERING"
    }
  }
}

Intelligent Tiering automatically moves objects between access tiers based on actual usage patterns. Objects accessed frequently stay in the frequent access tier at STANDARD pricing. Objects not accessed for 30 days move to infrequent access at 40 percent less. Objects not accessed for 90 days optionally move to Archive Access at 68 percent less. Objects not accessed for 180 days move to Deep Archive Access at 95 percent less. No retrieval fees, no management overhead.

Intelligent Tiering has a per-object monitoring fee of $0.0025 per 1,000 objects per month. For objects smaller than 128KB it is not cost-effective since the monitoring fee can exceed the storage savings. Apply it selectively to prefixes containing large objects with unpredictable access patterns.

Object Lock for Compliance

resource "aws_s3_bucket" "compliance" {
  bucket = "compliance-records"

  object_lock_enabled = true
}

resource "aws_s3_bucket_object_lock_configuration" "compliance" {
  bucket = aws_s3_bucket.compliance.id

  rule {
    default_retention {
      mode  = "COMPLIANCE"
      years = 7
    }
  }
}

COMPLIANCE mode means objects cannot be deleted or overwritten by any user, including the root account, for the retention period. GOVERNANCE mode allows users with the s3:BypassGovernanceRetention permission to override the lock. For regulatory requirements like SEC 17a-4 or FINRA, COMPLIANCE mode is required. Enable it at bucket creation time since Object Lock cannot be enabled on existing buckets.

Closing Thoughts

S3 lifecycle policies, replication, and Intelligent Tiering are the storage equivalent of right-sizing compute: the savings are automatic once configured and they compound over time. Lifecycle policies on log buckets alone typically reduce storage costs by 60 to 80 percent compared to leaving everything in STANDARD indefinitely.

Set up replication before you need disaster recovery. Configure lifecycle rules when you create buckets, not after the costs become visible. Use Intelligent Tiering for asset buckets where access patterns are genuinely unpredictable. Add Object Lock for any data your compliance framework requires to be immutable.

Enjoy the cloud.

Osama

#AWS #AmazonS3 #CloudStorage #CloudArchitecture #Terraform #InfrastructureAsCode #AmazonWebServices #SolutionsArchitect #CloudComputing #DataEngineering #DisasterRecovery #CostOptimization #CloudSecurity #TechBlog #CloudInfrastructure #S3Replication #DataProtection #Compliance #FinOps #DevOps