Object Storage on OCI is often treated as a simple file dump. Buckets get created, data lands in them, and nobody thinks about it again until the storage bill arrives or a compliance audit reveals that logs from three years ago are still sitting in standard storage at full price when they could have been archived or deleted months ago.
OCI Object Storage has three storage tiers, lifecycle policies that automate transitions and deletions, multipart upload for large objects, pre-authenticated requests for temporary access, and event-driven integration with OCI Functions and Streaming. Used properly, it is a complete data management platform, not just a file store. This post covers the full production configuration using Terraform.
Storage Tiers
OCI Object Storage has three tiers that differ in retrieval speed and cost.
Standard tier provides immediate access with no retrieval fee. Use it for active data that applications read and write regularly: application artifacts, current logs, configuration files, recent backups.
Infrequent Access tier costs less for storage but adds a retrieval fee and a minimum storage duration of 31 days. Use it for data you access occasionally: monthly reports, older backups, audit logs from the last 90 days.
Archive tier has the lowest storage cost but requires a restore operation before the data can be read, which takes up to an hour. Use it for compliance data, historical records, and anything you hope never to need but must retain for regulatory reasons.
Step 1: Bucket Configuration with Versioning and Encryption
resource "oci_objectstorage_bucket" "app_data" {
compartment_id = var.compartment_id
namespace = data.oci_objectstorage_namespace.ns.namespace
name = "production-app-data"
access_type = "NoPublicAccess"
storage_tier = "Standard"
versioning = "Enabled"
kms_key_id = var.vault_key_id
metadata = {
environment = "production"
application = "orders-api"
managed-by = "terraform"
}
}
resource "oci_objectstorage_bucket" "app_archive" {
compartment_id = var.compartment_id
namespace = data.oci_objectstorage_namespace.ns.namespace
name = "production-app-archive"
access_type = "NoPublicAccess"
storage_tier = "Archive"
kms_key_id = var.vault_key_id
}
data "oci_objectstorage_namespace" "ns" {
compartment_id = var.compartment_id
}
output "bucket_namespace" {
value = data.oci_objectstorage_namespace.ns.namespace
}Versioning on the standard bucket retains previous versions of objects when they are overwritten or deleted. This protects against accidental deletion and application bugs that corrupt data. Each version is stored as a separate object and billed at the same rate. Configure lifecycle policies to delete old versions automatically.
Step 2: Lifecycle Policies
Lifecycle rules automate tier transitions and deletions based on object age and prefix. This is where the cost control lives.
resource "oci_objectstorage_object_lifecycle_policy" "app_data_lifecycle" {
namespace = data.oci_objectstorage_namespace.ns.namespace
bucket = oci_objectstorage_bucket.app_data.name
rules {
name = "move-logs-to-infrequent-access"
action = "INFREQUENT_ACCESS"
is_enabled = true
time_amount = 30
time_unit = "DAYS"
object_name_filter {
inclusion_prefixes = ["logs/"]
}
}
rules {
name = "archive-logs-after-90-days"
action = "ARCHIVE"
is_enabled = true
time_amount = 90
time_unit = "DAYS"
object_name_filter {
inclusion_prefixes = ["logs/"]
}
}
rules {
name = "delete-archived-logs-after-2-years"
action = "DELETE"
is_enabled = true
time_amount = 730
time_unit = "DAYS"
object_name_filter {
inclusion_prefixes = ["logs/"]
}
}
rules {
name = "delete-old-object-versions"
action = "DELETE"
is_enabled = true
time_amount = 90
time_unit = "DAYS"
target = "previousObjectVersions"
}
rules {
name = "abort-incomplete-multipart-uploads"
action = "ABORT"
is_enabled = true
time_amount = 7
time_unit = "DAYS"
target = "multipartUploads"
}
rules {
name = "archive-backups-after-30-days"
action = "ARCHIVE"
is_enabled = true
time_amount = 30
time_unit = "DAYS"
object_name_filter {
inclusion_prefixes = ["backups/"]
}
}
rules {
name = "delete-temp-files"
action = "DELETE"
is_enabled = true
time_amount = 1
time_unit = "DAYS"
object_name_filter {
inclusion_prefixes = ["tmp/", "staging/"]
}
}
}The abort-incomplete-multipart-uploads rule is critical and frequently forgotten. When a multipart upload is initiated but never completed, the uploaded parts accumulate in storage and are billed at the standard rate indefinitely. Setting this rule to 7 days cleans up any abandoned uploads automatically.
The delete-old-object-versions rule with target = "previousObjectVersions" removes old versions of versioned objects after 90 days. Without this, versioning causes storage to grow without bound as every write creates a new version.
Step 3: Multipart Upload for Large Objects
Objects larger than 100MB should use multipart upload. It breaks the object into parts, uploads them in parallel, and assembles them on the server side. This is faster than a single-stream upload, recoverable on failure (only the failed part needs to be retried), and required for objects over 5GB.
import oci
import os
import math
import logging
import concurrent.futures
from typing import List
logger = logging.getLogger(__name__)
class MultipartUploader:
PART_SIZE_BYTES = 128 * 1024 * 1024 # 128 MB per part
MAX_WORKERS = 4
def __init__(self, namespace: str, bucket: str):
self.namespace = namespace
self.bucket = bucket
config = oci.config.from_file()
self.client = oci.object_storage.ObjectStorageClient(config)
def upload(self, object_name: str, file_path: str) -> str:
file_size = os.path.getsize(file_path)
num_parts = math.ceil(file_size / self.PART_SIZE_BYTES)
logger.info(f"Starting multipart upload: {object_name}, size: {file_size}, parts: {num_parts}")
# Initiate the multipart upload
response = self.client.create_multipart_upload(
namespace_name=self.namespace,
bucket_name=self.bucket,
create_multipart_upload_details=oci.object_storage.models.CreateMultipartUploadDetails(
object=object_name,
storage_tier="Standard"
)
)
upload_id = response.data.upload_id
logger.info(f"Multipart upload initiated, upload_id: {upload_id}")
try:
part_etags = self._upload_parts(
file_path, upload_id, object_name, num_parts
)
# Commit the upload
self.client.commit_multipart_upload(
namespace_name=self.namespace,
bucket_name=self.bucket,
object_name=object_name,
upload_id=upload_id,
commit_multipart_upload_details=oci.object_storage.models.CommitMultipartUploadDetails(
parts_to_commit=[
oci.object_storage.models.CommitMultipartUploadPartDetails(
part_num=i + 1,
etag=etag
)
for i, etag in enumerate(part_etags)
]
)
)
logger.info(f"Multipart upload committed: {object_name}")
return upload_id
except Exception as e:
logger.error(f"Upload failed, aborting: {e}")
self.client.abort_multipart_upload(
namespace_name=self.namespace,
bucket_name=self.bucket,
object_name=object_name,
upload_id=upload_id
)
raise
def _upload_parts(self, file_path: str, upload_id: str, object_name: str, num_parts: int) -> List[str]:
part_etags = [None] * num_parts
def upload_part(part_number: int):
offset = (part_number - 1) * self.PART_SIZE_BYTES
with open(file_path, 'rb') as f:
f.seek(offset)
part_data = f.read(self.PART_SIZE_BYTES)
response = self.client.upload_part(
namespace_name=self.namespace,
bucket_name=self.bucket,
object_name=object_name,
upload_id=upload_id,
upload_part_num=part_number,
upload_part_body=part_data
)
etag = response.headers['ETag']
logger.debug(f"Part {part_number}/{num_parts} uploaded, etag: {etag}")
return part_number, etag
with concurrent.futures.ThreadPoolExecutor(max_workers=self.MAX_WORKERS) as executor:
futures = {
executor.submit(upload_part, i + 1): i
for i in range(num_parts)
}
for future in concurrent.futures.as_completed(futures):
part_number, etag = future.result()
part_etags[part_number - 1] = etag
return part_etagsStep 4: Pre-Authenticated Requests
Pre-authenticated requests (PARs) provide time-limited access to specific objects or buckets without requiring OCI credentials. Use them to share files with external systems, give temporary upload access to third parties, or generate download links for end users.
resource "oci_objectstorage_preauthrequest" "report_download" {
namespace = data.oci_objectstorage_namespace.ns.namespace
bucket = oci_objectstorage_bucket.app_data.name
name = "quarterly-report-download"
access_type = "ObjectRead"
object_name = "reports/Q2-2026-summary.pdf"
time_expires = "2026-08-01T00:00:00Z"
}
output "report_download_url" {
value = oci_objectstorage_preauthrequest.report_download.full_path
sensitive = true
}From Python, generate PARs dynamically with a short expiry for user-facing download links:
import oci
from datetime import datetime, timezone, timedelta
def generate_download_url(namespace: str, bucket: str, object_name: str, expires_in_hours: int = 1) -> str:
config = oci.config.from_file()
client = oci.object_storage.ObjectStorageClient(config)
expiry = datetime.now(timezone.utc) + timedelta(hours=expires_in_hours)
response = client.create_preauthenticated_request(
namespace_name=namespace,
bucket_name=bucket,
create_preauthenticated_request_details=oci.object_storage.models.CreatePreauthenticatedRequestDetails(
name=f"download-{object_name.replace('/', '-')}-{int(expiry.timestamp())}",
object_name=object_name,
access_type="ObjectRead",
time_expires=expiry
)
)
return response.data.full_pathStep 5: Event-Driven Processing with OCI Events
When objects land in a bucket, OCI Events can trigger downstream processing automatically. Connect object creation events to an OCI Function that validates, transforms, or routes the data.
resource "oci_events_rule" "object_created_rule" {
compartment_id = var.compartment_id
display_name = "process-new-uploads"
is_enabled = true
condition = jsonencode({
eventType = ["com.oraclecloud.objectstorage.createobject"]
data = {
additionalDetails = {
bucketName = [oci_objectstorage_bucket.app_data.name]
namespaceName = [data.oci_objectstorage_namespace.ns.namespace]
}
}
})
actions {
actions {
action_type = "FAAS"
is_enabled = true
function_id = var.processing_function_id
description = "Trigger file processing function on new upload"
}
}
}
# Restrict the event rule to a specific prefix using condition filter
resource "oci_events_rule" "incoming_orders_rule" {
compartment_id = var.compartment_id
display_name = "process-incoming-orders"
is_enabled = true
condition = jsonencode({
eventType = ["com.oraclecloud.objectstorage.createobject"]
data = {
resourceName = [{
"condition": "LIKE",
"value": "incoming/orders/%"
}]
additionalDetails = {
bucketName = [oci_objectstorage_bucket.app_data.name]
}
}
})
actions {
actions {
action_type = "ONS"
is_enabled = true
topic_id = var.order_processing_topic_id
description = "Notify order processing pipeline of new order file"
}
}
}Step 6: Replication for Cross-Region DR
OCI Object Storage supports asynchronous replication to a bucket in another region. Objects written to the source bucket are copied to the destination bucket automatically. Use this for disaster recovery, compliance requirements that mandate geo-redundant storage, or serving downloads from a region closer to your users.
resource "oci_objectstorage_replication_policy" "cross_region_backup" {
namespace = data.oci_objectstorage_namespace.ns.namespace
bucket = oci_objectstorage_bucket.app_data.name
name = "cross-region-dr-replication"
destination_bucket_name = var.dr_bucket_name
destination_region_name = var.dr_region
}Replication is asynchronous. Objects typically appear in the destination bucket within minutes, but there is no guaranteed RPO. For workloads with strict RPO requirements, pair replication with OCI Block Volume cross-region backups or Autonomous Database Data Guard.
Step 7: Access Logging and Monitoring
resource "oci_logging_log" "bucket_access_log" {
display_name = "app-data-bucket-access"
log_group_id = var.log_group_id
log_type = "SERVICE"
configuration {
source {
category = "write"
resource = oci_objectstorage_bucket.app_data.name
service = "objectstorage"
source_type = "OCISERVICE"
}
compartment_id = var.compartment_id
}
retention_duration = 90
is_enabled = true
}
resource "oci_monitoring_alarm" "bucket_storage_high" {
compartment_id = var.compartment_id
display_name = "bucket-storage-cost-alert"
is_enabled = true
metric_compartment_id = var.compartment_id
namespace = "oci_objectstorage"
query = "StoredBytes[1d]{bucketName = '${oci_objectstorage_bucket.app_data.name}'}.max() > 1099511627776"
severity = "WARNING"
pending_duration = "PT24H"
destinations = [var.ops_notification_topic_id]
body = "Production app data bucket has exceeded 1TB. Review lifecycle policy configuration and data growth rate."
}IAM Policy for Bucket Access
resource "oci_identity_policy" "object_storage_policy" {
compartment_id = var.compartment_id
name = "object-storage-access-policy"
description = "Scoped access to production Object Storage buckets"
statements = [
# Application can read and write to the data bucket
"Allow dynamic-group app-instances to manage objects in compartment id ${var.compartment_id} where target.bucket.name = '${oci_objectstorage_bucket.app_data.name}'",
# Application can only read from the archive bucket
"Allow dynamic-group app-instances to read objects in compartment id ${var.compartment_id} where target.bucket.name = '${oci_objectstorage_bucket.app_archive.name}'",
# Ops team has full access to both buckets
"Allow group ${var.ops_group_name} to manage object-family in compartment id ${var.compartment_id}"
]
}Operational Notes
Always set a lifecycle rule to abort incomplete multipart uploads. It is easy to forget and expensive to overlook. A single large upload that fails and is never cleaned up can add meaningful storage costs over months.
Use prefix-based lifecycle rules rather than bucket-level rules wherever possible. A single bucket can hold logs, backups, reports, and application data, each with different retention requirements. Prefix-based rules let you apply the right policy to each data type without managing multiple buckets.
Enable versioning on any bucket that holds data written by automated processes. Application bugs, deployment errors, and runaway scripts delete or overwrite data regularly. Versioning gives you a recovery path. Pair it with a version cleanup rule so old versions do not accumulate indefinitely.
Pre-authenticated requests expire. When generating PARs for external systems, build the expiry timestamp into the URL metadata and monitor for PARs approaching expiry if the external dependency needs continuous access. A PAR that expires silently causes an integration failure with no obvious error message on the receiving end.
Regards,
Osama
#OCI #OracleCloud #ObjectStorage #Terraform #CloudNative #InfrastructureAsCode #IaC #OracleCloudInfrastructure #DataManagement #CloudArchitecture #PlatformEngineering #DevOps #TechBlog #CloudStorage #DataEngineering #Oracle #LifecyclePolicy #CloudCostOptimization #StorageTiers #EventDriven
Technology Geek 
Leave a comment