OCI Cache with Redis: Building a Production Caching Layer with Terraform

Database queries are expensive. Not in the sense of costing money directly, though they do that too, but in the sense of latency. Every round trip to a relational database under load adds milliseconds that accumulate into seconds at the tail end of your percentile distribution. For workloads where the same data is read far more often than it is written, a caching layer between the application and the database eliminates the majority of those round trips entirely.

OCI Cache is Oracle’s managed Redis service. It runs Redis 7 inside your VCN, handles replication, patching, and failover, and exposes a standard Redis endpoint that any Redis client library can connect to without modification. In this post I will walk through deploying OCI Cache with Terraform, configuring cluster mode with replication, setting up TLS, tuning eviction policies, and integrating it with a Python application using connection pooling.

Architecture Overview

OCI Cache runs inside your VCN in a private subnet. The application tier connects to it over the private network. No traffic crosses the public internet. The cache cluster sits between your application and the database, handling read-heavy workloads while writes go directly to the database and invalidate or update the corresponding cache entries.

			
Application Tier (OKE / Compute)
        |
   OCI Cache (Redis 7 - private subnet)
        |
   OCI Autonomous Database / MySQL
        |
   Object Storage (for cache warm-up datasets)

		

Step 1: IAM Policy

			
resource "oci_identity_policy" "cache_policy" {
  compartment_id = var.compartment_id
  name           = "oci-cache-management-policy"
  description    = "Permissions for OCI Cache service and operators"
  statements = [
    "Allow group ${var.ops_group_name} to manage redis-clusters in compartment id ${var.compartment_id}",
    "Allow group ${var.ops_group_name} to manage redis-family in compartment id ${var.compartment_id}",
    "Allow service redis to use virtual-network-family in compartment id ${var.compartment_id}"
  ]
}

		

Step 2: Network Security Group for Cache

The cache cluster should only accept connections from the application subnet. Create a dedicated NSG that enforces this at the network level.

			
resource "oci_core_network_security_group" "cache_nsg" {
  compartment_id = var.compartment_id
  vcn_id         = var.vcn_id
  display_name   = "cache-cluster-nsg"
}
# Allow Redis traffic only from the application subnet
resource "oci_core_network_security_group_security_rule" "cache_ingress" {
  network_security_group_id = oci_core_network_security_group.cache_nsg.id
  direction                 = "INGRESS"
  protocol                  = "6"
  source                    = var.app_subnet_cidr
  source_type               = "CIDR_BLOCK"
  tcp_options {
    destination_port_range {
      min = 6379
      max = 6380
    }
  }
}
# Allow TLS Redis traffic
resource "oci_core_network_security_group_security_rule" "cache_tls_ingress" {
  network_security_group_id = oci_core_network_security_group.cache_nsg.id
  direction                 = "INGRESS"
  protocol                  = "6"
  source                    = var.app_subnet_cidr
  source_type               = "CIDR_BLOCK"
  tcp_options {
    destination_port_range {
      min = 6380
      max = 6380
    }
  }
}

		

Step 3: Deploy the OCI Cache Cluster

			
resource "oci_redis_redis_cluster" "app_cache" {
  compartment_id     = var.compartment_id
  display_name       = "production-app-cache"
  node_count         = 3
  node_memory_in_gbs = 4
  software_version   = "REDIS_7_0"
  subnet_id          = var.cache_subnet_id
  cluster_mode       = "NONCLUSTER"
  nsg_ids = [oci_core_network_security_group.cache_nsg.id]
  defined_tags = {
    "Operations.Environment" = "production"
    "Operations.Application" = "orders-api"
    "Operations.ManagedBy"   = "terraform"
  }
}
output "cache_primary_endpoint" {
  value       = oci_redis_redis_cluster.app_cache.primary_endpoint_ip_address
  description = "Primary endpoint for write operations"
}
output "cache_replicas_endpoint" {
  value       = oci_redis_redis_cluster.app_cache.replicas_endpoint_ip_address
  description = "Replicas endpoint for read operations"
}

		

The node_count = 3 gives you one primary and two replicas. Write operations go to the primary endpoint. Read operations can be distributed across the replicas endpoint, which load-balances across both replicas. For production, always deploy with at least one replica so a node failure does not take the cache offline.

Step 4: Configure Eviction Policy and Memory Settings

OCI Cache exposes Redis configuration parameters through the cluster resource. The eviction policy determines what Redis does when memory is full and a new key needs to be written. For a session cache where all data has equal value, allkeys-lru evicts the least recently used key regardless of whether it has a TTL set. For a cache where only keys with TTLs should be evicted, use volatile-lru.

			
# Store Redis configuration as a Vault secret for the application to read
resource "oci_vault_secret" "cache_config" {
  compartment_id = var.compartment_id
  vault_id       = var.vault_id
  key_id         = var.vault_key_id
  secret_name    = "production-cache-config"
  secret_content {
    content_type = "BASE64"
    content = base64encode(jsonencode({
      primary_endpoint  = oci_redis_redis_cluster.app_cache.primary_endpoint_ip_address
      replicas_endpoint = oci_redis_redis_cluster.app_cache.replicas_endpoint_ip_address
      port              = 6379
      tls_port          = 6380
      max_memory_policy = "allkeys-lru"
      connection_timeout_ms = 500
      socket_timeout_ms     = 1000
    }))
  }
}

		

Step 5: Application Integration with Connection Pooling

Connect to OCI Cache from Python using the redis library with a connection pool. A connection pool reuses TCP connections across requests instead of opening a new connection for every cache operation, which adds significant overhead at scale.

			
import redis
import json
import logging
import oci
from typing import Optional, Any
from functools import wraps
import hashlib
logger = logging.getLogger(__name__)
class CacheClient:
    def __init__(self, compartment_id: str):
        config = self._load_cache_config(compartment_id)
        # Write pool - primary endpoint only
        self._write_pool = redis.ConnectionPool(
            host=config["primary_endpoint"],
            port=config["port"],
            max_connections=20,
            socket_connect_timeout=0.5,
            socket_timeout=1.0,
            retry_on_timeout=True,
            health_check_interval=30
        )
        # Read pool - replicas endpoint for load-balanced reads
        self._read_pool = redis.ConnectionPool(
            host=config["replicas_endpoint"],
            port=config["port"],
            max_connections=50,
            socket_connect_timeout=0.5,
            socket_timeout=1.0,
            retry_on_timeout=True,
            health_check_interval=30
        )
        self._write_client = redis.Redis(connection_pool=self._write_pool)
        self._read_client  = redis.Redis(connection_pool=self._read_pool)
    def _load_cache_config(self, compartment_id: str) -> dict:
        oci_config = oci.config.from_file()
        secrets_client = oci.secrets.SecretsClient(oci_config)
        vault_client   = oci.vault.VaultsClient(oci_config)
        secret = vault_client.get_secret_bundle_by_name(
            secret_name="production-cache-config",
            vault_id=oci_config["vault_id"]
        ).data
        import base64
        return json.loads(base64.b64decode(secret.secret_bundle_content.content))
    def get(self, key: str) -> Optional[Any]:
        try:
            value = self._read_client.get(key)
            if value is None:
                return None
            return json.loads(value)
        except redis.RedisError as e:
            logger.warning(f"Cache GET failed for key {key}: {e}")
            return None
    def set(self, key: str, value: Any, ttl_seconds: int = 300) -> bool:
        try:
            serialized = json.dumps(value, default=str)
            return self._write_client.setex(key, ttl_seconds, serialized)
        except redis.RedisError as e:
            logger.warning(f"Cache SET failed for key {key}: {e}")
            return False
    def delete(self, key: str) -> bool:
        try:
            return bool(self._write_client.delete(key))
        except redis.RedisError as e:
            logger.warning(f"Cache DELETE failed for key {key}: {e}")
            return False
    def invalidate_pattern(self, pattern: str) -> int:
        try:
            keys = self._write_client.keys(pattern)
            if not keys:
                return 0
            return self._write_client.delete(*keys)
        except redis.RedisError as e:
            logger.warning(f"Cache pattern invalidation failed for {pattern}: {e}")
            return 0
def cache_result(ttl_seconds: int = 300, key_prefix: str = ""):
    def decorator(func):
        @wraps(func)
        def wrapper(self, *args, **kwargs):
            cache_key = f"{key_prefix}:{func.__name__}:{hashlib.md5(str(args).encode()).hexdigest()}"
            cached = self.cache.get(cache_key)
            if cached is not None:
                logger.debug(f"Cache HIT: {cache_key}")
                return cached
            logger.debug(f"Cache MISS: {cache_key}")
            result = func(self, *args, **kwargs)
            self.cache.set(cache_key, result, ttl_seconds)
            return result
        return wrapper
    return decorator

		

Step 6: Cache-Aside Pattern for Database Queries

The cache-aside pattern is the most common caching strategy. The application checks the cache first. On a miss, it queries the database and populates the cache before returning the result. On the next request for the same data, the cache serves it directly.

			
class OrderRepository:
    def __init__(self, db_session, cache_client: CacheClient):
        self.db      = db_session
        self.cache   = cache_client
        self.cache_ttl = 600  # 10 minutes
    @cache_result(ttl_seconds=600, key_prefix="orders")
    def get_order_by_id(self, order_id: str) -> Optional[dict]:
        row = self.db.execute(
            "SELECT * FROM orders WHERE order_id = :id",
            {"id": order_id}
        ).fetchone()
        return dict(row) if row else None
    def update_order_status(self, order_id: str, status: str) -> bool:
        rows_affected = self.db.execute(
            "UPDATE orders SET status = :status WHERE order_id = :id",
            {"status": status, "id": order_id}
        ).rowcount
        self.db.commit()
        if rows_affected > 0:
            # Invalidate the cached entry immediately after a write
            cache_key = f"orders:get_order_by_id:{order_id}"
            self.cache.delete(cache_key)
            logger.info(f"Cache invalidated for order {order_id} after status update")
        return rows_affected > 0
    def get_orders_by_customer(self, customer_id: str) -> list:
        cache_key = f"orders:customer:{customer_id}"
        cached = self.cache.get(cache_key)
        if cached is not None:
            return cached
        rows = self.db.execute(
            "SELECT * FROM orders WHERE customer_id = :cid ORDER BY created_at DESC LIMIT 50",
            {"cid": customer_id}
        ).fetchall()
        result = [dict(row) for row in rows]
        self.cache.set(cache_key, result, ttl_seconds=300)
        return result

		

Step 7: Monitoring Cache Performance

OCI Cache publishes metrics to OCI Monitoring under the oci_redis namespace. The two most important metrics to watch are hit rate and memory utilization. A hit rate below 80% means your cache keys, TTLs, or caching strategy need tuning. Memory utilization above 85% means you are approaching eviction territory and need to either increase node memory or reduce TTLs.

			
resource "oci_monitoring_alarm" "cache_low_hit_rate" {
  compartment_id        = var.compartment_id
  display_name          = "cache-low-hit-rate"
  is_enabled            = true
  metric_compartment_id = var.compartment_id
  namespace             = "oci_redis"
  query                 = "CacheHits[5m].sum() / (CacheHits[5m].sum() + CacheMisses[5m].sum()) * 100  85"
  severity              = "CRITICAL"
  pending_duration      = "PT5M"
  destinations          = [var.ops_notification_topic_id]
  body                  = "Cache memory utilization exceeded 85%. Keys are at risk of eviction. Consider increasing node memory or reducing TTLs."
}

		

Cache Invalidation Strategy

Cache invalidation is the hard part of caching. There are three approaches that work in practice.

TTL-based expiry is the simplest. Every key gets a TTL and expires automatically. This means cached data can be stale for up to TTL seconds, which is acceptable for most read workloads. Set TTLs based on how often the underlying data changes. Product catalog data that changes weekly can have a TTL of hours. User session data that can change at any login should have a TTL of minutes.

Write-through invalidation deletes or updates the cache key synchronously when the underlying data is written. The example in Step 6 uses this pattern: after updating the order status in the database, the cache key for that order is deleted immediately. The next read populates the cache with fresh data. This gives you strong consistency at the cost of one extra cache miss after every write.

Event-driven invalidation uses OCI Streaming or OCI Events to broadcast cache invalidation messages when data changes. This is the right approach when multiple services write to the same data and all of them need to keep their caches in sync. A product service writes an updated price, publishes a product.updated event to OCI Streaming, and all services consuming that stream invalidate their cached copy of that product.

Where OCI Cache Fits

OCI Cache is the right tool when your application has a high read-to-write ratio, the cost of a cache miss is a database query with measurable latency, and you need the cache to survive application restarts because it holds session state or computed results that are expensive to regenerate.

It is not the right tool for data that changes on every request, for data that must be consistent to the millisecond, or for workloads where every user gets unique data with no overlap. In those cases, the overhead of cache management exceeds the benefit.

The operational advantage of OCI Cache over running Redis on a compute instance is the same as any managed service: Oracle handles node failures, replication lag monitoring, patching, and backups. Your team handles the caching logic, which is where the real engineering work lives anyway.

Regards,
Osama

#OCI #OracleCloud #Redis #Caching #Terraform #CloudNative #InfrastructureAsCode #IaC #OracleCloudInfrastructure #PlatformEngineering #CloudArchitecture #DevOps #DistributedSystems #BackendEngineering #CloudPerformance #TechBlog #DatabaseEngineering #CloudSecurity #OCICache #Oracle