AWS Lambda Layers and Container Images: Packaging Your Functions the Right Way

The way you package a Lambda function affects its cold start time, deployment speed, dependency management, and how much your team can share code across functions. Most teams start with zip deployments, hit a package size limit or a dependency conflict, and then scramble to find a solution.

Lambda gives you two serious packaging options: Layers for zip-based functions and container images for full Docker deployments. They solve different problems and the right choice depends on what you are actually trying to accomplish. In this article I will walk through both, cover when each is the right call, and share the production patterns that keep things manageable as your function count grows.

The Problem With Naive Zip Deployments

A basic Lambda deployment zips your function code and its dependencies together and uploads the result. This works fine for small functions. It breaks down when you have dozens of functions that all depend on the same libraries, when your dependencies push the 50MB compressed / 250MB unzipped size limit, or when different functions need different versions of the same library.

Without a packaging strategy, you end up with the same 80MB numpy and pandas installation duplicated across 20 functions. Every deployment uploads those same 80MB again. Your CI/CD pipeline slows down. Your S3 artifact bucket fills up. And when you need to patch a dependency, you update it in 20 places.

Lambda Layers

A Lambda Layer is a zip archive that contains libraries, a custom runtime, or other dependencies. You attach up to 5 layers to a function. At invocation time, Lambda mounts the layer contents under /opt in the execution environment and your function code can import from them directly.

Layers solve the duplication problem: build your shared dependencies once, publish them as a layer, and reference that layer from every function that needs it. Update the layer when you patch a dependency and update the layer reference in your functions. One place to maintain instead of many.

resource "aws_lambda_layer_version" "data_science" {
  layer_name          = "data-science-deps"
  description         = "numpy, pandas, scikit-learn for Python 3.12"
  compatible_runtimes = ["python3.12"]
  compatible_architectures = ["x86_64", "arm64"]

  filename         = "${path.module}/layers/data-science-deps.zip"
  source_code_hash = filebase64sha256("${path.module}/layers/data-science-deps.zip")

  license_info = "MIT"
}

resource "aws_lambda_layer_version" "internal_utils" {
  layer_name          = "internal-utils"
  description         = "Shared utilities: logging, metrics, error handling"
  compatible_runtimes = ["python3.12"]

  filename         = "${path.module}/layers/internal-utils.zip"
  source_code_hash = filebase64sha256("${path.module}/layers/internal-utils.zip")
}

resource "aws_lambda_function" "data_processor" {
  function_name = "data-processor"
  role          = aws_iam_role.lambda.arn
  handler       = "handler.process"
  runtime       = "python3.12"
  architectures = ["arm64"]

  filename         = "${path.module}/functions/data-processor.zip"
  source_code_hash = filebase64sha256("${path.module}/functions/data-processor.zip")

  layers = [
    aws_lambda_layer_version.data_science.arn,
    aws_lambda_layer_version.internal_utils.arn
  ]

  timeout     = 60
  memory_size = 1024

  environment {
    variables = {
      LOG_LEVEL = "INFO"
    }
  }
}

The function deployment package contains only your application code, which is typically a few kilobytes. The dependencies live in the layer. This means function deployments are fast and the heavy layer is only uploaded when dependencies change.

Building a layer correctly requires matching the architecture and Python version of the Lambda execution environment. The most reliable approach is building inside an Amazon Linux container:

#!/bin/bash

ARCH="arm64"
PYTHON_VERSION="python3.12"
LAYER_DIR="layers/data-science-deps"

mkdir -p "${LAYER_DIR}/python"

docker run --rm \
  --platform linux/arm64 \
  -v "$(pwd)/${LAYER_DIR}:/output" \
  public.ecr.aws/lambda/python:3.12-arm64 \
  pip install \
    numpy==1.26.4 \
    pandas==2.2.2 \
    scikit-learn==1.5.1 \
    --target /output/python \
    --no-cache-dir

cd "${LAYER_DIR}" && zip -r9 ../data-science-deps.zip python/
echo "Layer built: layers/data-science-deps.zip"

Building inside the Lambda container image ensures that compiled extensions like numpy are compiled against the same glibc version that runs in the Lambda environment. Building on macOS and deploying to Lambda is the most common source of Import errors in Lambda functions using native extensions.

Lambda Container Images

Container image deployment lets you package your function as a Docker image up to 10GB in size. The image is stored in ECR and Lambda pulls it at invocation time. Your function runs exactly the same environment in Lambda as it does locally, in your CI/CD pipeline, and in any other container environment.

Container images are the right choice when your dependencies push past the zip size limits, when you need custom system libraries that cannot be installed through pip or npm, when you want full local testing parity with production, or when your team already has a Docker-centric workflow.

FROM public.ecr.aws/lambda/python:3.12-arm64

WORKDIR ${LAMBDA_TASK_ROOT}

COPY requirements.txt .
RUN pip install -r requirements.txt --no-cache-dir --target "${LAMBDA_TASK_ROOT}"

COPY src/ .

CMD ["handler.lambda_handler"]

Always use the official AWS Lambda base images from public.ecr.aws/lambda rather than building from a generic Python or Node image. The Lambda base images include the Lambda Runtime Interface Client, which handles the event loop between Lambda’s control plane and your handler function. Without it, your container will not work as a Lambda function.

resource "aws_ecr_repository" "lambda_function" {
  name                 = "lambda-data-processor"
  image_tag_mutability = "IMMUTABLE"

  image_scanning_configuration {
    scan_on_push = true
  }
}

resource "aws_lambda_function" "container_processor" {
  function_name = "container-data-processor"
  role          = aws_iam_role.lambda.arn
  package_type  = "Image"
  architectures = ["arm64"]

  image_uri = "${aws_ecr_repository.lambda_function.repository_url}:${var.image_tag}"

  timeout     = 120
  memory_size = 2048

  image_config {
    command           = ["handler.lambda_handler"]
    working_directory = "/var/task"
  }

  environment {
    variables = {
      LOG_LEVEL    = "INFO"
      ENVIRONMENT  = var.environment
    }
  }
}

arm64 architecture costs 20 percent less per invocation-millisecond than x86_64 and typically offers better performance for compute-intensive workloads. Use arm64 unless you have a specific dependency that is not available for the ARM architecture.

Cold Starts and Image Optimization

Container images have longer cold starts than zip deployments because Lambda needs to pull and cache the image before starting the execution environment. Lambda caches images aggressively after the first pull, so cold starts primarily affect the very first invocation or invocations after a long period of inactivity.

Optimize your image to minimize cold start impact. Use multi-stage builds to keep the final image as small as possible. Install only the dependencies your function actually uses. Order your Dockerfile layers so the most stable dependencies come first, which maximizes layer cache reuse between builds.

FROM public.ecr.aws/lambda/python:3.12-arm64 AS builder

WORKDIR /build
COPY requirements.txt .
RUN pip install -r requirements.txt --no-cache-dir --target /build/deps

FROM public.ecr.aws/lambda/python:3.12-arm64

COPY --from=builder /build/deps ${LAMBDA_TASK_ROOT}/
COPY src/ ${LAMBDA_TASK_ROOT}/

CMD ["handler.lambda_handler"]

For latency-sensitive functions that cannot tolerate any cold start, use Provisioned Concurrency. Lambda pre-initializes a specified number of execution environments and keeps them warm. You pay for the provisioned capacity continuously but cold starts are eliminated entirely for that concurrency level.

CI/CD Pipeline for Container Functions

version: 0.2

phases:
  pre_build:
    commands:
      - aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $ECR_REGISTRY
      - IMAGE_TAG=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-8)

  build:
    commands:
      - docker buildx build
          --platform linux/arm64
          --tag $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          --cache-from type=registry,ref=$ECR_REGISTRY/$ECR_REPOSITORY:cache
          --cache-to type=registry,ref=$ECR_REGISTRY/$ECR_REPOSITORY:cache,mode=max
          --push
          .

  post_build:
    commands:
      - aws lambda update-function-code
          --function-name $FUNCTION_NAME
          --image-uri $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
      - aws lambda wait function-updated
          --function-name $FUNCTION_NAME
      - printf '[{"name":"%s","imageUri":"%s"}]' $FUNCTION_NAME $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG > imagedefinitions.json

artifacts:
  files:
    - imagedefinitions.json

Using BuildKit registry cache dramatically speeds up builds in CI. The first build is slow as it pulls and pushes all layers. Subsequent builds only upload layers that changed. For a function with stable dependencies, this reduces build time from several minutes to under 30 seconds in most cases.

Choosing Between Layers and Container Images

Use layers when your function code is small, your dependencies fit within the size limits, and you want the fastest possible deployment cycle for frequent updates. Layers work well for teams with many small focused functions that share a common set of utilities.

Use container images when your total package exceeds 250MB unzipped, when you need custom system libraries, when you want identical behavior between local development and production, or when your team already builds and deploys Docker containers for other services. Container images also make it straightforward to run your Lambda function locally with docker run before deploying.

Both approaches work well in production. The right choice is the one that fits your team’s workflow and your function’s actual requirements.

Closing Thoughts

Packaging is one of those Lambda topics that teams rarely think about until it becomes a problem. By then the codebase has dozens of functions with duplicated dependencies, inconsistent runtimes, and no clear strategy for updates.

Pick your packaging approach early and apply it consistently. Layers for shared dependencies in zip-based workflows. Container images for anything complex, large, or where local parity matters. Keep your function code separate from your dependency packaging so deployments stay fast. And build everything against the Lambda execution environment architecture to avoid the frustrating class of bugs that only surface after deployment.

Enjoy the cloud.

Osama

#AWS #Lambda #ServerlessArchitecture #LambdaLayers #ContainerImages #Docker #CloudArchitecture #Terraform #InfrastructureAsCode #CloudNative #AmazonWebServices #SolutionsArchitect #CloudComputing #BackendEngineering #DevOps #ECR #TechBlog #CloudInfrastructure #ColdStart #Serverless