You can upload any type of file to Amazon S3, such as images, videos, text files, and so on. For example, you might use Amazon S3 to store backup files, media files for a website, or archived documents. Amazon S3 offers unlimited storage space. The maximum file size for an object in Amazon S3 is 5 TB.
Amazon S3 storage classes
With Amazon S3, you pay only for what you use. You can choose from a range of storage classes to select a fit for your business and cost needs. When selecting an Amazon S3 storage class, consider these two factors:
How often you plan to retrieve your data
How available you need your data to be
Designed for frequently accessed data
Stores data in a minimum of three Availability Zones
S3 Standard provides high availability for objects. This makes it a good choice for a wide range of use cases, such as websites, content distribution, and data analytics. S3 Standard has a higher cost than other storage classes intended for infrequently accessed data and archival storage.
S3 Standard-Infrequent Access (S3 Standard-IA)
Ideal for infrequently accessed data
Similar to S3 Standard but has a lower storage price and higher retrieval price
S3 Standard-IA is ideal for data infrequently accessed but requires high availability when needed. Both S3 Standard and S3 Standard-IA store data in a minimum of three Availability Zones. S3 Standard-IA provides the same level of availability as S3 Standard but with a lower storage price and a higher retrieval price.
S3 One Zone-Infrequent Access (S3 One Zone-IA)
Stores data in a single Availability Zone
Has a lower storage price than S3 Standard-IA
Compared to S3 Standard and S3 Standard-IA, which store data in a minimum of three Availability Zones, S3 One Zone-IA stores data in a single Availability Zone. This makes it a good storage class to consider if the following conditions apply:
You want to save costs on storage.
You can easily reproduce your data in the event of an Availability Zone failure.
Ideal for data with unknown or changing access patterns
Requires a small monthly monitoring and automation fee per object
In the S3 Intelligent-Tiering storage class, Amazon S3 monitors objects’ access patterns. If you haven’t accessed an object for 30 consecutive days, Amazon S3 automatically moves it to the infrequent access tier, S3 Standard-IA. If you access an object in the infrequent access tier, Amazon S3 automatically moves it to the frequent access tier, S3 Standard.
Low-cost storage designed for data archiving
Able to retrieve objects within a few minutes to hours
S3 Glacier is a low-cost storage class that is ideal for data archiving. For example, you might use this storage class to store archived customer records or older photos and video files.
Low-cost storage designed for data archiving
Able to retrieve objects within a few minutes to hours
S3 Glacier is a low-cost storage class that is ideal for data archiving. For example, you might use this storage class to store archived customer records or older photos and video files.
S3 Glacier Deep Archive
Lowest-cost object storage class ideal for archiving
Able to retrieve objects within 12 hours
When deciding between Amazon S3 Glacier and Amazon S3 Glacier Deep Archive, consider how quickly you need to retrieve archived objects. You can retrieve objects stored in the S3 Glacier storage class within a few minutes to a few hours. By comparison, you can retrieve objects stored in the S3 Glacier Deep Archive storage class within 12 hours.
Apache Directory Studio is a complete directory tooling platform intended to be used with any LDAP server however it is particularly designed for use with the ApacheDS. It is an Eclipse RCP application, composed of several Eclipse (OSGi) plugins, that can be easily upgraded with additional ones.
Step 1: Create a New Connection in Apache Directory Studio
With Amazon EC2, you pay only for the compute time that you use. Amazon EC2 offers a variety of pricing options for different use cases. For example, if your use case can withstand interruptions, you can save with Spot Instances. You can also save by committing early and locking in a minimum level of use with Reserved Instances.
are ideal for short-term, irregular workloads that cannot be interrupted. No upfront costs or minimum contracts apply. The instances run continuously until you stop them, and you pay for only the compute time you use.
Sample use cases for On-Demand Instances include developing and testing applications and running applications that have unpredictable usage patterns. On-Demand Instances are not recommended for workloads that last a year or longer because these workloads can experience greater cost savings using Reserved Instances.
Amazon EC2 Savings Plans
AWS offers Savings Plans for several compute services, including Amazon EC2. Amazon EC2 Savings Plans enable you to reduce your compute costs by committing to a consistent amount of compute usage for a 1-year or 3-year term. This term commitment results in savings of up to 66% over On-Demand costs.
Any usage up to the commitment is charged at the discounted plan rate (for example, $10 an hour). Any usage beyond the commitment is charged at regular On-Demand rates.
Later in this course, you will review AWS Cost Explorer, a tool that enables you to visualize, understand, and manage your AWS costs and usage over time. If you are considering your options for Savings Plans, AWS Cost Explorer can analyze your Amazon EC2 usage over the past 7, 30, or 60 days. AWS Cost Explorer also provides customized recommendations for Savings Plans. These recommendations estimate how much you could save on your monthly Amazon EC2 costs, based on previous Amazon EC2 usage and the hourly commitment amount in a 1-year or 3-year plan.
are a billing discount applied to the use of On-Demand Instances in your account. You can purchase Standard Reserved and Convertible Reserved Instances for a 1-year or 3-year term, and Scheduled Reserved Instances for a 1-year term. You realize greater cost savings with the 3-year option.
At the end of a Reserved Instance term, you can continue using the Amazon EC2 instance without interruption. However, you are charged On-Demand rates until you do one of the following:
Terminate the instance.
Purchase a new Reserved Instance that matches the instance attributes (instance type, Region, tenancy, and platform).
are ideal for workloads with flexible start and end times, or that can withstand interruptions. Spot Instances use unused Amazon EC2 computing capacity and offer you cost savings at up to 90% off of On-Demand prices.
Suppose that you have a background processing job that can start and stop as needed (such as the data processing job for a customer survey). You want to start and stop the processing job without affecting the overall operations of your business. If you make a Spot request and Amazon EC2 capacity is available, your Spot Instance launches. However, if you make a Spot request and Amazon EC2 capacity is unavailable, the request is not successful until capacity becomes available. The unavailable capacity might delay the launch of your background processing job.
After you have launched a Spot Instance, if capacity is no longer available or demand for Spot Instances increases, your instance may be interrupted. This might not pose any issues for your background processing job. However, in the earlier example of developing and testing applications, you would most likely want to avoid unexpected interruptions. Therefore, choose a different EC2 instance type that is ideal for those tasks.
are physical servers with Amazon EC2 instance capacity that is fully dedicated to your use.
You can use your existing per-socket, per-core, or per-VM software licenses to help maintain license compliance. You can purchase On-Demand Dedicated Hosts and Dedicated Hosts Reservations. Of all the Amazon EC2 options that were covered, Dedicated Hosts are the most expensive.
Amazon EC2 instance types are optimized for different tasks. When selecting an instance type, consider the specific needs of your workloads and applications. This might include requirements for compute, memory, or storage capabilities.
General purpose instances
provide a balance of compute, memory, and networking resources. You can use them for a variety of workloads, such as:
backend servers for enterprise applications
small and medium databases
Suppose that you have an application in which the resource needs for compute, memory, and networking are roughly equivalent. You might consider running it on a general purpose instance because the application does not require optimization in any single resource area.
Compute optimized instances
are ideal for compute-bound applications that benefit from high-performance processors. Like general purpose instances, you can use compute optimized instances for workloads such as web, application, and gaming servers.
However, the difference is compute optimized applications are ideal for high-performance web servers, compute-intensive applications servers, and dedicated gaming servers. You can also use compute optimized instances for batch processing workloads that require processing many transactions in a single group.
Memory optimized instances
are designed to deliver fast performance for workloads that process large datasets in memory. In computing, memory is a temporary storage area. It holds all the data and instructions that a central processing unit (CPU) needs to be able to complete actions. Before a computer program or application is able to run, it is loaded from storage into memory. This preloading process gives the CPU direct access to the computer program.
Suppose that you have a workload that requires large amounts of data to be preloaded before running an application. This scenario might be a high-performance database or a workload that involves performing real-time processing of a large amount of unstructured data. In these types of use cases, consider using a memory optimized instance. Memory optimized instances enable you to run workloads with high memory needs and receive great performance.
Accelerated computing instances
use hardware accelerators, or coprocessors, to perform some functions more efficiently than is possible in software running on CPUs. Examples of these functions include floating-point number calculations, graphics processing, and data pattern matching.
In computing, a hardware accelerator is a component that can expedite data processing. Accelerated computing instances are ideal for workloads such as graphics applications, game streaming, and application streaming.
Storage optimized instances
are designed for workloads that require high, sequential read and write access to large datasets on local storage. Examples of workloads suitable for storage optimized instances include distributed file systems, data warehousing applications, and high-frequency online transaction processing (OLTP) systems.
In computing, the term input/output operations per second (IOPS) is a metric that measures the performance of a storage device. It indicates how many different input or output operations a device can perform in one second. Storage optimized instances are designed to deliver tens of thousands of low-latency, random IOPS to applications.
You can think of input operations as data put into a system, such as records entered into a database. An output operation is data generated by a server. An example of output might be the analytics performed on the records in a database. If you have an application that has a high IOPS requirement, a storage optimized instance can provide better performance over other instance types not optimized for this kind of use case.
The Object Storage service provides reliable, secure, and scalable object storage. Object storage is a storage architecture that stores and manages data as objects. Some typical use cases include data backup, file sharing, and storing unstructured data like logs and sensor-generated data.
Creating a Bucket
Open the navigation menu and click Storage. Under Object Storage, click Buckets.A list of the buckets in the compartment you’re viewing is displayed.
Select a compartment from the Compartment list on the left side of the page.A list of existing buckets is displayed.
Click Create Bucket.
Default Storage Tier: Select the default tier in which you want to store your data
Standard is the primary, default storage tier Use the Standard tier for storing frequently accessed data that requires fast and immediate access.
Archive is the default storage tier used for archive storage, Use the Archive tier for storing rarely accessed data that requires long retention periods. Access to data in the Archive tier is not immediate. Archived data must be restored before the data is accessible.
Object Events: Select Emit Object Events if you want to enable the bucket to emit events for object state changes. For more information about events.
Encryption: Buckets are encrypted with keys managed by Oracle by default, but you can optionally encrypt the data in this bucket using your own Vault encryption key. To use Vault for your encryption needs, select Encrypt Using Customer-Managed Keys
Uploading Files to a Bucket
To upload files to your bucket using the Console:
From the Object Storage Buckets screen, click the bucket name to view its details.
In the Object Name Prefix field, optionally specify a file name prefix for the files that you plan to upload.
If the Storage Tier field displays Standard, you can optionally change the storage tier to upload objects to.
All-at-once deployments instantly shift traffic from the original (old) Lambda function to the updated (new) Lambda function, all at one time. All-at-once deployments can be beneficial when the speed of your deployments matters. In this strategy, the new version of your code is released quickly, and all your users get to access it immediately.
A linear deployment is similar to canary deployment. In this strategy, you direct a small amount of traffic to
In a canary deployment, you deploy your new version of your application code and shift a small percentage of production traffic to point to that new version. After you have validated that this version is safe and not causing errors, you direct all traffic to the new version of your code.
A linear deployment is similar to canary deployment. In this strategy, you direct a small amount of traffic to your new version of code at first. After a specified period of time, you automatically increment the amount of traffic that you send to the new version until you’re sending 100% of production traffic.
Comparing deployment strategies
To help you decide which deployment strategy to use for your application, you’ll need to consider each option’s consumer impact, rollback, event model factors, and deployment speed. The comparison table below illustrates these points.
Event Model Factors
All at once
Redeploy older version
Any event model at low concurrency rate
1-10% typical initial traffic shift, then phased
Revert 100% of traffic to previous deployment
Better for high-concurrency workloads
Minutes to hours
Deployment preferences with AWS SAM
Traffic shifting with aliases is directly integrated into AWS SAM. If you’d like to use all-at-once, canary, or linear deployments with your Lambda functions, you can embed that directly into your AWS SAM templates. You can do this in the deployment preferences section of the template. AWS CodeDeploy uses the deployment preferences section to manage the function rollout as part of the AWS CloudFormation stack update. SAM has several pre-built deployment preferences you can use to deploy your code. See the table below for examples.
Deployment Preferences Type
Shifts 10 percent of traffic in the first increment. The remaining 90 percent is deployed 30 minutes later.
Shifts 10 percent of traffic in the first increment. The remaining 90 percent is deployed 5 minutes later.
Shifts 10 percent of traffic in the first increment. The remaining 90 percent is deployed 10 minutes later.
Shifts 10 percent of traffic in the first increment. The remaining 90 percent is deployed 15 minutes later.
Shifts 10 percent of traffic every 10 minutes until all traffic is shifted.
Shifts 10 percent of traffic every minute until all traffic is shifted.
Shifts 10 percent of traffic every 2 minutes until all traffic is shifted.
Shifts 10 percent of traffic every 3 minutes until all traffic is shifted.
Shifts all traffic to the updated Lambda functions at once.
Creating a deployment pipeline
When you check a piece of code into source control, you don’t want to wait for a human to manually approve it or have each piece of code run through different quality checks. Using a CI/CD pipeline can help automate the steps required to release your software deployment and standardize on a core set of quality checks.
Review the built-in Amazon CloudWatch metrics and their dimensions for each of the services you plan to use so that you can decide how to best leverage them vs. adding custom metrics. There are also many third-party tools that provide monitoring and metrics reporting from CloudWatch data.
Business Key Performance Indicators (KPIs) measure your application performance against business goals. It is extremely important to know when something is critically affecting your overall business (revenue wise or not).
Customer experience data dictates not only the overall effectiveness of the UI/UX but also whether changes or anomalies are affecting the customer experience in a particular section of your application. These metrics are often measured in percentiles to prevent outliers when trying to understand the impact over time and how widespread it is across your customer base.
Examples: Perceived latency, time it takes to add an item to a basket/to checkout, page load times
Vendor and application metrics are important to underpin root causes. System metrics also tell you if your systems are healthy, at risk, or already impacting your customers.
Examples: Percentage of HTTP errors/success, memory utilization, function duration/error/throttling, queue length, stream records length, integration latency
Ops metrics are important to understand sustainability and maintenance of a given system and crucial to pinpoint how stability has progressed/degraded over time.
Examples: Number of tickets([un]successful resolutions, etc.), number of times people on-call were paged, availability, CI/CD pipeline stats (successful/failed deployments, feedback time, cycle and lead time)
Logs let you dig into specific issues, but you can also use log data to create business-level metrics via CloudWatch Logs metric filters. You can interact with logs via CloudWatch Logs to drill into any specific log entry or filter them based on a pattern to create your own metrics. See how the services listed below interact with CloudWatch Logs.
Lambda automatically logs all requests handled by your function and stores them in CloudWatch Logs. This gives you access to information about each invocation of your Lambda function.
You can log almost anything to CloudWatch Logs by using print or standard out statements in your functions. When you create custom logs, use a structured format like a JSON event to make it easier to report from them.
API Gateway execution and access logs
API Gateway execution logs include information on errors as well as execution traces. Info like parameter values, payload, Lambda authorizers used, and API keys appear in the execution logs. You can log just errors or errors and info. Logging is set up per API stage. These logs are detailed, so you want to be thoughtful about what you need. Also, log groups don’t expire by default, so make sure to set retention values suitable to your workload.
You can also create custom access logs and send them to your preferred CloudWatch group to track who is accessing your APIs and how. You can specify the access details by selecting context variables and choosing the format you want to use.
CloudWatch Logs Insights
CloudWatch Log Insights lets you use prebuilt or custom queries on your logs to provide aggregated views and reporting. If you’ve created structured custom logs, CloudWatch Logs Insights can automatically discover the fields in your logs to make it easy to query and group your log data.
When a transaction fails, or completes slower than expected, how do you figure out where in the flow of services it failed? X-Ray gives you a visual representation of your services—a service map—that illustrates each integration point, and gives you quick insight into successes and failures. Then, you can drill down into the details of each individual trace.
You can enable X-Ray with one click for Lambda, API Gateway, and Amazon SNS. You can also turn it on for SQS queues that are not Lambda event sources, and you can add custom instrumentation to your function using the X-Ray SDK to write your own code. X-Ray integrations support both active and passive instrumentation.
You can add custom instrumentation to your function using the X-Ray SDK to write your own code. X-Ray integrations support both active and passive instrumentation:
Samples and instruments incoming requests
Instruments requests that have been sampled by another service
Writes traces to X-Ray
Can add information to traces
Amazon API Gateway
CloudWatch metrics – To view how resources are performing, CloudWatch metrics is the best solution. If a developer needs to check how many times a Lambda function has been invoked,
CloudWatch Logs Insights – CloudWatch Logs Insights enables you to interactively query your log data in CloudWatch Logs. If a team wants to search and query their logs for their API, CloudWatch Logs Insights would be the best option.
CloudWatch Logs – You can insert logging statements into your code to help you validate that your code is working as expected. Lambda automatically integrates with CloudWatch Logs and pushes all logs from your code to CloudWatch. If an engineer wants to see what parameters are being passed into a function, they can insert logging statements in the code and check the response in CloudWatch Logs.
X-Ray – X-Ray provides a visual map of successes and failures and lets you drill into individual traces for an execution and drill down into the details of how long each leg of the execution took.
Records IAM user, IAM role, and AWS service API activity in your account.
Is enabled when you create an account.
Provides full details about the API action, like identity of the requestor, time of the API call, request parameters, and response elements returned by the service.
When activity occurs in your AWS account, that activity is recorded in a CloudTrail event, and you can see recent events in the event history.
The CloudTrail event history provides a viewable, searchable, and downloadable record of the past 90 days of CloudTrail events. Use this history to gain visibility into actions taken in your AWS account in the AWS Management Console, AWS SDKs, command line tools, and other AWS services.
A trail is a configuration that enables delivery of CloudTrail events to an Amazon S3 bucket, CloudWatch Logs, and CloudWatch Events. If you need to maintain a longer history of events, you can create your own trail. When you create a trail, it tracks events performed on or within resources in your AWS account and writes them to an S3 bucket you specify.
For example, a trail could capture modifications to your API Gateway APIs. You can optionally add data events to track S3 object-level API activity (like when someone uploads something to the bucket) or Lambda invoke API operations on one or all future Lambda functions in the account.
You can configure CloudTrail Insights on your trails to help you identify and respond to unusual activity associated with write API calls. CloudTrail Insights is a feature that tracks your normal patterns of API call volume and generates Insights events when the volume is outside normal patterns.
we’ll look at considerations for migrating existing applications to serverless and common ways for extending the serverless
At a high level, there are three migration patterns that you might follow to migrate your legacy your applications to a serverless model.
As the name suggests, you bypass interim steps and go straight from an on-premises legacy architecture to a serverless cloud architecture
You move on-premises applications to the cloud in more of a “lift and shift” model. In this model, existing applications are kept intact, either running on Amazon Elastic Compute Cloud (Amazon EC2) instances or with some limited rewrites to container services like Amazon Elastic Kubernetes Service (Amazon EKS)/Amazon Elastic Container Service (Amazon ECS) or AWS Fargate.
Developers experiment with Lambda in low-risk internal scenarios like log processing or cron jobs. As you gain more experience, you might use serverless components for tasks like data transformations and parallelization of processes.
At some point in the adoption curve, you take a more strategic look at how serverless and microservices might address business goals like market agility, developer innovation, and total cost of ownership.
You get buy-in for a more long-term commitment to invest in modernizing your applications and select a production workload as a pilot. With initial success and lessons learned, adoption accelerates, and more applications are migrated to microservices and serverless.
With the strangler pattern, an organization incrementally and systematically decomposes monolithic applications by creating APIs and building event-driven components that gradually replace components of the legacy application.
Distinct API endpoints can point to old vs. new components, and safe deployment options (like canary deployments) let you point back to the legacy version with very little risk.
New feature branches can be “serverless first,” and legacy components can be decommissioned as they are replaced. This pattern represents a more systematic approach to adopting serverless, allowing you to move to critical improvements where you see benefit quickly but with less risk and upheaval than the leapfrog pattern.
Migration questions to answer:
What does this application do, and how are its components organized?
How can you break your data needs up based on the command query responsibility (CQRS) pattern?
How does the application scale, and what components drive the capacity you need?
Do you have schedule-based tasks?
Do you have workers listening to a queue?
Where can you refactor or enhance functionality without impacting the current implementation?
Application Load Balancer vs. API Gateway for directing traffic to serverless targets
Application Load Balancer
Amazon API Gateway
Easier to transition existing compute stack where you are already using an Application Load Balancer
Good for building REST APIs and integrating with other services and Lambda functions
Supports authorization via OIDC-capable providers, including Amazon Cognito user pools
Supports authorization via AWS Identity and Access Management (IAM), Amazon Cognito, and Lambda authorizers
Charged by the hour, based on Load Balancer Capacity Units
Charged based on requests served
May be more cost-effective for a steady stream of traffic
May be more cost-effective for spiky patterns
Additional features for API management: Export SDK for clients Use throttling and usage plans to control access Maintain multiple versions of an APICanary deployments
Consider three factors when comparing costs of ownership:
The infrastructure cost to run your workload (for example, the costs for your provisioned EC2 capacity vs. the per-invocation cost of your Lambda functions)
The development effort to plan, architect, and provision resources on which the application will run
The costs of your team’s time to maintain the application once it is in production
AWS Transit Gateway is a highly available and scalable service that provides interconnectivity between VPCs and your on-premises network. Within a Region, AWS Transit Gateway provides a method for consolidating and centrally managing routing between VPCs with a hub-and-spoke network architecture.
Between Regions, AWS Transit Gateway supports inter-regional peering with other transit gateways. It does this to facilitate routing network traffic between VPCs of different Regions over the AWS global backbone. This removes the need to route traffic over the internet. AWS Transit Gateway also integrates with hybrid network configurations when a Direct Connect or AWS Site-to-Site VPN connection is connected to the transit gateway.
AWS Transit Gateway concepts
AWS Transit Gateway supports the following connections:
One or more VPCs
A compatible Software-Defined Wide Area Network (SD-WAN) appliance
A Direct Connect gateway
A peering connection with another transit gateway
A VPN connection to a transit gateway
AWS Transit Gateway MTU
AWS Transit Gateway supports an MTU of 8,500 bytes for:
Direct Connect connections
Connections to other transit gateways
AWS Transit Gateway supports an MTU of 1,500 bytes for VPN connections.
AWS Transit Gateway route table
A transit gateway has a default route table and can optionally have additional route tables. A route table includes dynamic and static routes that decide the next hop based on the destination IP address of the packet. The target of these routes can be any transit gateway attachment.
Each attachment is associated with exactly one route table. Each route table can be associated with zero to many attachments.
A VPC, VPN connection, or Direct Connect gateway can dynamically propagate routes to a transit gateway route table. With a Direct Connect attachment, the routes are propagated to a transit gateway route table by default.
With a VPC, you must create static routes to send traffic to the transit gateway.
With a VPN connection or a Direct Connect gateway, routes are propagated from the transit gateway to your on-premises router using BGP.
With a peering attachment, you must create a static route in the transit gateway route table to point to the peering attachment.
AWS Transit Gateway inter-regional peering
AWS offers two types of peering connections for routing traffic between VPCs in different Regions: VPC peering and transit gateway peering. Both peering types are one-to-one, but transit gateway peering connections have a simpler network design and more consolidated management.
Suppose a customer has multiple VPCs in three different Regions. As the following diagram illustrates, to permit network traffic to route between each VPC requires creating 72 VPC peering connections. Each VPC needs 8 different routing configurations and security policies.
With AWS Transit Gateway, the same environment only needs three peering connections. The transit gateway in each Region facilitates routing network traffic to all the VPCs in its Region. Because all routing can be managed by the transit gateway, the customer only needs to maintain three routing configurations, simplifying management.