AWS Services – Technology Geek

Migrating to Serverless

Posted on October 11, 2021 by Osama Mustafa in Cloud

we’ll look at considerations for migrating existing applications to serverless and common ways for extending the serverless

At a high level, there are three migration patterns that you might follow to migrate your legacy your applications to a serverless model.

Leapfrog

As the name suggests, you bypass interim steps and go straight from an on-premises legacy architecture to a serverless cloud architecture

Organic

You move on-premises applications to the cloud in more of a “lift and shift” model. In this model, existing applications are kept intact, either running on Amazon Elastic Compute Cloud (Amazon EC2) instances or with some limited rewrites to container services like Amazon Elastic Kubernetes Service (Amazon EKS)/Amazon Elastic Container Service (Amazon ECS) or AWS Fargate.

Developers experiment with Lambda in low-risk internal scenarios like log processing or cron jobs. As you gain more experience, you might use serverless components for tasks like data transformations and parallelization of processes.

At some point in the adoption curve, you take a more strategic look at how serverless and microservices might address business goals like market agility, developer innovation, and total cost of ownership.

You get buy-in for a more long-term commitment to invest in modernizing your applications and select a production workload as a pilot. With initial success and lessons learned, adoption accelerates, and more applications are migrated to microservices and serverless.

Strangler

With the strangler pattern, an organization incrementally and systematically decomposes monolithic applications by creating APIs and building event-driven components that gradually replace components of the legacy application.

Distinct API endpoints can point to old vs. new components, and safe deployment options (like canary deployments) let you point back to the legacy version with very little risk.

New feature branches can be “serverless first,” and legacy components can be decommissioned as they are replaced. This pattern represents a more systematic approach to adopting serverless, allowing you to move to critical improvements where you see benefit quickly but with less risk and upheaval than the leapfrog pattern.

Migration questions to answer:

What does this application do, and how are its components organized?

How can you break your data needs up based on the command query responsibility (CQRS) pattern?

How does the application scale, and what components drive the capacity you need?

Do you have schedule-based tasks?

Do you have workers listening to a queue?

Where can you refactor or enhance functionality without impacting the current implementation?

Application Load Balancer vs. API Gateway for directing traffic to serverless targets

Application Load Balancer	Amazon API Gateway
Easier to transition existing compute stack where you are already using an Application Load Balancer	Good for building REST APIs and integrating with other services and Lambda functions
Supports authorization via OIDC-capable providers, including Amazon Cognito user pools	Supports authorization via AWS Identity and Access Management (IAM), Amazon Cognito, and Lambda authorizers
Charged by the hour, based on Load Balancer Capacity Units	Charged based on requests served
May be more cost-effective for a steady stream of traffic	May be more cost-effective for spiky patterns
	Additional features for API management: Export SDK for clients Use throttling and usage plans to control access Maintain multiple versions of an APICanary deployments

Consider three factors when comparing costs of ownership:

The infrastructure cost to run your workload (for example, the costs for your provisioned EC2 capacity vs. the per-invocation cost of your Lambda functions)

The development effort to plan, architect, and provision resources on which the application will run

The costs of your team’s time to maintain the application once it is in production

Reference

Choosing serverless or traditional infrastructure?

Cheers

Osama

AWS Transit Gateway

Posted on October 2, 2021 by Osama Mustafa in Cloud

AWS Transit Gateway is a highly available and scalable service that provides interconnectivity between VPCs and your on-premises network. Within a Region, AWS Transit Gateway provides a method for consolidating and centrally managing routing between VPCs with a hub-and-spoke network architecture.

Between Regions, AWS Transit Gateway supports inter-regional peering with other transit gateways. It does this to facilitate routing network traffic between VPCs of different Regions over the AWS global backbone. This removes the need to route traffic over the internet. AWS Transit Gateway also integrates with hybrid network configurations when a Direct Connect or AWS Site-to-Site VPN connection is connected to the transit gateway.

AWS Transit Gateway concepts

Attachments

AWS Transit Gateway supports the following connections:

One or more VPCs
A compatible Software-Defined Wide Area Network (SD-WAN) appliance
A Direct Connect gateway
A peering connection with another transit gateway
A VPN connection to a transit gateway

AWS Transit Gateway MTU

AWS Transit Gateway supports an MTU of 8,500 bytes for:

VPC connections
Direct Connect connections
Connections to other transit gateways
Peering connections

AWS Transit Gateway supports an MTU of 1,500 bytes for VPN connections.

AWS Transit Gateway route table

A transit gateway has a default route table and can optionally have additional route tables. A route table includes dynamic and static routes that decide the next hop based on the destination IP address of the packet. The target of these routes can be any transit gateway attachment.

Associations

Each attachment is associated with exactly one route table. Each route table can be associated with zero to many attachments.

Route propagation

A VPC, VPN connection, or Direct Connect gateway can dynamically propagate routes to a transit gateway route table. With a Direct Connect attachment, the routes are propagated to a transit gateway route table by default.

With a VPC, you must create static routes to send traffic to the transit gateway.

With a VPN connection or a Direct Connect gateway, routes are propagated from the transit gateway to your on-premises router using BGP.

With a peering attachment, you must create a static route in the transit gateway route table to point to the peering attachment.

AWS Transit Gateway inter-regional peering

AWS offers two types of peering connections for routing traffic between VPCs in different Regions: VPC peering and transit gateway peering. Both peering types are one-to-one, but transit gateway peering connections have a simpler network design and more consolidated management.

Suppose a customer has multiple VPCs in three different Regions. As the following diagram illustrates, to permit network traffic to route between each VPC requires creating 72 VPC peering connections. Each VPC needs 8 different routing configurations and security policies.

With AWS Transit Gateway, the same environment only needs three peering connections. The transit gateway in each Region facilitates routing network traffic to all the VPCs in its Region. Because all routing can be managed by the transit gateway, the customer only needs to maintain three routing configurations, simplifying management.

Cheers

Osama

Configuring Your Lambda Functions

Posted on August 18, 2021 by Osama Mustafa in Cloud

When building and testing a function, you must specify three primary configuration settings: memory, timeout, and concurrency. These settings are important in defining how each function performs. Deciding how to configure memory, timeout, and concurrency comes down to testing your function in real-world scenarios and against peak volume. As you monitor your functions, you must adjust the settings to optimize costs and ensure the desired customer experience with your application.

Memory

You can allocate up to 10 GB of memory to a Lambda function. Lambda allocates CPU and other resources linearly in proportion to the amount of memory configured. Any increase in memory size triggers an equivalent increase in CPU available to your function. To find the right memory configuration for your functions, use the AWS Lambda Power Tuning tool.

Timeout

The AWS Lambda timeout value dictates how long a function can run before Lambda terminates the Lambda function. At the time of this publication, the maximum timeout for a Lambda function is 900 seconds. This limit means that a single invocation of a Lambda function cannot run longer than 900 seconds (which is 15 minutes).

It is important to analyze how long your function runs. When you analyze the duration, you can better determine any problems that might increase the invocation of the function beyond your expected length. Load testing your Lambda function is the best way to determine the optimum timeout value.

Lambda billing costs

With AWS Lambda, you pay only for what you use. You are charged based on the number of requests for your functions and the duration, the time it takes for your code to run. Lambda counts a request each time it starts running in response to an event notification or an invoke call, including test invokes from the console.

Duration is calculated from the time your code begins running until it returns or otherwise terminates, rounded up to the nearest 1 ms. Price depends on the amount of memory you allocate to your function, not the amount of memory your function uses. If you allocate 10 GB to a function and the function only uses 2 GB, you are charged for the 10 GB. This is another reason to test your functions using different memory allocations to determine which is the most beneficial for the function and your budget.

In the AWS Lambda resource model, you can choose the amount of memory you want for your function and are allocated proportional CPU power and other resources. An increase in memory triggers an equivalent increase in CPU available to your function. The AWS Lambda Free Tier includes 1 million free requests per month and 400,000 GB-seconds of compute time per month.

The balance between power and duration

Depending on the function, you might find that the higher memory level might actually cost less because the function can complete much more quickly than at a lower memory configuration.

You can use an open-source tool called Lambda Power Tuning to find the best configuration for a function. The tool helps you to visualize and fine-tune the memory and power configurations of Lambda functions. The tool runs in your own AWS account—powered by AWS Step Functions—and supports three optimization strategies: cost, speed, and balanced. It’s language-agnostic so that you can optimize any Lambda functions in any of your languages.

Concurrency and scaling

Concurrency is the third major configuration that affects your function’s performance and its ability to scale on demand. Concurrency is the number of invocations your function runs at any given moment. When your function is invoked, Lambda launches an instance of the function to process the event. When the function code finishes running, it can handle another request. If the function is invoked again while the first request is still being processed, another instance is allocated. Having more than one invocation running at the same time is the function’s concurrency.

Concurrent invocations

As an analogy, you can think of concurrency as the total capacity of a restaurant for serving a certain number of diners at one time. If you have seats in the restaurant for 100 diners, only 100 people can sit at the same time. Anyone who comes while the restaurant is full must wait for a current diner to leave before a seat is available. If you use a reservation system, and a dinner party has called to reserve 20 seats, only 80 of those 100 seats are available for people without a reservation. Lambda functions also have a concurrency limit and a reservation system that can be used to set aside runtime for specific instances.

Concurrency types

Unreserved concurrency

The amount of concurrency that is not allocated to any specific set of functions. The minimum is 100 unreserved concurrency. This allows functions that do not have any provisioned concurrency to still be able to run. If you provision all your concurrency to one or two functions, no concurrency is left for any other function. Having at least 100 available allows all your functions to run when they are invoked.

Reserved concurrency

Guarantees the maximum number of concurrent instances for the function. When a function has reserved concurrency, no other function can use that concurrency. No charge is incurred for configuring reserved concurrency for a function.

Provisioned concurrency

Initializes a requested number of runtime environments so that they are prepared to respond immediately to your function’s invocations. This option is used when you need high performance and low latency.

You pay for the amount of provisioned concurrency that you configure and for the period of time that you have it configured.

For example, you might want to increase provisioned concurrency when you are expecting a significant increase in traffic. To avoid paying for unnecessary warm environments, you scale back down when the event is over.

Reasons for setting concurrency limits

Limit a function’s concurrency to achieve the following:

Limit costs
Regulate how long it takes you to process a batch of events
Match it with a downstream resource that cannot scale as quickly as Lambda

Reserve function concurrency to achieve the following:

Ensure that you can handle peak expected volume for a critical function
Address invocation errors

CloudWatch metrics for concurrency

When your function finishes processing an event, Lambda sends metrics about the invocation to Amazon CloudWatch. You can build graphs and dashboards with these metrics in the CloudWatch console. You can also set alarms to respond to changes in use, performance, or error rates.

CloudWatch includes two built-in metrics that help determine concurrency: ConcurrentExecutions and UnreservedConcurrentExecutions.

ConcurrentExecutions

Shows the sum of concurrent invocations for a given function at a given point in time. Provides historical data on how functions are performing.

You can view all functions in the account or only the functions that have a custom concurrency limit specified.

UnreservedConcurrentExecutions

Shows the sum of the concurrency for the functions that do not have a custom concurrency limit specified.

Enjoy the Cloud

Osama

Cheers

AWS security levels

Posted on August 15, 2021August 15, 2021 by Osama Mustafa in Cloud

Infrastructure Protection

Infrastructure protection ensures that systems and resources within your workloads are protected against unintended and unauthorized access, and other potential vulnerabilities. Amazon Virtual Private Cloud (Amazon VPC) allows you to isolate your AWS resources in the cloud. A VPC enables you to launch resources into a virtual network that you’ve defined and that closely resembles a traditional network that you’d operate in your own data center.

Services :-

AWS Firewall Manager is a security management service that allows you to centrally configure and manage AWS WAF rules across your accounts and applications. Firewall Manager is able to bring new applications and resources into compliance with a common set of security rules from the start.

AWS Direct Connect is a cloud service solution that is used to establish a dedicated and secure network connection from your premises to AWS. Using AWS Direct Connect, you can establish private connectivity between AWS and your data center, office, or colocation environment. In many cases, this can reduce your network costs, increase bandwidth throughput, and provide a more consistent network experience than internet-based connections.
AWS CloudFormation automates and simplifies the task of repeatedly creating and deploying AWS resources in a consistent manner. With AWS CloudFormation, you can ensure that all of your security and compliance controls are deployed along with your new environment.
Amazon Inspector is an automated security assessment service that helps improve the security and compliance of applications deployed on AWS. It assesses applications for vulnerabilities or deviations from best practices. After performing an assessment, Amazon Inspector produces a detailed list of security findings prioritized by level of severity.

Data Protection

Protecting data at rest has to do with encrypting data while using one of our storage services, including our database services. When it comes to Amazon S3, for example, there are two types of encryption options available:

Client side : you can do it by youself

Server Side : AWS will do it for you.

Any data that gets transmitted from one system to another is considered data in transit. AWS recommends the following solutions and best practices to help you provide the appropriate level of protection for your data in transit, including the confidentiality and integrity of your application’s data.

Additional AWS Services for Data Protection

AWS CloudHSM provides hardware security modules (HSM) in the AWS Cloud. An HSM is a computing device that processes cryptographic operations and provides secure storage for cryptographic keys. CloudHSM allows you to generate, store, import, export, and manage cryptographic keys, including symmetric keys and asymmetric key pairs.
Amazon S3 Glacier is a storage service optimized for infrequently used data, also called cold data. This service provides durable and extremely low-cost storage with security features for data archiving and backup. Amazon S3 Glacier stores data as archives within vaults. You can enforce compliance controls for individual Amazon S3 Glacier vaults with a vault lock policy.
AWS Certificate Manager (ACM) handles the complexity of creating and managing public SSL/TLS certificates for your AWS based websites and applications. ACM can also be used to issue private SSL/TLS X.509 certificates that identify users, computers, applications, services, servers, and other devices internally.
Amazon Macie uses machine learning to automatically discover, classify, and protect sensitive data in AWS. Macie recognizes sensitive data such as personally identifiable information (PII) or intellectual property. It provides you with dashboards and alerts that give visibility into how this data is being accessed or moved.
AWS Key Management Service (AWS KMS) is a managed service that allows you to create and control the keys used in data encryption. If you want a managed service for creating and controlling encryption keys, but do not want or need to operate your own hardware security module (HSM), consider using AWS KMS. You can use the key management and cryptographic features directly in your applications or through AWS services that are integrated with AWS KMS, including AWS CloudTrail, which helps meet your auditing, regulatory, and compliance needs.

DDoS Mitigation

Edge locations are physical data centers located in key cities, that are different from Availability Zones. As access to certain data increases with time, this data is copied to an edge location near your customer base for better performance and latency. Threats can then be taken care of at these edge locations, away from your web applications, AWS resources, and the original data.
Amazon Route 53 is a highly available and scalable DNS service that can be used to direct traffic to your web application. It includes many advanced features like traffic flow, latency-based routing, weighted round-robin, Geo DNS, health checks, and monitoring. You can use these features to improve the performance of your web application and to avoid site outages. Route 53 is hosted at numerous AWS edge locations, creating a global surface area capable of absorbing large amounts of DDoS traffic.
Amazon CloudFront is a content delivery network (CDN) service that can be used to deliver data, including your entire website, to end users. CloudFront only accepts HTTPS and HTTP well-formed connections to prevent many common DDoS attacks. These capabilities can greatly improve your ability to continue serving traffic to end users during larger DDoS attacks.
AWS Shield is a managed DDoS protection service that safeguards web applications that run on AWS. AWS Shield provides always-on detection and automatic inline mitigations that minimize application downtime and latency.
AWS Web Application Firewall (WAF) helps protect your web applications from common web exploits that could affect application availability, compromise security, or consume excessive resources. AWS WAF gives you control over which traffic to allow or block by defining customizable web security rules.

Reference

Overview of AWS Security – Network Security

Tips for Securing Your EC2 Instance

Amazon Inspector rules packages and rules

Data protection in Amazon S3

AWS Key Management Service Cryptographic Details

VPN Connections Overview

AWS Best Practices for DDoS Resiliency

AWS Edge Locations

Regards

Osama

Types of AWS Credentials

Posted on August 15, 2021August 15, 2021 by Osama Mustafa in Cloud

In this post, I will talk about AWS IAM Users and Groups and AWS credentials.

The careful management of access credentials is the foundation of how you will secure your resources in the cloud. As we saw in the previous video, every interaction you make with AWS is authenticated. When you open an AWS account, the identity you begin with has access to all AWS services and resources in that account. You use this identity to establish less-privileged users and role-based access in IAM. IAM is a centralized mechanism for creating and managing individual users and their permissions with your AWS account.

An IAM group is a collection of users. Groups allow you to specify permissions for similar types of users. For example, if you have a group named “Developers,” you can give that group the types of permissions that developers typically need. This can be considered a form of role-based access control. Create groups that reflect organization roles, not technical commonality.

AWS Credentials

Username/Password
- A password policy is a set of rules that define the type of password an IAM user can set. You should define a password policy for all of your IAM users to enforce strong passwords and regular changing of passwords. Password requirements are similar to those found in most secure online environments.

Multi-factor authentication
- Multi-factor authentication (MFA) is an additional layer of security for accessing AWS services. With this authentication method, more than one authentication factor is checked before access is granted, which consists of a user name and password, and the single-use code from the MFA device. AWS CLI also supports MFA. Please click here for a list of supported MFA devices.

User Access Key
- Users need their own access keys to make programmatic calls to AWS using the AWS CLI, the AWS SDKs, or direct HTTPS calls using the APIs for individual AWS services. Access keys are used to digitally sign API calls made to AWS services. Each access key credential is comprised of an access key ID and a secret key. Each user can have two active access keys, which is useful when you need to rotate the user’s access keys or revoke permissions.

Amazon EC2 key Pair

To enable SSH or RDP connections to an Amazon Elastic Cloud Compute (EC2) instance, AWS uses a public–key infrastructure to sign the login request. The public and private keys are known as a key pair. To log in to your instance, you must create a key pair, or use an existing key pair, and provide the private key when you connect to the instance. You can choose to have the EC2 key pairs generated by AWS or import your own set of keys.

EC2 key pairs do not provide accountability (as in who is using the keys); therefore, they are not recommended for routine usage. If you require daily access to the instance, AWS recommends that EC2 instances be part of a directory domain (Active Directory or LDAP) in order to enable federated access and provide accountability by tracking which user is logging into which instance.

Additional AWS Services for Identity and Access Management

AWS Secrets Manager is designed to centrally manage secrets used to access resources on AWS, on-premises, and third-party services. Secrets can be database credentials, passwords, third-party API keys, and even arbitrary text. Secrets Manager enables you to replace hardcoded credentials in your code with an API call to Secrets Manager to retrieve the secret programmatically. Also, you can configure Secrets Manager to automatically rotate the secret for you according to a schedule that you specify.
AWS Single Sign-On (SSO) is a cloud SSO service that allows for the central management of SSO access to multiple AWS accounts and business applications. It enables users to sign in to a user portal with their existing corporate credentials and access all of their assigned accounts and applications from one place. AWS SSO includes built-in SAML integrations to many business applications. AWS SSO may be integrated with Microsoft Active Directory, which means your employees can sign in to your AWS SSO user portal using their corporate Active Directory credentials.
The AWS Security Token Service (STS) is a web service that enables you to request temporary, limited-privilege credentials for IAM users who are taking on a different role or for users who are being federated. A scenario in which someone, or something, needs access to your account to perform a specific task that is not done on a daily basis would be a great candidate for temporary credentials.

AWS Directory Service for Microsoft Active Directory, also known as AWS Managed Microsoft AD, enables your domain workloads and AWS resources to use managed Active Directory in the AWS Cloud. AWS Managed Microsoft AD is built on actual Microsoft Active Directory and does not require you to synchronize or replicate data from your existing Active Directory to the cloud.
AWS Organizations lets you centrally manage and enforce policies for multiple AWS accounts. This service allows grouping accounts into organizational units and use service control policies to centrally control AWS services across multiple AWS accounts. With Organizations, you can also automate the creation of new accounts through APIs and simplify billing by allowing you to set up a single payment method for all the accounts in your organization through consolidated billing. Organizations is available to all AWS customers at no additional charge.
Amazon Cognito lets you add user sign-up, sign-in, and access controls to your web and mobile apps. You can define roles and map users to different roles so your app can access only the resources that are authorized for each user. User sign in can be done either by a third-party identity provider, or directly via Amazon Cognito.

An Amazon Cognito user pool is a user directory that manages the overhead of handling the tokens that are returned from social sign-in providers, such as Facebook, Google, and Amazon, and enterprise identity providers via SAML 2.0. After a successful user pool sign-in, your web or mobile app will receive user pool tokens from Amazon Cognito. These tokens can then be used to retrieve AWS credentials via Amazon Cognito identity pools. These credentials allow your app to access other AWS services and you don’t have to embed long-term AWS credentials in your app.

Reference :-

Security best practices in IAM

Setting an account password policy for IAM users

Regards

Osama