BLOG

Automation

Posted on by Osama Mustafa in AWS, Cloud

Infrastructure as code (IaC)

You can simplify the deployment of your AWS resources using an infrastructure as code approach. With and IaC solution, you create a template that describes all the resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and IaC solution takes care of provisioning and configuring those resources for you.

The benefits of infrastructure as code 

Gain the benefits of repeatability and reusability while building your environments. Build the same complex environments with one template, or a combination of templates.

For instance, a template can be designed so that different AMIs are used in the development or the production environments.

In this scenario, the template has been updated to add new security groups to the instance stacks. With one change to the templates, both environments can have the new security group resource added.

CloudFormation

The template describes the resources to be created

Essentially, CloudFormation is an API wrapper. When you create an EC2 instance in the AWS Management Console wizard, you initiate an API call to the Amazon EC2 service. The information you enter through the wizard is passed on as parameters. 

CloudFormation manages the dependencies and relationships.

Author your CloudFormation template with any code editor, check it into a version-control system such as GitHub or CodeCommit, and review files before deploying. 

Stacks

All resources in a stack are defined by the stack’s CloudFormation template. Stacks are a collection of AWS resources managed as a single unit. Stacks allow the creation and deletion of resources as a unit.

Change management in stacks

Change sets

Change sets allow you to preview how proposed changes to a stack might impact your running resources. For example, whether your changes will delete or replace any critical resources. AWS CloudFormation makes the changes to your stack only when you decide to execute the change set. You can create and manage change sets using the CloudFormation console, AWS CLI, or CloudFormation API.

Infrastructure tools

When building on AWS you can use different tools to help automate the deployment of infrastructure and manage those resources once deployed.

Tools for deployment

When choosing infrastructure deployment tools, you need to find a balance between convenience and control. Some tools give you complete control and have you choose every component and configuration. Though you can customize your deployment to fit your business needs, this requires greater expertise and more resources to manage and maintain. Other tools are designed for convenience and include preconfigured infrastructure templates for common solutions. Though these tools are easier to use and require less maintenance, you do not always have the ability to customize your infrastructure components. 

AWS Elastic Beanstalk

The goal of Elastic Beanstalk is to help developers deploy and maintain scalable web applications and services in the cloud without having to worry about the underlying infrastructure. Elastic Beanstalk configures each EC2 instance in your environment with the components necessary to run applications for the selected platform. With Elastic Beanstalk you can provision infrastructure to support common application designs, such as web applications and worker services.

AWS Solutions Library

AWS Solutions Library helps you solve common problems and build faster using the AWS platform. Solutions are vetted by AWS architects and are designed to be operationally effective, reliable, secure, and cost efficient.

AWS Cloud Development Kit (AWS CDK)

AWS CDK is a software development framework that defines your cloud application resources using a declarative model and familiar programming languages. AWS CDK includes a library of customizable constructs, which are building blocks consisting of one or more resources and include common configurations. You can use AWS CDK to generate CloudFormation templates and deploy your infrastructure along with your application runtime assets.

Automating infrastructure management with AWS Systems Manager


AWS Systems Manager makes it easier to bridge your existing infrastructure with AWS. Systems Manager helps you automatically collect software inventory, apply operating system (OS) patches, create system images, and configure Windows and Linux OSs. These capabilities help you:

  • Define and track system configurations
  • Prevent drift
  • Maintain software compliance of your Amazon EC2 and on-premises configurations 

With AWS Systems Manager, you can:

  • Centralize operational data from multiple AWS services and automate tasks across your AWS resources.
  • Create logical groups of resources such as applications, different layers of an application stack, or development and production environments.
  • Select a resource group and view its recent API activity, resource configuration changes, related notifications, operational alerts, software inventory, and patch compliance status.
  • Take action on each resource group depending on your operational needs.

You can open AWS Systems Manager from the Amazon EC2 console. Select the instances you want to manage, and define the management tasks you want to perform. Systems Manager is available at no cost to manage your Amazon EC2 and on-premises resources.

Benefits of Systems Manager

Shortens the time to detect problems

View operational data for groups of resources, so you can quickly identify any issues that might impact applications that use those resources. 

Automates tasks to increase efficiency

Automate operational tasks to help make your teams more efficient.

Improves visibility and control

Understand and control the state of your resource groups. 

Manages hybrid environments

Manage servers running on AWS and in your on-premises data center through a single interface.

Maintains security and compliance

Maintain security and compliance by scanning your instances against your patch, configuration, and custom policies.

Regards

Osama

AWS Auto Scaling 

Posted on by Osama Mustafa in AWS, Cloud

AWS Auto Scaling lets you build scaling plans that automate how groups of different resources respond to changes in demand. You can optimize availability, costs, or a balance of both. AWS Auto Scaling automatically creates all of the scaling policies and sets targets for you, based on your needs.

Auto scaling

AWS Auto Scaling monitors your applications and automatically adjusts capacity to maintain steady, predictable performance at the lowest possible cost. Using AWS Auto Scaling, you can set up application scaling for multiple resources across multiple services in minutes. 

The service provides a simple, powerful user interface that lets you build scaling plans for resources including Amazon EC2 instances and Spot Fleets, and other compute and database services that will be addressed later in the course. 

AWS Auto Scaling makes scaling simple with recommendations that let you optimize performance, costs, or balance between them.

Amazon EC2 Auto Scaling

With Amazon EC2 Auto Scaling, you can build scaling plans that automate how groups of different EC2 resources respond to changes in demand. You can optimize availability, costs, or a balance of both.

If you specify scaling policies, then Amazon EC2 Auto Scaling can launch or terminate instances as demand on your application increases or decreases. Amazon EC2 Auto Scaling integrates with ELB so you can attach one or more load balancers to an existing Amazon EC2 Auto Scaling group. After you attach the load balancer, it automatically registers the instances in the group and distributes incoming traffic across the instances.

Amazon EC2 Auto Scaling components

Launch templateAmazon EC2 Auto Scaling groupAuto scaling policy
What resources do you need?Where and how many do you need?When and for how long do you need them?
AMI
Instance type
Security groups
Roles		VPC and subnets
Load balancer
Define:
Minimum instances
Maximum instances
Desired capacity (optional)		Scheduled
On-demand
Predictive auto scaling
Scale-out policy
Scale-in policy

Launch template

A launch template is an instance configuration template that a group uses to launch EC2 instances. Included are the instance type, EBS volume, ID of the AMI, a key pair, security groups, and the other parameters that you use to launch EC2 instances.

Launch templates are the preferred method to configure your group in AWS because they provide:

  • A consistent experience
  • Simple permissions
  • Governance and best practices
  • Increased productivity

Optimize cost with EC2 Auto Scaling

Amazon EC2 Auto Scaling supports multiple purchasing options within the same group. You can launch and automatically scale a fleet of On-Demand Instances and Spot Instances within a single Auto Scaling group. In addition to receiving discounts for using Spot Instances, you can use Reserved Instances or a Savings Plan to receive discounted rates of the regular On-Demand Instance pricing. All of these factors combined help you to optimize your cost savings for EC2 instances, while making sure that you obtain the desired scale and performance for your application.

Using Amazon EC2 Fleet, you can define a combination of EC2 instance types to make up the desired capacity of your group. This is defined as a percentage of each type of purchasing option. Amazon EC2 Auto Scaling will maintain the desired cost optimization as your group scales in or out. Groups made up of mixed fleets still support the same lifecycle hooks, instance health checks, and scheduled scaling as a single-fleet group.

Regards

Osama

Connect to AKS cluster nodes

Posted on by Osama Mustafa in Azure, Cloud

sometimes you need to access AKS worker node to troubelshoot, but how to do that with AKS

Run the below command

kubectl get nodes

Output will give an idea about the worker nodes you have

Run a container image on the node by issuing the kubectl debug command in order to establish a connection to it. The following command begins the process of connecting to a privileged container that has been started on your node.

kubectl debug node/<node-name-you-wish-to-connect> -it --image=mcr.microsoft.com/dotnet/runtime-deps:6.0

Regards

Osama

AWS Load Balancing

Posted on by Osama Mustafa in AWS, Cloud

A load balancer distributes incoming application traffic across multiple targets, such as EC2 instances, in multiple Availability Zones to increase the availability of your application. A load balancer works with listeners. You can have more than one listener per load balancer. 

A listener checks for connection requests from clients, using the protocol and port that you configure. The load balancer forwards requests to one or more target groups, based on the rules that you define.

Each rule specifies a target group, condition, and priority. The traffic is forwarded to that group when the condition is met. You define a default rule for each listener. You can add rules that specify different target groups based on the content of the request. Each target group routes requests to one or more registered targets, for example EC2 instances, using the specified protocol and port number. You can register a target with multiple target groups.

Elastic Load Balancing

AWS Elastic Load Balancing (ELB) is one of the most widely used AWS services. It has been adopted by organizations of all sizes, in all geographies, and across every industry. ELBs automatically distribute traffic across multiple targets, provide high availability, incorporate security features, and perform health checks.

ELB features

ELB load balancers are the only load balancers available on AWS that natively connect users to your EC2 instances, container deployments, and AWS Lambda functions. Some key feature sets include the following:

  • High availability – ELB automatically distributes your traffic across multiple targets in a single Availability Zone or multiple Availability Zones. Examples of targets include EC2 instances, containers, and IP addresses.
  • Layer 4 or Layer 7 HTTP and HTTPS load balancing – You can load balance your HTTP or HTTPS applications for Layer 7-specific features. Alternatively, you can use strict Layer 4 load balancing for applications that rely purely on the TCP.
  • Security features – Use Amazon VPC to create and manage security groups associated with load balancers to provide additional networking and security options. You can also create an internal (non-internet-facing) load balancer.
  • Health checks – ELB load balancers can detect unhealthy targets, stop sending traffic to them, and spread the load across the remaining healthy targets.
  • Monitoring operations – To monitor the performance of your applications in real time, ELB integrates with CloudWatch metrics and provides request tracing.

Types of load balancers

Application Load Balancer

This load balancer functions at the application layer, the seventh layer of the Open Systems Interconnection (OSI) model. Application Load Balancers support the following: Content-based routing, applications that run in containers, and open standard protocols (WebSocket and HTTP/2). This type of balancer is ideal for advanced load balancing of HTTP and HTTPS traffic. 

Network Load Balancer

This load balancer is designed to handle tens of millions of requests per second while maintaining high throughput at ultra low-latency. Network Load Balancer operates at the connection level (Layer 4), routing connections to targets based on IP protocol data. Targets include EC2 instances, containers, and IP addresses. It is ideal for balancing TCP traffic.

Gateway Load Balancer

This load balancer makes it easy to deploy, scale, and manage your third-party virtual appliances. It provides one gateway for distributing traffic across multiple virtual appliances, and scales them up, or down, based on demand. This distribution reduces potential points of failure in your network and increases availability. Gateway Load Balancer transparently passes all Layer 3 traffic through third-party virtual appliances. It is invisible to the source and destination.

Classic Load Balancer

ELB common features

FeaturesApplication Load BalancerNetwork  Load BalancerGateway  Load Balancer
Health checks✔✔✔
CloudWatch metrics✔✔✔
Logging✔✔✔
Secure Sockets Layer (SSL) offloading✔✔
Connection draining✔✔✔
Preserve source IP address✔✔✔
Static IP address**✔
Lambda functions as a target✔✔✔
Redirects✔
Fixed-response actions✔

Regards

Osama

Principals and Identities

Posted on by Osama Mustafa in Uncategorized

IAM users

By default, a new Identity and Access Management (IAM) user has no permissions assigned to them. The user is not authorized to perform any AWS operations or access any AWS resources. An advantage of having individual IAM users is that you can assign permissions individually to each user. 

Setting permissions with IAM policies

To allow IAM users to create or modify resources and perform tasks:

  1. Create IAM policies that grant IAM users permission to access the specific resources, and API actions they will need.
  2. Attach the policies to the IAM users or groups that require those permissions.

Users only have the permissions specified in the policy. Most users have multiple policies. Together, they represent the permissions for that user.

IAM roles

IAM roles deliver temporary AWS credentials. They’re easy to manage because multiple employees and applications can use the same role. Use roles to delegate access to users, applications, or services that don’t normally have access to your AWS resources.

Roles can be assumed through the API call, using one of the following:

  • The console
  • AWS CLI
  • AssumeRole API
  • AWS Security Token Service (AWS STS)

The AssumeRole action returns a set of temporary security credentials consisting of an access key ID, a secret access key, and a security token. AssumeRole is typically used for cross-account access or federation.

Policy types

Identity-based policies

You attach managed and inline policies to IAM identities. This includes users, groups to which users belong, and roles.

Resource-based policies

You attach inline policies to resources. The most common examples of resource-based policies are Amazon S3 bucket policies and IAM role trust policies.

IAM permissions boundaries

An entity’s permissions boundary allows it to perform only the actions that are allowed by both its identity-based policies and its permissions boundaries. Use a managed policy as the permissions boundary for an IAM entity (user or role). The managed policy limits the permissions that the identity-based policy can grant to an entity. It does not grant permissions. 

AWS Organizations service control policies (SCPs)

Use Organizations SCPs to define the maximum permissions for account members of an organization or organizational unit (OU).

Access control lists (ACLs)

Use ACLs to control which principals in other accounts can access a resource to which the ACL is attached. ACLs are similar to resource-based policies. However, they are the only policy type that does not use the JSON policy document structure. We will discuss ACLs in more detail in Module 3.

Policy elements

ElementDescriptionEffectUse Allow or Deny to indicate whether the policy allows or denies access. PrincipalIf you create a resource-based policy, you must indicate the account, user, role, or federated user to which you want to allow or deny access. If you are creating an IAM permissions policy to attach to a user or role, you cannot include this element. The principal is implied as that user or role. ActionInclude a list of actions that the policy allows or denies.ResourceIf you create an IAM permissions policy, you must specify a list of resources to which the actions apply. If you create a resource-based policy, this element is optional. If you do not include this element, the resource to which the action applies is the resource to which the policy is attached.ConditionSpecify the circumstances under which the policy grants permission.

Explicit allow and explicit deny

Use IAM policies to allow and deny access to resources. 

{
"Effect": "Allow",
"Action": ["s3:ListObject”,  “s3:GetObject”],
"Resource": ["arn:aws:s3:::DOC-EXAMPLE-BUCKET/*"]
}

The following policy denies access.

{
"Effect": "Deny",
"Action": [”ec2:*", "s3:*"],
"Resource": “*”
}

When a principal tries to use the console, the AWS API, or the AWS CLI, that principal sends a request to AWS. When an AWS service receives the request, AWS completes several steps to determine whether to grant or deny the request.

By default, all requests are denied. This is called an implicit deny. The AWS enforcement code evaluates all policies within the account that apply to the request. These include Organizations SCPs, resource-based policies, IAM permissions boundaries, role session policies, and identity-based policies. In all those policies, the enforcement code looks for a deny statement that applies to the request. This is called an explicit deny. If the code finds even one explicit deny that applies, the code returns a final decision of deny. If there is no explicit deny, the code continues.

How IAM policies interact with SCPs

An SCP is a type of organization policy that you can use to manage permissions in your organization. 

Attach identity-based or resource-based policies to IAM users, or to the resources in your organization’s accounts. Attach an SCP to an Organizations entity (root, OU, or account) to define a guardrail. The SCP sets limits upon the actions that the IAM users and roles in the affected accounts can perform.

Regards

Osama

AWS Infrastructure

Posted on by Osama Mustafa in AWS, Cloud

The AWS Global Cloud Infrastructure is the most secure, extensive, and reliable cloud platform, offering over 200 fully featured services from data centers globally.

AWS Data Center

AWS pioneered cloud computing in 2006 to provide rapid and secure infrastructure. AWS continuously innovates on the design and systems of data centers to protect them from man-made and natural risks. Today, AWS provides data centers at a large, global scale. AWS implements controls, builds automated systems, and conducts third-party audits to confirm security and compliance. As a result, the most highly-regulated organizations in the world trust AWS every day.

Availability Zone – AZ

An Availability Zone (AZ) is one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region. Availability Zones are multiple, isolated areas within a particular geographic location. When you launch an instance, you can select an Availability Zone or let AWS choose one for you. If you distribute your instances across multiple Availability Zones and one instance fails, you can design your application so that an instance in another Availability Zone can handle requests.

Region

Each AWS Region consists of multiple, isolated, and physically separate Availability Zones within a geographic area. This achieves the greatest possible fault tolerance and stability. In your account, you determine which Regions you need. You can run applications and workloads from a Region to reduce latency to end users. You can do this while avoiding the upfront expenses, long-term commitments, and scaling challenges associated with maintaining and operating a global infrastructure.

AWS Local Zone

AWS Local Zones can be used for highly demanding applications that require single-digit millisecond latency to end users. Media and entertainment content creation, real-time multiplayer gaming, and Machine learning hosting and training are some use cases for AWS Local Zones.

CloudFront – Edge Location

An edge location is the nearest point to a requester of an AWS service. Edge locations are located in major cities around the world. They receive requests and cache copies of your content for faster delivery.

Regards

Osama

AWS Snow Family memberS

Posted on by Osama Mustafa in AWS, Cloud

The AWS Snow Family is a collection of physical devices that help to physically transport up to exabytes of data into and out of AWS. 

AWS Snow Family is composed of AWS Snowcone, AWS Snowball, and AWS Snowmobile.

These devices offer different capacity points, and most include built-in computing capabilities. AWS owns and manages the Snow Family devices and integrates with AWS security, monitoring, storage management, and computing capabilities.  

AWS Snowcone

AWS Snowcone is a small, rugged, and secure edge computing and data transfer device. 

It features 2 CPUs, 4 GB of memory, and 8 TB of usable storage.

AWS Snowball

AWS Snowball offers two types of devices:

  • Snowball Edge Storage Optimized devices are well suited for large-scale data migrations and recurring transfer workflows, in addition to local computing with higher capacity needs.
    • Storage: 80 TB of hard disk drive (HDD) capacity for block volumes and Amazon S3 compatible object storage, and 1 TB of SATA solid state drive (SSD) for block volumes. 
    • Compute: 40 vCPUs, and 80 GiB of memory to support Amazon EC2 sbe1 instances (equivalent to C5).
  • Snowball Edge Compute Optimized provides powerful computing resources for use cases such as machine learning, full motion video analysis, analytics, and local computing stacks.
    • Storage: 42-TB usable HDD capacity for Amazon S3 compatible object storage or Amazon EBS compatible block volumes and 7.68 TB of usable NVMe SSD capacity for Amazon EBS compatible block volumes. 
    • Compute: 52 vCPUs, 208 GiB of memory, and an optional NVIDIA Tesla V100 GPU. Devices run Amazon EC2 sbe-c and sbe-g instances, which are equivalent to C5, M5a, G3, and P3 instances.

AWS Snowmobile

AWS Snowmobile is an exabyte-scale data transfer service used to move large amounts of data to AWS. 

You can transfer up to 100 petabytes of data per Snowmobile, a 45-foot long ruggedized shipping container, pulled by a semi trailer truck.

Cheers

Osama

Create IAM Users – OCI

Posted on by Osama Mustafa in Cloud, OCI

You have the ability to establish users for Oracle Cloud Infrastructure Identity and Access Management (IAM) for user situations that are not as common.

  • Open the navigation menu and click Identity & Security. Under Identity, click Users.
  • Click Create user and then select IAM User.
  • Fill the required fields, and click Create.
  • Add the user to an IAM group with specific access.
    • Under Identity, select Groups
    • From the groups list, click the group to which you want to add the user.
    • Click Add User to Group.
    • In the Add User to Group dialog, select the user you created from the drop-down list in the Users field, and click Add.
  • Create the user’s password.
    • From the Group Members table on the Group Details screen, select the user you added.
    • Click Create/Reset Password. The Create/Reset Password dialog is displayed with a one-time password listed.
    • Click Copy, then Close.
  • Welcome to OCI

Regards

Osama

Create a Bastion – OCI

Posted on by Osama Mustafa in Cloud, OCI

What is a Bastion?

It’s essential to consider the security implications before allowing direct access to cloud services and resources, particularly as the latter expands. Some individuals get around this problem by setting up a virtual machine within the virtual cloud network and linking it to all the cloud services. This cuts down on publicly accessible services while facilitating connections for developers and system administrators. This virtual machine (VM) is like a manual bastion or leap box.

Create a Bastion

  • Connect to Oracle’s cloud service. To access the main menu, choose the hamburger icon in the upper left corner.
  • On the menu select “Identity & Security > Bastion”.
  • Select the compartment and click the “Create bastion” button.
  • Enter the bastion name and select the VCN and subnet for the bastion. We need to enter a CIDR block allowlist. In this case I’ve used the subnet for my IP address from my internet service provider. Click the “Create bastion” button.
  • Click on the “Create session” button.
  • Connect

Our previously copied connection information should look something like this at this point.

ssh -i  -N -L :ip-connection:22 -p 22 ocid1.bastionsession.oc1.uk-london-1.amaa...3acq@host.bastion.uk-london-1.oci.oraclecloud.com

Regards

Osama

AWS Support

Posted on by Osama Mustafa in Uncategorized

AWS offers four different Support plans to help you troubleshoot issues, lower costs, and efficiently use AWS services. 

You can choose from the following Support plans to meet your company’s needs: 

  • Basic
  • Developer
  • Business
  • Enterprise

Basic Support

Basic Support is free for all AWS customers. It includes access to whitepapers, documentation, and support communities. With Basic Support, you can also contact AWS for billing questions and service limit increases.

With Basic Support, you have access to a limited selection of AWS Trusted Advisor checks. Additionally, you can use the AWS Personal Health Dashboard, a tool that provides alerts and remediation guidance when AWS is experiencing events that may affect you. 

If your company needs support beyond the Basic level, you could consider purchasing Developer, Business, or Enterprise Support.

Developer, Business, and Enterprise Support

The Developer, Business, and Enterprise Support plans include all the benefits of Basic Support, in addition to the ability to open an unrestricted number of technical support cases. These three Support plans have pay-by-the-month pricing and require no long-term contracts.

The information in this course highlights only a selection of details for each Support plan. A complete overview of what is included in each Support plan, including pricing for each plan, is available on the AWS Support site.

In general, for pricing, the Developer plan has the lowest cost, the Business plan is in the middle, and the Enterprise plan has the highest cost. 

Developer Support

Customers in the Developer Support plan have access to features such as:

  • Best practice guidance
  • Client-side diagnostic tools
  • Building-block architecture support, which consists of guidance for how to use AWS offerings, features, and services together

For example, suppose that your company is exploring AWS services. You’ve heard about a few different AWS services. However, you’re unsure of how to potentially use them together to build applications that can address your company’s needs. In this scenario, the building-block architecture support that is included with the Developer Support plan could help you to identify opportunities for combining specific services and features.

Business Support

Customers with a Business Support plan have access to additional features, including: 

  • Use-case guidance to identify AWS offerings, features, and services that can best support your specific needs
  • All AWS Trusted Advisor checks
  • Limited support for third-party software, such as common operating systems and application stack components

Suppose that your company has the Business Support plan and wants to install a common third-party operating system onto your Amazon EC2 instances. You could contact AWS Support for assistance with installing, configuring, and troubleshooting the operating system. For advanced topics such as optimizing performance, using custom scripts, or resolving security issues, you may need to contact the third-party software provider directly.

Enterprise Support

In addition to all the features included in the Basic, Developer, and Business Support plans, customers with an Enterprise Support plan have access to features such as:

  • Application architecture guidance, which is a consultative relationship to support your company’s specific use cases and applications
  • Infrastructure event management: A short-term engagement with AWS Support that helps your company gain a better understanding of your use cases. This also provides your company with architectural and scaling guidance.
  • A Technical Account Manager