Category: DevOps Tools

I want to share this image with you, since it’s very useful, this is the steps how to troubleshoot with kubernetes :-

Cheers

Osama

Happy New Year Everyone

This is the first blog post for the 2020, i Wish everyone will have healthy and wonderful year, may your dreams come true.

I post recently or let’s say Last Year about Full automation project using DevOps tools, and i didn’t to be honest except that much download and questions on that post. you can read it from here.

I decided to create new project but this time to show the power of DevOps and how to use it more in your daily Job, task or even configuration.

The idea of this project like the following:-

• You have two code, one Go based application, one Java-based application. Both are providing an HTTP service with the same endpoints.
• The Endpoints which is :-

We have to setup Load Balancer for this application to be like the following :-

traffic distribution should be as follows: 70% of the requests are going to the application written in Go, 30% of the requests are going to the application written in Java, also i will do it using Docker

I upload the code, and the application ( the two part which is Go application and Java) to my Github HERE, all the configuration has been uploaded to my github,

The Solution files like the below;

• docker-compose.yml file in root directory is the main compose file for setting up containers for all services
• go-app directory contains binary of Golang Application and Dockerfile of relavant setup
• java-app directory contains binary of Java Application and Dockerfile of relavant setup
• load-balancer directory contains nginx.conf file which is configuration file of Nginx and have load balancer rules written in it. And containers a Dockerfile for setting up Nginx with defined configurations

The final architecture will be like this instead of the image you saw above

Enjoy And Happy New Year

Osama Mustafa (The Guy who known as OsamaOracle)

## Problem definition

The aim of test is to create a simple HTTP service that stores and returns configurations that satisfy certain conditions. Since I love automating things, the service should be automatically deployed to kubernetes.

You can read more about the project, once you access to my GitHub using the README.MD, I explained the project step by step also the documentation explained every thing.

the code has been uploaded to GitHub, include to this, the documentation uploaded to Slide-share.

The code configuration here

The documentation here

Enjoy

Osama

What is Terraform?

What is Terraform?Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

by HashiCorp, an AWS Partner Network (APN) Advanced Technology Partner and member of the AWS DevOps Competency,

To configure terraform, you have to deal and create files with extension “.tf” like the following :-

• Providers : – A provider is responsible for understanding API interactions and exposing resources. Providers generally are an IaaS (e.g. AWS, GCP, Microsoft Azure, OpenStack), PaaS (e.g. Heroku), or SaaS services (e.g. Terraform Enterprise, DNSimple, CloudFlare).
• Resource : – The resource block creates a resource of the given TYPE (first parameter) and NAME (second parameter). The combination of the type and name must be unique. Within the block (the { }) is configuration for the resource. The configuration is dependent on the type, and is documented for each resource type in the providers section. Providers : – A provider is responsible for understanding API interactions and exposing resources. Providers generally are an IaaS (e.g. AWS, GCP, Microsoft Azure, OpenStack), PaaS (e.g. Heroku), or SaaS services (e.g. Terraform Enterprise, DNSimple, CloudFlare).
• Variables: – defined by stating a name, type and a default value. However, the type and default values are not strictly necessary. Terraform can deduct the type of the variable from the default or input value.
• VPC : which is used to define security group, subnets and ports in AWS environment.

In this post, i will do the following with terraform, you have to create and sign up for AWS account so you will be able to test this code and use terraform, what will i do here is

• creating private subnet
• creating public subnet
• an SSH bastion on the public subnet only.
• adding two ec2 to private subnets.

Let’s Start, as mentioned earlier you should have 4 files, provider.tf, resource.tf, varaibale.tf, and vpc.tf

Provider.tf

As you see from the below file, it’s contains our cloud provider and the region depends on varaible that will be defined later

# Define AWS as our provider
provider "aws" {
  region = "${var.aws_region}"}

resource.tf

The reosuce file where i create ssh key, to create it there are different way to do it, for example in my case i used puttygen then copied the key over here and save the public/private key so i can use them later, the other way, which is automaitcally generated., then i define which ami i will be used for the server/ec2 that will be created in AWS and the ID for this ami will be defiend in varaiable file,

# Define SSH key pair for the instances
resource "aws_key_pair" "default" {
  key_name = "terraform_key"
  public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAtGCFXHeo9igKRzm3hNG5WZKXkQ3/NqQc1WPN8pSrEb0ZjW8mTKXRWzePuVYXYP9txqEKQmJJ1bk+pYX/zDdhJg/yZbZGH4V0LvDY5X5ndnAjN6CHkS6iK2EK1GlyJs6fsa+6oUeH23W2w49GHivSsCZUZuaSwdORoJk9QLeJ7Qz+9YQWOk0efOr+eIykxIDwR71SX5X65USbR8JbuT2kyrp1kVKsmPMcfy2Ehzd4VjShlHsZZlbzKTyfgaX/JJmYXO5yD4VLSjY8BVil4Yq/R9Tkz9pFxCG230XdFWCFEHSqS7TIDFbhPkp18jna6P6hlfNb9WM2gVZbYvr1MMnAVQ== rsa-key-20190805"
}

# Define a server 1 inside the public subnet
resource "aws_instance" "public_server" {
   ami  = "${var.ami}"
   instance_type = "t1.micro"
   key_name = "${aws_key_pair.default.id}"
   subnet_id = "${aws_subnet.public-subnet.id}"
   vpc_security_group_ids = ["${aws_security_group.sgpub.id}"]
   associate_public_ip_address = true
   source_dest_check = false
   user_data = "${file("userdata.sh")}"

  tags = {
    Name = "public_server"
  }
}

# Define database inside the private subnet
resource "aws_instance" "private_server1" {
   ami  = "${var.ami}"
   instance_type = "t1.micro"
   key_name = "${aws_key_pair.default.id}"
   subnet_id = "${aws_subnet.private-subnet.id}"
   vpc_security_group_ids = ["${aws_security_group.sgpriv.id}"]
   associate_public_ip_address = true
   source_dest_check = false
   user_data = "${file("userdata.sh")}"

  tags = {
    Name = "private_server1"
  }
}

# Define database inside the private subnet
resource "aws_instance" "private_server2" {
   ami  = "${var.ami}"
   instance_type = "t1.micro"
   key_name = "${aws_key_pair.default.id}"
   subnet_id = "${aws_subnet.private-subnet.id}"
   vpc_security_group_ids = ["${aws_security_group.sgpriv.id}"]
   associate_public_ip_address = true
   source_dest_check = false
   user_data = "${file("userdata.sh")}"

  tags = {
    Name = "private_server2"
  }
}

variables.tf

as you see from the below, the variables file where i defined all the infomation such as AWS region, THe Subnet that will be using, the AMI ID ( you can find it by access to aws console and copy the id), finally the SSH key path in my server/ec2.

variable "aws_region" {
  description = "Region for the VPC"
  default = "ap-southeast-1"
}

variable "vpc_cidr" {
  description = "CIDR for the VPC"
  default = "10.0.0.0/16"
}

variable "public_subnet_cidr" {
  description = "CIDR for the public subnet"
  default = "10.0.1.0/24"
}

variable "private_subnet_cidr" {
  description = "CIDR for the private subnet"
  default = "10.0.2.0/24"
}

variable "ami" {
  description = "Amazon Linux AMI"
  default = "ami-01f7527546b557442"
}

variable "key_path" {
  description = "SSH Public Key path"
  default = "~/.ssh/id_rsa.pub"
}

VPC.tf

This define anything related to network, security group and subnects in AWS Cloud, as you see from the file, i assigned one ec2/public to my public subnect in the vpc, the two ec2/private to my private subnect in the vpc file, then i condifured which port will be used on the public subnect which is ssh (22), http (80), TCP (443) and ICMP, the same for private but i open the connection between public and private using ssh which mean you can access private only by access the public server this done by the subnet also open MYSQL port which is 3306

# Define our VPC
resource "aws_vpc" "default" {
  cidr_block = "${var.vpc_cidr}"
  enable_dns_hostnames = true

  tags  ={
    Name = "test-vpc"
  }
}

# Define the public subnet
resource "aws_subnet" "public-subnet" {
  vpc_id = "${aws_vpc.default.id}"
  cidr_block = "${var.public_subnet_cidr}"
  availability_zone = "ap-southeast-1a"

  tags =  {
    Name = "PublicSubnet"
  }
}

# Define the private subnet
resource "aws_subnet" "private-subnet" {
  vpc_id = "${aws_vpc.default.id}"
  cidr_block = "${var.private_subnet_cidr}"
#  availability_zone = "ap-southeast-1"

  tags =  {
    Name = "Private Subnet"
  }
}

# Define the internet gateway
resource "aws_internet_gateway" "gw" {
  vpc_id = "${aws_vpc.default.id}"

  tags =  {
    Name = "VPC IGW"
  }
}

# Define the route table
resource "aws_route_table" "web-public-rt" {
  vpc_id = "${aws_vpc.default.id}"

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = "${aws_internet_gateway.gw.id}"
  }

  tags =  {
    Name = "PublicSubnetRT"
  }
}

# Assign the route table to the public Subnet
resource "aws_route_table_association" "web-public-rt" {
  subnet_id = "${aws_subnet.public-subnet.id}"
  route_table_id = "${aws_route_table.web-public-rt.id}"
}

# Define the security group for public subnet
resource "aws_security_group" "sgpub" {
  name = "vpc_test_pub"
  description = "Allow incoming HTTP connections & SSH access"

  ingress {
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port = 443
    to_port = 443
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

# The ICMP packet does not have source and destination port numbers because it was designed to 
# communicate network-layer information between hosts and routers, not between application layer processes.

  ingress {
    from_port = -1
    to_port = -1
    protocol = "icmp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port = 22
    to_port = 22
    protocol = "tcp"
    cidr_blocks =  ["0.0.0.0/0"]
  }
  egress {
    from_port       = 0
    to_port         = 0
    protocol        = "-1"
    cidr_blocks     = ["0.0.0.0/0"]
  }

  vpc_id="${aws_vpc.default.id}"

  tags =  {
    Name = "Public Server SG"
  }
}

# Define the security group for private subnet
resource "aws_security_group" "sgpriv"{
  name = "sg_test_web"
  description = "Allow traffic from public subnet"

# You can delete this port, add it her to make it as real environment
  ingress {
    from_port = 3306
    to_port = 3306
    protocol = "tcp"
    cidr_blocks = ["${var.public_subnet_cidr}"]
  }

  ingress {
    from_port = -1
    to_port = -1
    protocol = "icmp"
    cidr_blocks = ["${var.public_subnet_cidr}"]
  }

  ingress {
    from_port = 22
    to_port = 22
    protocol = "tcp"
    cidr_blocks = ["${var.public_subnet_cidr}"]
  }

  vpc_id = "${aws_vpc.default.id}"

  tags =  {
    Name = "PrivateServerSG"
  }
}

userdata.sh

This file used to run the command that should be run using terraform.

#!/bin/sh
set -x
# output log of userdata to /var/log/user-data.log
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
yum install -y httpd
service httpd start
chkonfig httpd on
echo "<html><h3>Welcome to the Osama Website</h3></html>" > /var/www/html/index.html

Once you preapre these files, create free tier aws ec2 machines and upload these files to folder “terraform-example”, now with these files the output will be like the following :-

• three ec2 server with the following names: private server1 , private server 2 , public server1
• two seucrity group for public and private
• Key pair called terraform_key
• the AWS region will singapore.

Now Run the following command : –

terraform plan

wait the output, the command should run successfully without any errors, you can add attribute “-output NAME_OF_LOG_FILE”, this will list you the output of the command.

terraform apply

the above command will apply everything to AWS enviroment and allow you create this environment within less than 1 min.

Amazing huh ? This is Called infrastructure as code.

The files uploaded to my Github here

Cheers

Osama