DevOps Tools – Technology Geek

Install k8s as one control plane and one worker node

Posted on August 10, 2022 by Osama Mustafa in DevOps Tools

The first thing you will need to do which is configure the two servers, either you can choose one of the following options:-

VMWARE
Cloud

Master Node Setup

Step #1

Create configuration file for containerd:

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

After the above step, you need to load the modules.

sudo modprobe overlay
sudo modprobe br_netfilter

Step #2

Set system configurations for Kubernetes networking

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

Apply new settings

sudo sysctl --system

Step #3

Install Containerd

sudo apt-get update && sudo apt-get install -y containerd

Step #4

Create default configuration file for containerd

sudo mkdir -p /etc/containerd

Generate default containerd configuration and save to the newly created default file

sudo containerd config default | sudo tee /etc/containerd/config.toml

Load the new configuration

sudo systemctl restart containerd
sudo systemctl status containerd

Step #5

Disable Swap

sudo swapoff -a

Step #6

Install dependency packages:

sudo apt-get update && sudo apt-get install -y apt-transport-https curl

Download and add GPG key

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

Add Kubernetes to repository list

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

sudo apt-get update

Step #6

Install Kubernetes packages (Note: If you get a dpkg lock message, just wait a minute or two before trying the command again):

sudo apt-get install -y kubelet=1.24.0-00 kubeadm=1.24.0-00 kubectl=1.24.0-00

Just in case Turn off automatic updates

sudo apt-mark hold kubelet kubeadm kubectl

The above steps should be done on the worker node even if you have 3 or 4.

Initialize the Cluster

Initialize the Kubernetes cluster on the control plane node using kubeadm (Note: This is only performed on the Control Plane Node):

sudo kubeadm init --pod-network-cidr 192.168.0.0/16 --kubernetes-version 1.24.0

Set kubectl access:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

You can test your cluster by run

kubectl get nodes

Install the Calico Network Add-On

On the control plane node, install Calico Networking

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

Check status of the control plane node:

kubectl get nodes

Join the Worker Nodes to the Cluster

In the control plane node, create the token and copy the kubeadm join command (NOTE:The join command can also be found in the output from kubeadm init command):

kubeadm token create --print-join-command

Copy the output

Worker node Setup.

from the above command of Kubeadm join run it using sudo command.

In the control plane node, view cluster status (Note: You may have to wait a few moments to allow all nodes to become ready)

kubectl get nodes

Cheers

Enjoy the DevOps

infrastructure as code tools

Posted on July 14, 2022 by Osama Mustafa in DevOps Tools

Infrastructure as code is one of the most common uses to set up a cloud environment, either Cloudformation, Oracle resource stack, or 3rd party such as Pulumi or terraform.

For this, I would like to share the tools I use for the perfect IaC tools that could be useful for someone

Enjoy

Osama

New Cloud Project

Posted on November 12, 2021 by Osama Mustafa in DevOps Tools

The Idea of this project the following :

You need to develop and deploy a python app that writes a new file to S3 on every execution. These files need to be maintained only for 24h.

The content of the file is not important, but add the date and time as prefix for you files name.

The name of the buckets should be the following ones for QA and Staging respectively:

qa-FIRSTNAME-LASTNAME-platform-challenge

staging-FIRSTNAME-LASTNAME-platform-challenge

The app will be running as a docker container in a Kubernetes cluster every 5 minutes. There is a Namespace for QA and a different Namespace for Staging in the cluster. You don’t need to provide tests but you need to be sure the app will work.

Github HERE

Cheers

Osama

Creating a Helm Chart

Posted on July 16, 2021 by Osama Mustafa in DevOps Tools

Helm is the first application package manager running atop Kubernetes. It allows describing the application structure through convenient helm-charts and managing it with simple commands. Because it’s a huge shift in the way the server-side applications are defined, stored and managed.

Helm Charts provide “push button” deployment and deletion of apps, making adoption and development of Kubernetes apps easier for those with little container or microservices experience. Apps deployed from Helm Charts can then be leveraged together to meet a business need, such as CI/CD or blogging platforms.

Install Helm

Use curl to create a local copy of the Helm install script

 curl https://raw.githubusercontent.com/helm/helm/master/scripts/get > /tmp/get_helm.sh

cat /tmp/get_helm.sh

Use chmod to modify access permissions for the install script

chmod 700 /tmp/get_helm.sh

Set the version to v2.8.2

 DESIRED_VERSION=v2.8.2 /tmp/get_helm.sh

Ensure Helm uses the correct stable chart repo (the default one used by Helm has been decommissioned)

helm init --stable-repo-url https://charts.helm.sh/stable

Initialize Helm:

helm init --wait

Give Helm the permissions it needs to work with Kubernetes

kubectl --namespace=kube-system create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default

Make sure our configuration is working properly

Create a Helm Chart

mkdir charts

cd charts

Create the chart for httpd

helm create httpd

Verify our directory was created correctly by running ls command
Navigate to the httpd directory by using cd command “cd httpd
view the files and directory cd httpd/
This directory contains two files: Chart.yaml and values.yaml. We need to edit the values.yaml file.

Open values.yaml

Under image, change the repository to httpd.
Change the tag to latest.
Under service, change type to NodePort.

replicaCount: 1
image:
  repository: httpd
  tag: latest
  pullPolicy: IfNotPresent
service:
  type: NodePort
  port: 80

ingress:
  enabled: false
  annotations: {}
  path: /
  hosts:
    - chart-example.local

  tls: []
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}

Create Your Application Using Helm
Back to directory httpd and run the command

helm install --name my-httpd ./httpd/

Copy the commands listed under the NOTES section of the output, and then paste and run them. It should return the private IP address and port number of our application.

Let’s check to see if our pods have come online

kubectl get pods
kubectl get services

Finished

Thank you for reading

Osama

Scaling Pods in Kubernetes

Posted on July 15, 2021 by Osama Mustafa in DevOps Tools

Continue to pervious post of Configure Kubernetes on my blog.

This post will discuss how to scale the pods, I will assume the Kubernetes installed if not back to the above post.

If you did these steps below , you can skip

Initialize the cluster

kubeadm init --pod-network-cidr=10.244.0.0/16 --kubernetes-version=v1.11.3

As mentioned the command will generate commands like the picture.

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install Flannel

Flannel is an open-source virtual network project managed by CoreOS network designed for Kubernetes. Each host in a flannel cluster runs an agent called flanneld . It assigns each host a subnet, which acts as the IP address pool for containers running on the host.

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml

Create deployment

vi deployment.yml

apiVersion: apps/v1

kind: Deployment

metadata:

  name: httpd-deployment

  labels:

    app: httpd

spec:

  replicas: 3

  selector:

    matchLabels:

      app: httpd

  template:

    metadata:

      labels:

        app: httpd

    spec:

      containers:

      - name: httpd

        image: httpd:latest

        ports:

        - containerPort: 80

Spin up the deployment

kubectl create -f deployment.yml

Create the service

vim service.yml

kind: Service

apiVersion: v1

metadata:

  name: service-deployment

spec:

  selector:

    app: httpd

  ports:

  - protocol: TCP

    port: 80

    targetPort: 80

  type: NodePort

kubectl create -f service.yml

Scale the deployment up to 5 replicas.

vi deployment.yml

Change the number of replicas to 5:

spec: replicas: 5

Apply the changes:

kubectl apply -f deployment.yml

Enjoy

Hope it’s useful

Osama

Setting up a Kubernetes Cluster with Docker – CentOS

Posted on July 13, 2021 by Osama Mustafa in DevOps Tools

Moving to Docker container series blog post, I choose to continue with Kubernetes and discuss it more start with configuration and installation.

This configuration discuss on-premise side and to do that you have at least 2 servers

Server	purpose	description
The Master	node which controls and manages a set of worker nodes (workloads runtime) and resembles a cluster in Kubernetes. A master node has the following components to help manage worker nodes: … Kube-Controller-Manager, which runs a set of controllers for the running cluster.
The worker node	A Node is a worker machine in Kubernetes and may be either a virtual or a physical machine, depending on the cluster. … Each Node is managed by the Master. A Node can have multiple pods, and the Kubernetes master automatically handles scheduling the pods across the Nodes in the cluster.

Configure The Kubernetes cluster

On all nodes, add the Kubernetes repo to /etc/yum.repos.d:

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo

[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF

Disable SELinux:

sudo setenforce 0

sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

Install Kubernetes

sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

Enable and start kubelet

sudo systemctl enable --now kubelet

From Node 1 (Master) , initialize the controller node, and set the code network CIDR to 10.244.0.0/16 or depends on your IP range :

kubeadm init --pod-network-cidr=10.244.0.0/16

From Node 1 (Master), check the status of your cluster:

 docker ps -a

Repeat this step on the worker nodes. Can the worker nodes see the cluster

Once you are done, the init command will create a commands for you , you needs to run them or you will have permission issues.

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

Copy the kubeadm join command, then paste and run it in your worker nodes terminal windows.

From the worker nodes, verify that they can see the cluster

docker ps -a

From Node 1 (Master), check the status of the nodes

 kubectl get nodes

Now, Kubernetes installed but it’s empty to have pods or services the next will be for you, it can be change depends on your application type but it’s Just for testing to show the reader how it’s goes.

Install flannel

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Create POD

vim pod.yml

apiVersion: v1

kind: Pod

metadata:

  name: nginx-pod-demo

  labels:

    app: nginx-demo

spec:

  containers:

  - image: nginx:latest

    name: nginx-demo

    ports:

    -  containerPort: 80

    imagePullPolicy: Always

Create the pod

 kubectl create -f pod.yml

Check the status of the pod

kubectl get pods

Create Services

vim service.yml

apiVersion: v1

kind: Service

metadata:

  name: service-demo

spec:

  selector:

    app: nginx-demo

  ports:

  - protocol: TCP

    port: 80

    targetPort: 80

  type: NodePort

Create the service

kubectl apply -f service.yml

Run the following command to view the service

 kubectl get services

Take note of the service-demo port number.

In a web browser, navigate to the public IP address for a server in the cluster, and verify connectivity:

<PUBLIC_IP_ADDRESS>:<SERVICE_DEMO_PORT_NUMBER>

Enjoy the automation🤗

Osama

Using Grafana with Prometheus for Alerting and Monitoring

Posted on July 12, 2021July 12, 2021 by Osama Mustafa in DevOps Tools

This post continue to the pervious one which discussing “Monitor the Container using Prometheus” To use Grafana we need to do the following :-

The first thing we need to do is create a daemon.json file for Docker, Once /etc/docker/daemon.json is open in the vi text editor, add the following:

{ "metrics-addr" : "0.0.0.0:9323", "experimental" : true }

Restart The docker

systemctl restart docker

Update the firewall rules to communicate with Prometheus Server

firewall-cmd --zone=public --add-port=9323/tcp

Update Promotheus

Edit the Prometheus from the pervious post to be like the below , vi prometheus.yml

scrape_configs:

  - job_name: prometheus

    scrape_interval: 5s

    static_configs:

    - targets:

      - prometheus:9090

      - node-exporter:9100

      - pushgateway:9091

      - cadvisor:8080

 

  - job_name: docker

    scrape_interval: 5s

    static_configs:

    - targets:

      - <PRIVATE_IP_ADDRESS>:9323

Edit Docker-compose also from the pervious post

vi ~/docker-compose.yml

prometheus:

    image: prom/prometheus:latest

    container_name: prometheus

    ports:

      - 9090:9090

    command:

      - --config.file=/etc/prometheus/prometheus.yml

    volumes:

      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro

    depends_on:

      - cadvisor

  cadvisor:

    image: google/cadvisor:latest

    container_name: cadvisor

    ports:

      - 8080:8080

    volumes:

      - /:/rootfs:ro

      - /var/run:/var/run:rw

      - /sys:/sys:ro

      - /var/lib/docker/:/var/lib/docker:ro

  pushgateway:

    image: prom/pushgateway

    container_name: pushgateway

    ports:

      - 9091:9091

  node-exporter:

    image: prom/node-exporter:latest

    container_name: node-exporter

    restart: unless-stopped

    expose:

      - 9100

  grafana:

    image: grafana/grafana

    container_name: grafana

    ports:

      - 3000:3000

    environment:

      - GF_SECURITY_ADMIN_PASSWORD=password

    depends_on:

      - prometheus

      - cadvisor

Run Docker compose command

docker-compose up -d

Check Grafana if it’s working by

http://PUBLIC_IP_ADDRESS:3000

Once you access you have to do the following to connect Grafana with Prometheus

Adding DataSource

In the Grafana Home Dashboard, click the Add data source icon. For Name, type “Prometheus”. Click into the Type field, and select Prometheusfrom the dropdown. Under URL, select http://localhost:9090. (But we’re going to change this in a moment.) copy the private IP address of your server. Then, replace “localhost” in the URL with the private IP address. (It should look like this: http://PRIVATE_IP_ADDRESS:9090).

Add the Docker Dashboard to Grafana

lick the plus sign (+) on the left side of the Grafana interface, and click Import. Then, Open the JSON file Uploaded to my GitHub here. Copy the contents of the file to your clipboard.

We now have our Grafana visualization. In the upper right corner, click on Refresh every 5m and select Last 5 minutes.

Final Results

Enjoy

Osama

Monitoring Containers with Prometheus

Posted on July 10, 2021 by Osama Mustafa in DevOps Tools

Using Prometheus, you can monitor application metrics like throughput (TPS) and response times of the Kafka load generator (Kafka producer), Kafka consumer, and Cassandra client. Node exporter can be used for monitoring of host hardware and kernel metrics.

Create a prometheus.yml File

In root’s home directory, create prometheus.yml

vi prometheus.yml

We’ve got to stick a few configuration lines in here. When we’re done, it should look like this

scrape_configs:

- job_name: cadvisor

  scrape_interval: 5s

  static_configs:

  - targets:

    - cadvisor:8080

Create a docker-compose.yml file

version: '3'

services:

  prometheus:

    image: prom/prometheus:latest

    container_name: prometheus

    ports:

      - 9090:9090

    command:

      - --config.file=/etc/prometheus/prometheus.yml

    volumes:

      - ./prometheus.yml:/etc/prometheus/prometheus.yml

    depends_on:

      - cadvisor

    

  cadvisor:

    image: google/cadvisor:latest

    container_name: cadvisor

    ports:

      - 8080:8080

    volumes:

      - /:/rootfs:ro

      - /var/run:/var/run:rw

      - /sys:/sys:ro

      - /var/lib/docker:/var/lib/docker:ro

In order to stand up the environment, we’ll run this

docker-compose up -d

And to see if everything stood up properly, let’s run a quick docker ps. The output should show four containers: prometheus, cadvisor, nginx, and redis.

Let’s so see in a web browser as well. and browse to it, using the correct port number: http://<IP_ADDRESS>:9090/graph/

investigating CAdvisor

In a browser, navigate to http:// <IP_ADDRESS> :8080/containers/. Take a peek around, then change the URL to one of our container names (like nginx) so we’re at http://:8080/docker/nginx/.

If we run docker stats, we’re going to get some output that looks a lot like docker ps, but this stays open and reports what’s going on as far as the various aspects (CPU and memory usage, etc.) of our containers.

docker stats --format "table {{.Name}} {{.ID}} {{.MemUsage}} {{.CPUPerc}}"

Regards 🤞😁

Osama

Dockerize a Flask Application

Posted on July 6, 2021 by Osama Mustafa in DevOps Tools

The Flask Application uploaded to my GitHub Here

I will dockerize the above application and show you the steps to do that

Let’s Start 🤞

First will add some files i don’t want to Dockerignore file

vim .dockerignore

.dockerignore

Dockerfile

.gitignore

Pipfile.lock

migrations/

Write the dockerfile, which is already included to the above Repo vim Dockerfile


FROM python:3

 

ENV PYBASE /pybase

ENV PYTHONUSERBASE $PYBASE

ENV PATH $PYBASE/bin:$PATH

RUN pip install pipenv

WORKDIR /tmp

COPY Pipfile .

RUN pipenv lock

RUN PIP_USER=1 PIP_IGNORE_INSTALLED=1 pipenv install -d --system --ignore-pipfile

COPY . /app/notes

 

WORKDIR /app/notes

EXPOSE 80

CMD ["flask", "run", "--port=80", "--host=0.0.0.0"]

Build and Test

docker build -t notesapp:0.1 .

docker run --rm -it --network notes -v /home/Osama/notes/migrations:/app/notes/migrations notesapp:0.1 bash

The above commands build and run the container, once you are inside the container configure the database

Configure Database

flask db init

flask db migrate

flask db upgrade

Run and Upgrade

docker run --rm -it --network notes -p 80:80 notesapp:0.1

In a web browser, navigate to the public IP address for the server, and log in to your account.
Verify that you can create a new note.

Perfect , we are done now

Enjoy the learning 👍

Osama

How to enable docker logging

Posted on July 4, 2021 by Osama Mustafa in DevOps Tools

Docker includes multiple logging mechanisms to help you get information from running containers and services. These mechanisms are called logging drivers. Each Docker daemon has a default logging driver, which each container uses unless you configure it to use a different logging driver, or “log-driver” for short.

STEPS :-

Configure Docker to user Syslog

vim /etc/rsyslog.conf

In the file editor, uncomment the two lines under `Provides UDP syslog reception` by removing `#`.

#ModLoad imudp

#UDPServerRun 514

Then

systemctl start rsyslog

Now that syslog is running, let’s configure Docker to use syslog as the default logging driver. We’ll do this by creating a file called daemon.json

sudo mkdir /etc/docker

vi /etc/docker/daemon.json

{ "log-driver":

"syslog",

"log-opts": {

"syslog-address": "udp://<PRIVATE_IP>:514" }

}

Then

systemctl start docker

Time to use for docker

For example , first method

docker container run -d --name syslog-logging httpd

Check by

docker logs syslog-logging

tail /var/log/messages

second way to use the enable logging

docker container run -d --name json-logging --log-driver json-file httpd

Check

docker logs json-logging

Docker power 👌

Enjoy

Osama