BLOG

RMOUG 2021

Happy New Year everyone and wish you all the best during this year.

I got accepted to speak at RMOUG Training Day 2021, the conference will be virtual so you can attend from your own home and learn something new.

I will speaking about the chaging Role of the DBA on Thurs Feb 11, 9:30 10:25 ( Eastern Standard Time) , you can register HERE.

And you can find RMOUG Agenda HERE.

See you soon

Osama

Docker & kubernetes example – Full project for free

Okay, I love to post free examples/projects on my github from while to while, i choose docker and kubernetes this time, the project idea it’s very nice and easy to implement.

What this project do ?

This can be a simple web app that reads a ‘hello world’ string from the MySQL database.Run a database app. Data volume should be persistent.Application from step 1 needs to discover the database from step 2 using Kubernetes native features.
Database credentials should NOT be hardcoded in application or helm chart code.The application should be accessible from the outside of Kubernetes.Create a helm chart which implements all these steps

  • Create an application that connects to a database, reads some data, and returns this data upon HTTP request, This can be a simple web app that reads a ‘hello world’ string from the MySQL database.
  • Run a database app. Data volume should be persistent.
  • Application from step 1 needs to discover the database from step 2 using Kubernetes native features, Database credentials should NOT be hardcoded in application or helm chart code.
  • The application should be accessible from the outside of Kubernetes.
  • Create a helm chart which implements all these steps

I Choose to use Java as programing language because the springboot framework it’s already defined and easy to use.

Please follow the readme file and everything will working fine without any issue, if you have any question comment below and i will answer

GitHub Link HERE

Enjoy the free learning

Osama

AUSOUG Virtual Connect 2020

I posted about this earlier , since AUSOUG had Virtual conference with different topics covering Cloud and databases.

my presentation was on Thursday about Oracle on azure. it was great pleasure to present again, the virtual connect had very big names in Oracle commuinty and had pleasure to present with them.

The Link for recorded session HERE

I recorded the session and it’s avaliable on my youtube channel HERE.

Enjoy learning

Osama

Alibaba Cloud MVP

Alibaba Cloud Most Valuable Professional (MVP) is an award for thought leaders. who are devoted to helping others fully understand and use Alibaba Cloud technologies.

Benefits

  • MVP Medal
  • Exclusive Services
  • Gain Exposure through Alibaba Cloud Official Channels
  • Two Tickets to the Computing Conference
  • Invitation to MVP Global Annual Summit
  • Visit to Alibaba Headquarters
  • MVP Lab
  • MVP Custom Gift Packs
  • Global Technical Study Tour
  • Alibaba Cloud High End Closed Conferences
  • High Level Resources
  • More benefits will be available soon.

MY PROFILE HERE

To be Officially the first and the only one so far that organized as Alibaba Cloud MVP in the middle east, it’s great pleasure.

I would like to thank alibaba for this opportunity and looking forward to serve the commuinty and make it better for all of us.

Alibaba Cloud Global AI Innovation Challenge with $116000 in prizes HERE.

Retail’s High-Tech Renaissance online Conference HERE

Public Courses on Cloud Native Technologies HERE

Regards

Osama

Next Event- Speaking Schedule/2020

I was honored this year to speak in different conferences around the world, and Due to Covid-19 most of the conference was Virtual, I miss my friends around the world, but saftey comes first, i hope everyone is safe.

I am writing this post to share with people the schedule of different conferences some of them already done but in case you missed something there is always next time, register in the virtual conferences and don’t miss the chance to learn something new.

My 2020 Schedule

Rule #1 in IT, never, and I do mean NEVER, stop learning.

Conference NameGroupDateLinkNote
Oracle Groundbreakers EMEA Virtual TourKSAOUG10-Oct-2020 1 2:30pmhttp://ogbemea.com/10-oct-2020/
SPOUG20SPOUGOctober 20, 2020 10:50 amhttp://spoug.es/spoug-20/
Oracle Groundbreakers APAC Virtual Tour 2020APAC Group19 October 2020, 10:00 amhttps://www.apacgbt.com/

This table will be updated depends on the schedule or any new conferences.

Regards

Osama

configure PMM for Percona MYSQL

Percona Monitoring and Management (PMM) is a best-of-breed open source database monitoring solution. It helps you reduce complexity, optimize performance, and improve the security of your business-critical database environments, no matter where they are located or deployed.

PMM is a free and open-source solution that you can run in your own environment for maximum security and reliability. It provides thorough time-based analysis for MySQL and MongoDB servers to ensure that your data works as efficiently as possible.

PMM, at a high-level, is made up of two basic components: the client and the server. The PMM Client is installed on the database servers themselves and is used to collect metrics. The client contains technology specific exporters (which collect and export data), and an “admin interface” (which makes the management of the PMM platform very simple). The PMM server is a “pre-integrated unit” (Docker, VM or AWS AMI) that contains four components that gather the metrics from the exporters on the PMM client(s). The PMM server contains Consul, Grafana, Prometheus and a Query Analytics Engine that Percona has developed. Here is a graphic from the architecture section of our documentation. In order to keep this post to a manageable length.

In this post i will setup the PMM on docker.

Pulling the PMM Server Docker Image

docker pull percona/pmm-server:2

Create a persistent data container.

docker create --volume /srv \
--name pmm-data percona/pmm-server:2 /bin/true

Run the image to start PMM Server.

docker run --detach --restart always \ --publish 80:80 --publish 443:443 \ --volumes-from pmm-data --name pmm-server \ percona/pmm-server:2

Once you completed the server configuration, you have to install the client on desired one, for example in our case we want to install it to monitor MySQL so we will go to install PMM Client on MySQL server.

To install the PMM client package, follow these steps.

Configure Percona repositories using the percona-release tool

wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb

Note

If you have previously enabled the experimental or testing Percona repository, don’t forget to disable them and enable the release component of the original repository as follows:

sudo percona-release disable all
sudo percona-release enable original release

Install the PMM client package:

sudo apt-get update
sudo apt-get install pmm2-client

Register your Node:

Before doing this, there are some MySQL requirements should be done from Database side.

Enable Logs in MySQL follow these step

SET GLOBAL slow_query_log_file = '/path/to/slow_query.log';

Determine what makes a query “slow”, by setting the limit (in seconds) after which a query is logged to the slow query log. The example below logs every query that exceeds 10 seconds in duration

mysql> SET GLOBAL long_query_time = 10;

Now enable the Slow Query log.

mysql> SET GLOBAL slow_query_log = 'ON';
mysql> FLUSH LOGS;

If you want to make these changes persistent, modify the my.cnf and add these lines to the [mysqld] part of the config.

[mysqld]
...
slow_query_log = /path/to/slow_query.log
long_query_time = 10
log_queries_not_using_indexes = ON

Verify

mysql> SHOW GLOBAL VARIABLES LIKE 'log_queries_not_using_indexes';

Once you are done you have to create username/password for PMM,this user should have necessary privileges for collecting data. If the pmm user already exists, you can grant the required privileges as follows:

CREATE USER 'pmm'@'localhost' IDENTIFIED BY 'pass' WITH MAX_USER_CONNECTIONS 10;

GRANT SELECT, PROCESS, SUPER, REPLICATION CLIENT, RELOAD ON *.* TO 'pmm'@'localhost';

Once you done, register your node.

pmm-admin config --server-insecure-tls --server-url=https://admin:admin@<IP Address>:443

You have to wait for couple of minutes till it will be sync.

You should see the following output:

Checking local pmm-agent status...
pmm-agent is running.
Registering pmm-agent on PMM Server...
Registered.
Configuration file /usr/local/percona/pmm-agent.yaml updated.
Reloading pmm-agent configuration...
Configuration reloaded.

Regards

Osama

Setting up a Jenkins-Based Continuous Delivery Pipeline with Docker

As an important step in agile development, continuous integration is designed to maintain high quality while accelerating product iteration. Every time when the codes are updated, an automatic test is performed to test the codes and function validity. The codes can only be delivered and deployed after they pass the automatic test, This post describes how to combine Jenkins, one of the most popular integration tools, with Alibaba Cloud Container Service to realize automatic test and image building pushing.

1

Deploying Jenkins Applications and the Slave Nodes

1. Create a Jenkins orchestration template.

Create a new template and create the orchestration based on the following content.

jenkins:  image: 'registry.aliyuncs.com/acs-sample/jenkins:latest'  ports:      - '8080:8080'      - '50000:50000'  volumes:      - /var/lib/docker/jenkins:/var/jenkins_home  privileged: true  restart: always   labels:      aliyun.scale: '1'      aliyun.probe.url: 'tcp://container:8080'      aliyun.probe.initial_delay_seconds: '10'      aliyun.routing.port_8080: jenkins  links:      - slave-nodejs slave-nodejs:  image: 'registry.aliyuncs.com/acs-sample/jenkins-slave-dind-nodejs'  restart: always   volumes:      - /var/run/docker.sock:/var/run/docker.sock  labels:      aliyun.scale: '1' 

2. Use the template to create Jenkins applications and slave nodes.

You can also directly use a Jenkins sample template provided by Alibaba Cloud Container Service to create Jenkins applications and slave nodes.

2

3. After the successful creation, Jenkins applications and slave nodes will be displayed in the service list.

3

4. After opening the access endpoint provided by the Container Service, you can use the Jenkins application deployed just now.

4

Realizing Automatic Test and Automatic Build and Push of Image

Configure the slave container as the slave node of the Jenkins application.

Open the Jenkins application and enter the System Settings interface. Select Manage Node > Create Node, and configure corresponding parameters. See the figure below.

5

Note: Label is the only identifier of the slave. The slave container and Jenkins container run on the Alibaba Cloud platform at the same time. Therefore, you can fill in a container node IP address that is inaccessible to the Internet to isolate the test environment.

6

Use the jenkins account and password (the initial password is jenkins) in Dockerfile for the creation of the slave-nodejs image when adding Credential. Image Dockerfile address HERE

1. Create a project to implement the automatic test.

  1. Create an item and choose to build a software project of free style.
  2. Enter the project name and select a node for running the project. In this example, enter the slave-nodejs-ut node created above.
7

Configure the source code management and code branch. In this example, use GitHub to manage source codes.

8

Configure the trigger for building. In this example, automatically trigger project execution by combining GitHub Webhooks and services.

9

Add the Jenkins service hook to GitHub to implement automatic triggering.

Click the Settings tab on the Github project homepage, and click Webhooks & services > Add service and select Jenkins (Git plugin). Enter ${Jenkins IP}/github-webhook/ in the Jenkins hook URL dialog box.

1. http://jenkins.cd****************.cn-beijing.alicontainer.com/github-webhook/
10

Add a build step of Executes shell type and write shell scripts to execute the test.

11

The command in this example is as follows.

1. pwd
2. ls
3. cd chapter2
4. npm test

Create a project to automatically build and push images.

  1. Create an item and choose to build a software project of free style.
  2. Enter the project name and select a node for running the project. In this example, enter the slave-nodejs-ut node created above.
  3. Configure the source code management and code branch. In this example, use GitHub to manage source codes.
  4. Add the following trigger and set it to implement automatic image building only after success of the unit test.
12

Write shell scripts for building and pushing images.

13

The command in this example is as follows.

a.cd chapter2 b.docker build -t registry.aliyuncs.com/qinyujia-test/nodejs-demo . c.docker login -u ${yourAccount} -p ${yourPassword} registry.aliyuncs.com d.docker push registry.aliyuncs.com/qinyujia-test/nodejs-demo 

Automatically Redeploy the Application

Deploy the application for the first time

Use the orchestration template to deploy the image created above to the Container Service and create the nodejs-demo application.

Example

1. 
2. express:
3. image: 'registry.aliyuncs.com/qinyujia-test/nodejs-demo'
4. expose:
5. - '22'
6. - '3000'
7. restart: always
8. labels:
9. aliyun.routing.port_3000: express
10. 

1. Select the application nodejs-demo just created, and create the trigger.

14

 Add a line to the shell scripts you wrote in Realize automatic test and automatic build and push of image. The address is the trigger link given by the trigger created above.

i.curl 'https://cs.console.aliyun.com/hook/trigger?triggerUrl=***==&secret=***' 

Change the Command in the example from Realize automatic test and automatic build and push of image as follows.

i. cd chapter2
ii. docker build -t registry.aliyuncs.com/qinyujia-test/nodejs-demo .
iii. docker login -u ${yourAccount} -p ${yourPassword} registry.aliyuncs.com iv.docker push registry.aliyuncs.com/qinyujia-test/nodejs-demo
v. curl 'https://cs.console.aliyun.com/hook/trigger?triggerUrl=***==&secret=***'

After pushing the image, Jenkins automatically triggers redeployment of the nodejs-demo application.

Configure The Email Notification for the Results

If you want to send the unit test or image configuration results to relevant developers or project execution initiators through email, perform the following configurations.

On the Jenkins homepage, click System Management > System Settings, and configure a Jenkins system administrator email.

15

Install the Extended Email Notification plugin, configure SMTP server and other relevant information, and set the default recipient list. See the figure below.

16

The above example shows the parameter settings of the Jenkins application system. The following example shows the relevant configurations for Jenkins projects whose results are to be pushed through email.

1. Add post-building operation steps in the Jenkins project, select Editable Email Notification, and enter a recipient list.

17

2. Add a mailing trigger.

18

Cheers

Osama

Set Up Your First Server on Alibaba Cloud

I posted before about AWS, Azure and OCI but this time i am trying to cover Alibaba , in this post i will show you how to setup your first Compute instance whatever OS you want CentOS or Ubuntu depeds on what you want

What you need ?

  • A valid Alibaba Cloud account., create free one with 300$ credit
  • An ECS instance running CentOS 7 or whatever OS you want.
  • A root password for your server.

Connect to your Alibaba Cloud

Locate the Internet IP address (Public IP address) associated with your Alibaba Cloud ECS Instance.

If you are running Linux or Mac, use a terminal application to connect to the instance via SSH. If you are on Windows, you can use PuTTy.

Change the Hostname

The hostname is a default identifier when you communicate to a Linux server. It is like a computer name that is associated with your home PC or laptop. Naming your CentOS 7 server with a descriptive hostname helps you to differentiate your machines especially if you are running a bunch of them.

check the update

$ sudo yum update

check your hostname

$ hostname

change your hostname, we need to install nano text editor using the command below:

$ sudo yum install nano

Then, edit the /etc/cloud/cloud.cfg file and find the entry preserve_hostname. Change its value from false to true.

$ sudo nano /etc/cloud/cloud.cfg
preserve_hostname true

Then, edit the /etc/hostname file using a nano editor by typing the command below:

$ sudo nano /etc/hostname

You will need to add two entries on this file just below the 127.0.0.1 localhost entry. The first entry you are adding uses the loopback interface address 127.0.1.1. Please note that this is different from the address 127.0.0.1 which have a ‘localhost’ value in the same file.

Reboot your Alibaba Cloud ECS instance for the changes to take effect by typing the command below:

$ sudo reboot

Also you can create non root user by

$ sudo adduser

For instance, to add a user identified as osama on your server, use the command below:

$ sudo adduser osama

Next, we assign a password to the user we have created above:

$ sudo passwd osama

now assign the administrative task to this user

$ sudo gpasswd -a osama wheel

Cheers

Osama