Cloud – Page 3 – Technology Geek

Implementing Data Replication and Disaster Recovery with OCI Autonomous Database

Posted on July 25, 2024 by Osama Mustafa in Uncategorized

Introduction

Overview of OCI Autonomous Database and its capabilities.
Importance of data replication and disaster recovery for business continuity.

Step-by-Step Guide

Setting Up OCI Autonomous Database

Creating an Autonomous Database Instance:

oci db autonomous-database create --compartment-id <compartment_OCID> --db-name "MyDatabase" --cpu-core-count 1 --data-storage-size-in-tbs 1 --admin-password "<password>" --display-name "MyAutonomousDB" --db-workload "OLTP" --license-model "BRING_YOUR_OWN_LICENSE" --wait-for-state AVAILABLE

2. Configuring Data Replication

Creating a Database Backup:

oci db autonomous-database backup create --autonomous-database-id <db_OCID> --display-name "MyBackup" --wait-for-state COMPLETED

3. Setting Up Data Guard for High Availability:

Creating a Data Guard Association:

oci db autonomous-database create-data-guard-association --compartment-id <compartment_OCID> --primary-database-id <primary_db_OCID> --standby-database-id <standby_db_OCID> --display-name "MyDataGuardAssociation"

4. Implementing Disaster Recovery

Configuring Backup Retention Policies:
Set up automated backups with a specific retention period through the OCI Console or CLI:

oci db autonomous-database update --autonomous-database-id <db_OCID> --backup-retention-period 30

Restoring a Database from Backup:

oci db autonomous-database restore --autonomous-database-id <db_OCID> --restore-timestamp "2024-01-01T00:00:00Z" --display-name "RestoredDatabase"

4. Testing and Validating Disaster Recovery

Performing a Failover Test:
- Failover to Standby Database:

oci db autonomous-database failover --autonomous-database-id <standby_db_OCID>

Verifying Data Integrity:
- Connect to the standby database and validate data consistency and application functionality.

5. Automating and Monitoring

Automating Backups and Replication:
- Use OCI’s built-in scheduling features to automate backup creation and data replication.
Monitoring Database Health and Performance:
Use OCI Monitoring to set up alarms and dashboards for tracking the health and performance of your Autonomous Database.
Example Alarm:

oci monitoring alarm create --compartment-id <compartment_OCID> --display-name "HighIOWaitTime" --metric-name "io_wait_time" --threshold 1000 --comparison ">" --enabled true

Building a Secure Data Pipeline with OCI Data Flow and OCI Data Integration

Posted on July 21, 2024 by Osama Mustafa in Cloud, OCI

Setting Up OCI Data Flow

Creating a Data Flow Application:

oci data-flow application create --compartment-id <compartment_OCID> --display-name "MyDataFlowApp" --image-id <image_OCID> --description "Data processing application"

Creating a Data Flow Run:

oci data-flow run create --application-id <application_OCID> --display-name "MyDataFlowRun" --compartment-id <compartment_OCID> --arguments '{"input":"<input_data_location>", "output":"<output_data_location>"}' --wait-for-state SUCCEEDED

Setting Up OCI Data Integration

Creating a Data Integration Task:
- Go to Data Integration → Data Tasks → Create Task.
- Define your task type (e.g., Copy Data, Data Mapping) and configure source and target data stores.
Setting Up Data Flows:
Define and configure data flows that transform and move data between different sources and targets.
Example: Copy data from an OCI Object Storage bucket to a database
Securing Your Data Pipeline
Data Encryption:
- At Rest: Ensure data stored in OCI Object Storage is encrypted using server-side encryption.
- In Transit: Use HTTPS for secure data transfers between services.
Access Control:
- Configure IAM policies to restrict access to data sources and pipelines.
- Example IAM Policy:

allow group <group_name> to manage data-integrations in compartment <compartment_OCID>

Network Security:

Use VCNs and subnets to isolate data processing environments.
Example: Set up a private endpoint for data flow applications.

Monitoring and Managing Data Pipelines

Monitoring Data Flow Runs:

oci data-flow run list --compartment-id <compartment_OCID> --application-id <application_OCID>

Setting Up Alarms:

Use OCI Monitoring to create alarms based on metrics from data flows and integration tasks.

Example Alarm:

oci monitoring alarm create --compartment-id <compartment_OCID> --display-name "HighErrorRate" --metric-compartment-id <compartment_OCID> --metric-name "error_rate" --threshold 5 --comparison "<" --enabled true

putting in place a safe data pipeline that uses OCI Data Integration to import log data into an OCI Autonomous Database, OCI Data Flow to process the log data, and OCI Object Storage bucket to modify it. To protect the security and integrity of the data, the pipeline has access controls, encryption, and monitoring.

Thank you
Osama

How to setup the OCI CLI

Posted on July 13, 2024July 14, 2024 by Osama Mustafa in Cloud

Setting up the OCI CLI (Command Line Interface) involves several steps to authenticate, configure, and start using it effectively. Here’s a detailed guide to help you set up OCI CLI.

Step 1: Prerequisites

OCI Account: Ensure you have an Oracle Cloud Infrastructure account.
Access: Make sure you have appropriate permissions to create and manage resources.
Operating System: OCI CLI supports Windows, macOS, and Linux distributions.

Step 2: Install OCI CLI

Install Python: OCI CLI requires Python 3.5 or later. Install Python if it’s not already installed:

On Linux:

sudo apt update
sudo apt install python3

On macOS:
Install via Homebrew:

brew install python3

On Windows: Download and install Python from python.org.

Install OCI CLI: Use pip, Python’s package installer, to install OCI CLI:

pip3 install oci-cli

Step 3: Configure OCI CLI

Generate API Signing Keys: OCI CLI uses API signing keys for authentication. If you haven’t created keys yet, generate them through the OCI Console:
- Go to Identity → Users.
- Select your user.
- Under Resources, click on API Keys.
- Generate a new key pair if none exists.

Configure OCI CLI: After installing OCI CLI, configure it with your tenancy, user details, and API key:

Open a terminal or command prompt.
Run the following command:

oci setup config

Enter a location for your config file: Choose a path where OCI CLI configuration will be stored (default is ~/.oci/config).
Enter a user OCID: Enter your user OCID (Oracle Cloud Identifier).
Enter a tenancy OCID: Enter your tenancy OCID.
Enter a region name: Choose the OCI region where your resources are located (e.g., us-ashburn-1).
Do you want to generate a new API Signing RSA key pair?: If you haven’t generated API keys, choose yes and follow the prompts.

Once configured, OCI CLI will create a configuration file (config) and a key file (oci_api_key.pem) in the specified location.

Thank you

Osama

Exploring Oracle Cloud Infrastructure (OCI)

Posted on July 13, 2024July 14, 2024 by Osama Mustafa in Cloud

In today’s rapidly evolving digital landscape, choosing the right cloud infrastructure is crucial for organizations aiming to scale, secure, and innovate efficiently. Oracle Cloud Infrastructure (OCI) stands out as a robust platform offering a comprehensive suite of cloud services tailored for enterprise-grade performance and reliability.

1. Overview of OCI: Oracle Cloud Infrastructure (OCI) provides a highly scalable and secure cloud computing platform designed to meet the needs of both traditional enterprise workloads and modern cloud-native applications. Key components include:

Compute Services: OCI offers Virtual Machines (VMs) for general-purpose and high-performance computing, Bare Metal instances for demanding workloads, and Container Engine for Kubernetes clusters.
Storage Solutions: Includes Block Volumes for persistent storage, Object Storage for scalable and durable data storage, and File Storage for file-based workloads.
Networking Capabilities: Virtual Cloud Network (VCN) enables customizable network topologies with VPN and FastConnect for secure and high-bandwidth connectivity. Load Balancer distributes incoming traffic across multiple instances.
Database Options: Features Autonomous Database for self-driving, self-securing, and self-repairing databases, MySQL Database Service for fully managed MySQL databases, and Exadata Cloud Service for high-performance databases.

Example 2: Implementing Autonomous Database

Autonomous Database handles routine tasks like patching, backups, and updates automatically, allowing the IT team to focus on enhancing customer experiences.

Security and Compliance: OCI provides robust security features such as Identity and Access Management (IAM) for centralized control over access policies, Security Zones for isolating critical workloads, and Web Application Firewall (WAF) for protecting web applications from threats.

Management and Monitoring: OCI’s Management Tools offer comprehensive monitoring, logging, and resource management capabilities. With tools like Oracle Cloud Infrastructure Monitoring and Logging, organizations gain insights into performance metrics and operational logs, ensuring proactive management and troubleshooting.

Integration and Developer Tools: For seamless integration, OCI offers Oracle Integration Cloud and API Gateway, enabling organizations to connect applications and services securely across different environments. Developer Tools like Oracle Cloud Developer Tools and SDKs support agile development and deployment practices.

Oracle Cloud Infrastructure (OCI) emerges as a robust solution for enterprises seeking a secure, scalable, and high-performance cloud platform. Whether it’s deploying mission-critical applications, managing large-scale databases, or ensuring compliance and security, OCI offers the tools and capabilities to drive innovation and business growth.

AWS Infrastructure

Posted on February 24, 2023February 24, 2023 by Osama Mustafa in AWS, Cloud

The AWS Global Cloud Infrastructure is the most secure, extensive, and reliable cloud platform, offering over 200 fully featured services from data centers globally.

AWS Data Center

AWS pioneered cloud computing in 2006 to provide rapid and secure infrastructure. AWS continuously innovates on the design and systems of data centers to protect them from man-made and natural risks. Today, AWS provides data centers at a large, global scale. AWS implements controls, builds automated systems, and conducts third-party audits to confirm security and compliance. As a result, the most highly-regulated organizations in the world trust AWS every day.

Availability Zone – AZ

An Availability Zone (AZ) is one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region. Availability Zones are multiple, isolated areas within a particular geographic location. When you launch an instance, you can select an Availability Zone or let AWS choose one for you. If you distribute your instances across multiple Availability Zones and one instance fails, you can design your application so that an instance in another Availability Zone can handle requests.

Region

Each AWS Region consists of multiple, isolated, and physically separate Availability Zones within a geographic area. This achieves the greatest possible fault tolerance and stability. In your account, you determine which Regions you need. You can run applications and workloads from a Region to reduce latency to end users. You can do this while avoiding the upfront expenses, long-term commitments, and scaling challenges associated with maintaining and operating a global infrastructure.

AWS Local Zone

AWS Local Zones can be used for highly demanding applications that require single-digit millisecond latency to end users. Media and entertainment content creation, real-time multiplayer gaming, and Machine learning hosting and training are some use cases for AWS Local Zones.

CloudFront – Edge Location

An edge location is the nearest point to a requester of an AWS service. Edge locations are located in major cities around the world. They receive requests and cache copies of your content for faster delivery.

Regards

Osama

Another year – Top 100 Oracle Blogs and Website

Posted on November 13, 2021November 13, 2021 by Osama Mustafa in News

The best Oracle blogs from thousands of blogs on the web ranked by traffic, social media followers, domain authority & freshness.

Happy to share that my blog has been choosen for another year as the Top 100 Blogs around the world, the list contains talened, experience and professional people 🎉🎉🎉

Thank you all for the support.

Cheers

Osama

New Cloud Project

Posted on November 12, 2021 by Osama Mustafa in DevOps Tools

The Idea of this project the following :

You need to develop and deploy a python app that writes a new file to S3 on every execution. These files need to be maintained only for 24h.

The content of the file is not important, but add the date and time as prefix for you files name.

The name of the buckets should be the following ones for QA and Staging respectively:

qa-FIRSTNAME-LASTNAME-platform-challenge

staging-FIRSTNAME-LASTNAME-platform-challenge

The app will be running as a docker container in a Kubernetes cluster every 5 minutes. There is a Namespace for QA and a different Namespace for Staging in the cluster. You don’t need to provide tests but you need to be sure the app will work.

Github HERE

Cheers

Osama

DOAG 2021 Conference

Posted on November 5, 2021August 13, 2022 by Osama Mustafa in News

One of the bigges conference in europe, if you want to learn something new, DON’T MISS THE CHANCE

The Agenda is here

See you there

Cheers

Osama

APAC Groundbreakers Virtual Tour 2021

Posted on November 5, 2021August 13, 2022 by Osama Mustafa in News

I will have two presentation about the DevOps

Database Automation, Is this even possible ?
Kuberenetes in Depth but in simple way

You can register here

The hashtag in use is #APACGBT2021

Enjoy

Cheers

Introduction

Step-by-Step Guide

Step 1: Prerequisites

Step 3: Configure OCI CLI

Where to start?

Social media: