DevOps Overview, Cloud Version this time Azure

DevOps is the union of people, process, and products to enable continuous delivery of value to your end users. Azure DevOps is a set of services that gives you the tools you need to do just that. With Azure DevOps, you can build, test, and deploy any application, either to the cloud or on premises. DevOps practices that enable transparency, cooperation, continuous delivery and continuous deployment become embedded in your software development lifecycle.

Azure DevOps provides several tools you can use for better team collaboration. It also has tools for automated build processes, testing, version control, and package management. That’s quite a bit to cover! We’ll get to all the tools eventually. For now, let’s follow the team as they begin with an overview of what Azure DevOps is and how they can get started.

Azure DevOps ServicesDescriptions
Azure Boardsagile tools that help us plan, track, and discuss our work, even with other teams.
Azure Pipelines build, test, and deploy with CI/CD that works with any language, platform, and cloud.
Azure Test Plansmanual and exploratory testing tools.
Azure Reposprovide unlimited, cloud-hosted private, and public Git repos.
Azure Artifacts create, host, and share packages.

What is Agile?

Agile is a term that’s used to describe approaches to software development, emphasizing incremental delivery, team collaboration, continual planning, and continual learning. Agile isn’t a process as much as it is a philosophy or mindset for planning the work that a team will do. It’s based on iterative development and helps a team better plan for and react to the inevitable changes that occur in software development. Let’s listen in on Mara’s discussion with Andy after the latest release.

Recommendations for adopting Agile

  • Create an organizational structure that supports Agile practices
  • Mentor team members on Agile techniques and practices
  • Enable in-team and cross-team collaboration:- If collaboration is the key to becoming successful at Agile, what are some of the ways you can encourage it? Here are some ideas.

What is Azure Boards?

Azure Boards is a tool in Azure DevOps to help teams plan the work that needs to be done. The Tailspin team will use this tool to get a better idea of what work needs to be done and how to prioritize it.

Set up Azure Boards using the Basic process

  • Create the project
  1. Sign into your account at dev.azure.com.
  2. Select + Create project.
  3. In the Project name field, type Space Game – web.
  4. In the Description field, type The Space Game website.
  5. Under Visibility, you choose whether to make your project public or private. For now, you can choose private.
  6. Select Advanced.
  7. Under Version control, make sure that Git is selected. Under Work item process, make sure that Basic is selected.
  8. Select Create.
  • Create a team
  1. Select Project settings in the lower-left corner.
  2. On the Project details page, under General, select Teams.
  3. Select Space Game – web Team.
  • Add team members
  1. Under Members, select + Add.
  2. Enter the email address of the user you’d like to add. Then select Save changes.
  3. Repeat the process for any other members you’d like to add.
  • Create the board
  1. In the column on the left, point to Boards and select Boards from the menu that appears.
  2. Select Space Game – web Team boards. A blank board appears.
  3. In the To Do column, select the green + button next to the New item field.
  4. Enter Stabilize the build server and then press Enter.
  5. Select the ellipsis (), and then select Open.
  6. In the Description field, enter this text (The build server keeps falling over. The OS, Ubuntu 16.04, requires security patches and updates. It’s also a challenge to keep build tools and other software up to date.)
  7. Select Save & Close.
  8. Follow the same steps for the next two items.
TitleDescription
Create a Git-based workflowMigrate source code to GitHub and define how we’ll collaborate.
Create unit testsAdd unit tests to the project to help minimize regression bugs.

Drag Stabilize the build server to the top of the stack. Then, drag Create a Git-based workflow to the second item position. Your final board looks like this.

  • Define a sprint
  1. In the left-side column, select Sprints.
  2. Select Set dates from the upper right.
  3. Leave the name as Sprint 1.
  4. In the Start date field, select the calendar and pick today’s date.
  5. In the End date field, select the calendar and pick the date two weeks from today.
  6. Select Save and Close.

Assign tasks and set the iteration

  1. Under Boards, select Work items.
  2. Select Stabilize the build server.
  3. In the Iteration drop-down list, select Sprint 1.
  4. From the same window, select Unassigned and set yourself as the task owner.
  5. Repeat the process for the other two work items.
    1. Create a Git-based workflow
    2. Create unit tests

Cheers
Osama

Cloud Services Mapping For AWS, Azure, GCP ,OCI, IBM and Alibaba provider

This blog post is one of that kind that took much time and consume so much energy, to complete this post it took me around ten days to make sure that I will cover most of the available services and make it readable for people, Be sure the services can change while you are reading this post ; if you have any comments,or add something to this post, please send me an email – using contact us page or by comments below.

I am writing this post to share a different cloud providers services and the comparison between each one of them, this will show various naming services for each one of them.

Earlier we used to store our data to H.D.D or USB flash, Cloud Computing services have replaced such hard drive technology. Cloud Computing service is nothing but providing services like Storage, Databases, Servers, networking, and software through the Internet.

Cloud Computing is moving so fast, in 2020 the cloud now is more mature, going multi-cloud, and likely to become more focused on vertical and a sales ground war as the leading vendors battle for market share.

Notes :

  • GCP : Google Cloud Provider
  • OCI :- oracle cloud infrastructure
  • None : not meaning the services is not available necessarily by cloud provider but i didn’t look deeper into this or i didn’t use it before.

Marketplace

AWSAzureOCIGCPIBM CloudAliBaba Cloud
AWS MarketplaceAzure Marketplace Oracle Cloud MarketplaceGoogle Cloud Platform (GCP) MarketplaceIBM Marketplace Alibaba Cloud Marketplace 

AI and machine learning

AWSAzureOCIGCPIBM CloudAliBaba Cloud
SageMakerMachine LearningOCI Machine LearningGoogle Datalab Cloud AutoML (Alpha)
Cloud Machine Learning Services
Machine Learning Machine Learning Platform for AI 
Alexa Skills KitBot FrameworkOracle Digital AssistantGoogle AssistantNoneNone
Polly, TranscribeSpeech Services Bing Speech APINoneTranslation API
Speech API
NoneNone
LexSpeech ServicesOracle Chatbots Cloud Text-to-Speech DialogFlow Enterprise Edition (Beta) Natural Language APIWatson Assistant Intelligent Service Robot 
RekognitionCognitive ServicesNoneCloud Video Intelligence Vision APIVisual Recognition Image Search 
Skills KitVirtual AssistantNoneNoneNoneNone
Amazon ComprehendLanguage Understanding (LUIS)NoneCloud Text-to-Speech DialogFlow Enterprise Edition (Beta)
Natural Language API
Visual RecognitionImage Search

Big data and analytics

AWSAzureOCIGCPIBM CloudAliBaba Cloud
RedshiftSynapse Analyticsoracle autonomous data warehouseBigQueryDb2 Warehouse Alibaba MaxCompute ODPS 
Lake FormationData ShareNoneNoneNoneNone
Amazon EMRHDInsight Data Lake Storage
Oracle Big Data Service
Cloud DataProcAnalytics Engine E-MapReduce Service 

Data orchestration / ETL

AWSAzureOCIGCPIBM CloudAliBaba Cloud
Data Pipeline, GlueData Factory Data CatalogData IntegratorCloud DataPrep Cloud ComposerDataStage DataWorks  Data Integration 
Dynamo DBTable Storage, Cosmos DBNoSQL DatabaseCloud Datastore Cloud BigTableCloudant NoSQL DB  Compose for JanusGraph 
Databases for MongoDB 
Apsaradb for Mongodb  Table Store 

Analytics and visualization

AWSAzureOCIGCPIBM CloudAliBab Cloud
Kinesis AnalyticsStream Analytics
 Data Lake Analytics
Data Lake Store
Event Hub (Apache Kafka as a Service)Cloud DataflowStreaming AnalyticsNone
QuickSightPower BIData Visualization Business IntelligenceGoogle Data StudioWatson studio Data IDE 
CloudSearchCognitive SearchNoneNoneNoneNone
AthenaData Lake AnalyticsNoneBigQuerySQL QueryE-MapReduce Service

Compute

AWSAzureOCIGCPIBM CloudAliBaba Cloud
Elastic Compute Cloud (EC2) InstancesVirtual MachinesComputeCompute EngineClassic Virtual Server Alibaba ECS
BatchBatchNonePreemptible VMsIBM Cloud FunctionsBatch Compute 
Auto ScalingVirtual Machine Scale SetsAuto ScalingAuto ScalerAuto Scaling Auto Scaling 
VMware Cloud on AWSVMware by CloudSimpleNoneNoneNoneNone
Parallel ClusterCycleCloudCluster Networkingslurm gcpNoneNone
Amazon EC2 – I3.metalNoneCompute – Bare MetalNoneNoneNone
Amazon EC2 – P2, P3, G3 instancesAzure N-SeriesOracle Cloud Infrastructure Compute – GPUGoogle GPUNoneNone

Containers and container orchestrators

AWSAzureOCIGCPIBM CloudAlibaba Cloud
Elastic Container Service (ECS)Container InstancesOracle Cloud Infrastructure RegistryContainer RegistryIBM Cloud Container Registry Container Registry 
Elastic Kubernetes Service (EKS)Kubernetes Service (AKS)Container Engine for Kubernetes (OKE) Kubernetes Engine IBM Cloud Kubernetes Service Container Service for Kubernetes 
App MeshService Fabric MeshNoneGoogle Istio Service MeshNoneNone

Serverless

AWSAzureOCIGCPIBM CloudAlibaba Cloud
LambdaFunctionsOracle FunctionsGoogle Cloud FunctionsIBM Cloud Functions Function Compute 

Database

AreaAWSAzureOCIGCPIBM CloudAlibaba Cloud
Relational databaseRDSSQL Database Database for MySQL
Database for PostgreSQL
Oracle Database Cloud Service
MySQL Service 
Cloud SQL  Cloud Spanner Db2 
Db2 Hosted  Informix  Databases for PostgreSQL  Compose for MySQL(Beta
ApsaraDB for RDS MYSQL  ApsaraDB for RDS SQL Server  ApsaraDB for RDS PostgreSQL  Distributed Relational Database Service (DRDS) 
NoSQL / Document DynamoDB
SimpleDB
Amazon DocumentDB
Cosmos DBNoSQL Database Cloud Datastore  Cloud BigTable Cloudant NoSQL DB  Compose for JanusGraph  Databases for MongoDB ApsaraDB for RDS MYSQL  ApsaraDB for RDS SQL Server  ApsaraDB for RDS PostgreSQL  Distributed Relational Database Service (DRDS) 
CachingElastiCacheCache for RedisIn-memory OptionCloud MemoryStore (Beta) Informix HiTSDB (High-Performance Time Series Database) 
Database migration Database Migration ServiceDatabase Migration ServiceMigrate to the Cloud NoneLift CLI Data Transmission Service 
Relational Database Management ServiceAuroraAzure SQL Database; Azure Cosmos DB  Oracle Autonomous Transaction ProcessingCloud SQL ; Cloud SpannerCloudant NoSQL DB  Compose for JanusGraph  Databases for MongoDB Apsaradb for Mongodb  Table Store 

DevOps and application monitoring

AWSAzureOCIGCPIBM CloudAlibaba Cloud
CloudWatch,
X-Ray
AWS Cloud9  AWS Code Star 
AWS CodeBuild  CodeDeploy CodeCommit CodePipeline
Monitor Azure Boards  Azure Pipelines  Azure Repos  Azure Test Plans  Azure Artifacts  DevOpsDeveloper Cloud Service Cloud Source Repositories  Cloud Build Continuous Delivery  DevOps Insights  Globalization Pipeline None
Developer ToolsDeveloper ToolsDeveloper Cloud Service Cloud Source Repositories  Cloud Build Continuous Delivery  DevOps Insights  Globalization Pipeline None
Command Line InterfaceCLI PowerShellOCI CLICloud Shell  Cloud Console NoneAlibaba Cloud CLI 
OpsWorks (Chef-based)AutomationOracle Orchestration Cloud Service NoneNoneResource Orchestration Service 
CloudFormationResource Manager
VM extensions
Azure Automation Azure Building Blocks
Stack Manager Cloud Resource Manager  Cloud Deployment ManagerSchematics Resource Orchestration Service 

Internet of things (IoT)

AWS AzureOCIGCPIBM CloudAlibaba Cloud
IoTIoT HubInternet of Things Cloud Service Cloud IoT Core (Beta)  Google Cloud IoT Internet of Things Platform IoT Platform 
GreengrassIoT Edge Azure IoT SDKNoneNoneNoneNone
Kinesis Firehose
Kinesis Streams
Event Hubs Azure Stream Analytics Event Hub (Apache Kafka as a Service) Cloud Dataflow Streaming Analytics None
IoT Things GraphDigital TwinsNoneNoneDigital TranscodingApsaraVideo Live 
AWS IoT ButtonAzure SphereNoneNoneNoneNone

Management

AWSAzureOCIGCPIBM CloudAlibaba Cloud
Trusted AdvisorAdvisorNoneGoogle Cloud Platform Security NoneNone
Usage and Billing ReportBilling APIOracle Management ConsoleBilling API NoneAlibaba Cloud CLI 
Management ConsolePortalConsolePortal/ConsoleConsoleConsole
Application Discovery ServiceMigrate Azure Active Directory NoneNoneNoneNone
EC2 Systems ManagerMonitorOracle Management Cloud NoneNoneNone
Personal Health DashboardResource Health Azure Monitor Oracle Management Cloud NoneNoneCloud monitoring, Notification and Alerts 
CloudTrailMonitorApplication Performance Monitoring Google StackDriver  Monitoring  Logging Error Reporting  Trace  Debugger Application Performance Monitoring CloudMonitor 
Cost ExplorerCost ManagementOracle Management Cloud NoneNoneNone
CloudWatchApplication InsightsApplication Performance MonitoringGoogle StackDriver 
Monitoring 
Logging
Error Reporting 
Trace 
Debugger 
IBM Cloud Log Analysis with LogDNA CloudMonitor

Messaging and eventing

AWSAzureOCIGCPIBM CloudAlibaba Cloud
Simple Queue Service (SQS) Amazon MQ Queue Storage Service Bus Service Bus topics Service Bus relayIntegration Messaging Cloud Pub/SubEvent Steams Message Queue 
Simple Notification ServiceEvent Grid Azure Notification ServicesMessaging Firebase Cloud MessagingPush Notifications Short Message Service (SMS) 
Amazon SESMarketplace – EmailOracle Cloud Infrastructure Email DeliveryPartnersSendgrid Direct Mail 

Mobile services

AWSAzureOCIGCPIBM CloudAlibaba Cloud
Mobile HubApp Center Xamarin AppsMobile & ChatbotsCloud Mobile AppMobile FoundationNone
Mobile SDKApp Center
Azure Mobile SDK,Offline/Sync Azure DevTest Labs (Back End) Hockey App
Mobile Cloud ServiceCloud Tools for Android StudioNone
Amazon PinpointAzure Mobile EngagementNoneNoneMobile FoundationNone
CognitoApp CenterMobile Cloud Service Cloud Tools for Android StudioApp ID None
Mobile AnalyticsHockey App Mobile Cloud Service Firebase Analytics Mobile Foundation None

Networking

AreaAWSAzureOCIGCPIBM CloudAlibaba cloud
Cloud virtual networkingVirtual Private Cloud (VPC)Virtual NetworkOracle Virtual Cloud Network Virtual Private CloudIBM Cloud VPC on Classic Virtual Private Cloud 
Cross-premises connectivityAmazon VPN VPN Gateway VPN Connect Cloud VPN Classic IPSEC-VPN VPN Gateway 
DNS Managment Route 53DNSOracle DNS Cloud DNS Internet Services Alibaba Cloud DNS 
Global Traffic ManagmentAmazon Route 53 Traffic Flow Azure Traffic Manager OCI Traffic Management NoneInternet Services None
Dedicated networkDirect ConnectExpressRouteFast Connect Cloud InterConnect Direct Link Express Connect 
Load balancingElastic Load BalancingLoad BalancerOracle Load Balancer Cloud Load Balancing IBM Cloud Load Balancing Server Load Balancer 

Security, identity, and access

AWSAzureOCIGCPIBM CloudAlibaba Cloud
Identity and Access Management (IAM)Azure Active Directory
Role Based Access Control
Identity Cloud IAM Identity & Access Management Resource Access Management 
OrganizationsSubscription Management + RBAC
Policy
Management Groups
Audit NoneResource Group None
Multi-Factor AuthenticationMulti-Factor AuthenticationMulti-factor authenticationMulti-factor authenticationMulti-factor authenticationMulti-factor authentication
Directory ServiceAzure Active Directory Domain ServicesNoneNoneNoneNone
CognitoAzure Active Directory B2CMobile Cloud Service Firebase Authentication

App ID None

Encryption

AWSAzureOCIGCPIBM cloudAlibaba Cloud
Server-side encryption with Amazon S3 Key Management ServiceAzure Storage Service EncryptionNonenoneNoneNone
Key Management Service (KMS)
CloudHSM
Key VaultKey ManagementCloud Key Management Service Key Protect Key Management Service 

Firewall

AWSAzureOCIGCPIBM CloudAlibaba Cloud
Web Application FirewallWeb Application Firewall
Firewall
Web Application Firewall NoneInternet Services Web Application Firewall 

Security

AWSAzureOCIGCPIBM CloudAlibaba Cloud
InspectorSecurity CenterConfiguration and Compliance Service NoneInfrastructure Vulnerability Scan None
Certificate ManagerApp Service Certificates available on the PortalNoneCloud Key Management Service Key Protect Key Management Service 
GuardDutyAdvanced Threat ProtectionNoneNoneNoneNone
ArtifactService Trust PortalComplianceCloud Security Command Center (Alpha) NoneAlibaba Truster Center 
ShieldDDos Protection ServiceOracle Cloud Infrastructure DDoS ProtectionCloud Armor (Beta) Internet Services DDOS Pro and Basic 

Storage

AWSAzureOCIGCPIBM CloudAlibaba Cloud
Simple Storage Services (S3)Blob storageObject Storage Cloud Storage Cloud Object Storage Object Storage Service 
Elastic Block Store (EBS)managed disksBlock Storage Persistent Disk Block Storage Block Storage

Elastic File SystemFilesOCI File Storage File Store File Storage NAS File Storage 
S3 Infrequent Access (IA)Storage cool tierNoneNoneNoneNone
S3 GlacierStorage archive access tierArchive Storage Cloud Storage Object Storage-ColdVault Object Storage Archive 
BackupBackupNoneNoneNoneNone
Storage GatewayStorSimple
Storage Software Appliance NoneNoneHybrid Cloud Storage Array 
DataSyncFile SyncNoneNoneNoneNone

Bulk data transfer

AWS AzureOCIGCPIBM CloudAlibaba Cloud
Import/Export DiskImport/Export
Data Transfer Services – Hard Disk Import NoneData Transfer Service None
Import/Export c
Snowball Edge
Snowmobile
Data BoxData Transfer Services – Storage applicance import Transfer Appliance (Beta) Mass Data Migration Service Data Transport 

Web applications

AWSAzureOCIGCPIBM CloudAlibaba Cloud
Elastic BeanstalkApp ServiceApplication Container Cloud 
Java Cloud Service 
Google App engine Cloud Foundry Apps Enterprise Distributed Application Service 
API GatewayAPI ManagementAPI Platform Cloud EndpointsAPI Connect API Gateway 
CloudFrontContent Delivery NetworkNoneCloud CDN Content Delivery Network Alibaba Content Delivery Network 
Global AcceleratorFront DoorNoneNoneNoneNone
LightSailApp ServiceNoneNoneClassic Virtual Server 
Virtual Server for VPC 
Simple Application Server 

Miscellaneous

AreaAWSAzureOCIGCPIBM CloudAlibaba Cloud
Backend process logicStep FunctionsLogic AppsFunctions
App Engine IBM Cloud Functions None
Enterprise application servicesWorkMailWorkDocsOffice 365NoneG Suite
NoneNone
GamingGameLiftGameSparksPlayFabNoneNoneNoneNone
Media transcoding
Elastic TranscoderMedia ServicesNoneNoneDigital TranscodingApsaraVideo Live 
WorkflowSimple Workflow Service (SWF)Logic AppsData Integrator Cloud DataPrep (Private Beta) 
Cloud Composer (Beta) 
DataStage 
Watson Knowledge Catalog 
DataWorks
Data Integration  
HybridOutposts
Stack
Cloud At Customer Anthos NoneNone
MediaElemental MediaConvertMedia ServicesNoneNoneDigital Transcoding
ApsaraVideo Live 
Region Availability Zone (AZ)Availability Zone (AZ)Availability Domain (AD)Zonesavailability zonesZones

Disaster Recovery Services

AWSAzureOCIGCPIBM CloudAlibaba Cloud
AWS Disaster Recovery Azure Site Recovery Oracle Database Backup 
DR Site
NoneNoneAlibaba Disaster Recovery 
Hybrid Backup Recovery 

Enjoy

Cheers

Osama

OCI, AWS and Azure services in One pictures

Just quick post to show and share what services for each cloud provider, be notice that the services can be change while we are talking now, and this is not a complete list of services but it’s only shows the basic one.

OCI Services

Oracle Cloud Iinfrastructure Services

Amazon AWS

Amazon AWS services

Microsoft Azure

Cheers

Osama

Azure Resource quick guide

In gernal,

load balancer distributes traffic evenly among each system in a pool. A load balancer can help you achieve both high availability and resiliency.

Say you start by adding additional VMs, each configured identically, to each tier. The idea is to have additional systems ready, in case one goes down, or is serving too many users at the same time.

Azure Load Balancer is a load balancer service that Microsoft provides that helps take care of the maintenance for you. Load Balancer supports inbound and outbound scenarios, provides low latency and high throughput, and scales up to millions of flows for all Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) applications. You can use Load Balancer with incoming internet traffic, internal traffic across Azure services, port forwarding for specific traffic, or outbound connectivity for VMs in your virtual network.

When you manually configure typical load balancer software on a virtual machine, there’s a downside: you now have an additional system that you need to maintain. If your load balancer goes down or needs routine maintenance, you’re back to your original problem.

Azure Application Gateway

If all your traffic is HTTP, a potentially better option is to use Azure Application Gateway. Application Gateway is a load balancer designed for web applications. It uses Azure Load Balancer at the transport level (TCP) and applies sophisticated URL-based routing rules to support several advanced scenarios.

Benefits

  • Cookie affinity. Useful when you want to keep a user session on the same backend server.
  • SSL termination. Application Gateway can manage your SSL certificates and pass unencrypted traffic to the backend servers to avoid encryption/decryption overhead. It also supports full end-to-end encryption for applications that require that.
  • Web application firewall. Application gateway supports a sophisticated firewall (WAF) with detailed monitoring and logging to detect malicious attacks against your network infrastructure.
  • URL rule-based routes. Application Gateway allows you to route traffic based on URL patterns, source IP address and port to destination IP address and port. This is helpful when setting up a content delivery network.
  • Rewrite HTTP headers. You can add or remove information from the inbound and outbound HTTP headers of each request to enable important security scenarios, or scrub sensitive information such as server names.

What is a Content Delivery Network (CDN)?

A content delivery network (CDN) is a distributed network of servers that can efficiently deliver web content to users. It is a way to get content to users in their local region to minimize latency. CDN can be hosted in Azure or any other location. You can cache content at strategically placed physical nodes across the world and provide better performance to end users. Typical usage scenarios include web applications containing multimedia content, a product launch event in a particular region, or any event where you expect a high-bandwidth requirement in a region.

DNS

DNS, or Domain Name System, is a way to map user-friendly names to their IP addresses. You can think of DNS as the phonebook of the internet.

How can you make your site, which is located in the United States, load faster for users located in Europe or Asia?

network latency in azure

Latency refers to the time it takes for data to travel over the network. Latency is typically measured in milliseconds.

Compare latency to bandwidth. Bandwidth refers to the amount of data that can fit on the connection. Latency refers to the time it takes for that data to reach its destination.

One way to reduce latency is to provide exact copies of your service in more than one region, or Use Traffic Manager to route users to the closest endpoint, One answer is Azure Traffic Manager. Traffic Manager uses the DNS server that’s closest to the user to direct user traffic to a globally distributed endpoint, Traffic Manager doesn’t see the traffic that’s passed between the client and server. Rather, it directs the client web browser to a preferred endpoint. Traffic Manager can route traffic in a few different ways, such as to the endpoint with the lowest latency.

Cheers

Osama

Terraform for Oracle Cloud infrastructure

This post provide steps for downloading and installing both Terraform and the Oracle Cloud Infrastructure Terraform provider.

Terraform Overview

Terraform is “infrastructure-as-code” software that allows you to define your infrastructure resources in files that you can persist, version, and share. These files describe the steps required to provision your infrastructure and maintain its desired state; it then executes these steps and builds out the described infrastructure.

Infrastructure as Code is becoming very popular. It allows you to describe a complete blueprint of a datacentre using a high-level configuration syntax, that can be versioned and script-automated, Terraform can seamlessly work with major cloud vendors, including Oracle, AWS, MS Azure, Google, etc

Download and Install Terraform

In this section, i will show and explain how to download and install Terraform on  your laptop/PC Host Operating System, you can download using the below link :-

Terraform Download
  • After you download the terraform, Unzip the Terraform to whatever location you want to run it from. Then, add that location to your OS PATH.
    • Windows : By adding to Path –> environment variables
    • Linux : Profile –> export Path

You can check by run the CMD and check the version:-

Check Terraform commands

Download the OCI  Terraform Provider

Prerequisites:-

  • OCI User credentials that has  sufficient permission to  execute a Terraform plan.
  • Required keys and Oracle Cloud Infrastructure IDs (OCIDs).
  • The correct Terraform binary file for your operating system

Installing and Configuring the Terraform Provider

In my personal opioion about this section (The title of the section same as Oracle Documentation) I found it wrong, i worked with Terraform in different cloud vendor, AWS, Azure and OCI so Terraform will recognize it and automatically install the provider for you.

to do that, all of you have to do is create folder , then create file “variables.tf” that only contains

provider "oci" {<br>}

and run terraform command

terraform init

Now Let’s Talk small examples about OCI and Terraform, First you have to read “Creating Module” to understand the rest of this post here.

I will upload to my Github here Small Sample for OCI Terraform to allow you underatand how we can use it instead of the GUI and make it easy for you.

I upload to my github example of Terraform for OCI Proiver, In the this example i will create autonomous database but not using the GUI,

to work with Terraform, you have to understand what is the OCI Provider and the parameters of it.

The Terraform configuration resides in two files: variables.tf (which defines the provider oci) and main.tf (which defines the resource).

For more terraform examples here

Configuration File Requirements

Terraform configuration (.tf) files have specific requirements, depending on the components that are defined in the file. For example, you might have your Terraform provider defined in one file (provider.tf), your variables defined in another (variables.tf), your data sources defined in yet another.

Some of the examples for Terraform files here

Provider Definitions

The provider definition relies on variables so that the configuration file itself does not contain sensitive data. Including sensitive data creates a security risk when exchanging or sharing configuration files.

To understand more about provider read here

provider "oci" {
   tenancy_ocid = "${var.tenancy_ocid}"
   user_ocid = "${var.user_ocid}"
   fingerprint = "${var.fingerprint}"
   private_key_path = "${var.private_key_path}"
   region = "${var.region}"
}

Variable Definitions

Variables in Terraform represent parameters for Terraform modules. In variable definitions, each block configures a single input variable, and each definition can take any or all of three optional arguments:

  • Type (Optional): Defines the variable type as one of three allowed values: string, list, and map. If this argument is not used, the variable type is inferred based on default. If no default is provided, the type is assumed to be string
  • Default (Optional) : Sets the default value for the variable. If no default value is provided, the caller must provide a value or Terraform throws an error.
  • Description (Optional) : A human-readable description of the variable.

More information here

For example

variable "AD" {
    default     = "1"
    description = "Availability Domain"
}

Output Configuration

Output variables provide a means to support Terraform end-user queries. This allows users to extract meaningful data from among the potentially massive amount of data associated with a complex infrastructure.

More information here

Example

output "InstancePublicIPs" {
value = ["${oci_core_instance.TFInstance.*.public_ip}"]
}

Resource Configuration

Resources are components of your Oracle Cloud Infrastructure. These resources include everything from low-level components such as physical and virtual servers, to higher-level components such as email and database providers, your DNS record.

For more information here

One of the example :-

resource "oci_core_virtual_network" "vcn1" {
   cidr_block = "10.0.0.0/16"
   dns_label = "vcn1"
   compartment_id = "${var.compartment_ocid}"
   display_name = "vcn1"
}

Data Source Configuration

Data sources represent read-only views of existing infrastructure intended for semantic use in Terraform configurations, for example Get DB node list

data "oci_database_db_nodes" "DBNodeList" {
  compartment_id = "${var.compartment_ocid}"
  db_system_id = "${oci_database_db_system.TFDBNode.id}"
}

Another example, Gets the OCID of the first (default) vNIC


data "oci_core_vnic" "DBNodeVnic" {
  vnic_id = "${data.oci_database_db_node.DBNodeDetails.vnic_id}"
}

Follow me on GitHub here

Cheers

Osama

Create Your First VM with Azure Cloud In different ways

To create your first server/VM on Azure cloud, you have different ways to do that :-

  • Azure Resource Manager
  • Azure PowerShell
  • Azure CLI
  • Azure REST API
  • Azure Client SDK
  • Azure VM Extensions
  • Azure Automation Services

The Azure portal is the easiest way to create resources such as VMs, i will describe each one of them,

The first way which is The Portal here, to do this it’s very simple :-

  • Click on the Create a resource option in the top-left corner of the portal page.
  • Use the Search the Marketplace search bar to find “Ubuntu Server” for example.
  • Press on Create , then new page will be open.
  • Configure the VM, by enter the name, the region, The Subscription,Availability options
  • There are several other tabs you can explore to see the settings you can influence during the VM creation. Once you’re finished exploring, click Review + create to review and validate the settings.
  • On the review screen, Azure will validate your settings. You might need to supply some additional information based on the requirements of the image creator.


This is was the first way to create the VM which is consider the easiet one also.

Azure Resource Manager

assumig you want to create a copy of a VM with the same settings. You could create a VM image, upload it to Azure, and reference it as the basis for your new VM,Azure provides you with the option to create a template from which to create an exact copy of a VM.

You can do this, after create the VM –> Setting –> export template.

Azure PowerShell

Azure PowerShell is ideal for one-off interactive tasks and/or the automation of repeated tasks, note that PowerShell is a cross-platform shell that provides services like the shell window and command parsing.

New-AzVm  -ResourceGroupName "TestResourceGroup"  -Name "test-wp1-eus-vm"  -Location "East US"  -VirtualNetworkName "test-wp1-eus-network"  -SubnetName "default"  -SecurityGroupName "test-wp1-eus-nsg"  -PublicIpAddressName "test-wp1-eus-pubip"  -OpenPorts 80,3389

Azure CLI

The Azure CLI is Microsoft’s cross-platform command-line tool for managing Azure resources such as virtual machines and disks from the command line. It’s available for macOS, Linux, and Windows, this is also found in Different cloud vendor for example For Amazon it’s called aws cli, for Oracle it’s Called OCI-CLI and Google it’s called GCP-CLI.

az vm create --resource-group TestResourceGroup --name test-wp1-eus-vm --image win2016datacenter --admin-username osama --admin-password anything

Programmatic (APIs)

This is no my expertise so i will no go deep dive with it, But we were talking about Azure CLI and powershell, you can install something called Azure REST API and start using differen programing language to deal with Azure, i did this with python for AWS using Boto3 module, i post about it before here.

The same can be done for Azure or any Cloud vendor.

Azure VM Extensions

Azure VM extensions are small applications that allow you to configure and automate tasks on Azure VMs after initial deployment. Azure VM extensions can be run with the Azure CLI, PowerShell, Azure Resource Manager templates, and the Azure portal.

Thank you

Osama Mustafa

Cloud Talk : How much my IaaS will cost on the cloud ?

When the company will move to the cloud, the biggest question to ask , how much it will cost ? there are different ways to determine your IaaS cost, but at first you need to know that PaaS and IaaS much cheapter than IaaS, Each cloud vendor having their own calculator so at least you can estimate the value for one year or understand how much it will cost ? which is good.

Azure

Let’s Start with Azure for example ( since i post a lot about it recently )

When you are estimate the price for any cloud you should take different factors in your mind such as the following :-

  • Region
  • Tier it’s free , Basic … etc
  • How will the clinet/customer pay ? monthly , Yearly , Pay as you go .. etc
  • Supprot for the cloud which option you will choose
  • The deployement princing for example in Azure Dev/test .. etc

Now Azure provides the client with real pricing calculator that allow people to estimate the cost, From here.

to use the portal you should know what services you will choose, and some esstinal information such as How many VM, Database, networking, after you add all the information the report will be generated depends on the period of paying.

But what if i want to move from On Premis to the cloud , is this tool will work ? Total Cost of Ownership or TCO from here

The TCO Calculator helps you understand the cost areas that affect your applications today, such as server hardware, software licenses, electricity, and labor by Define the following :-

  • Servers : -details of your current on-premises
  • Databases :- on-premises database infrastructure
  • Storage :- on-premises storage infrastructure
  • Networking :- on-premises environment

The Genterated report will be like this :-

Amazon

As i already mentioned each Cloud vendor having different apporach of Cloud Pricing but it’s all the same, in AWS you can access the pricing from here , also they have somthing called SIMPLE MONTHLY CALCULATOR From here.

When you generate an estimate, you can either add services directly to your estimate or create a group and add the services to your group.

The AWS Pricing Calculator is an estimation tool that provides an approximate cost of using AWS services based on the usage parameters that you specify. The AWS Pricing Calculator is not a quote tool, and does not guarantee the cost for your actual use of AWS services. The cost estimated by the AWS Pricing Calculator may vary from your actual costs for a number of reasons. Common reasons the estimate may be different from your actual cost include different thing such as Actual Usage, Region used, Change in price, Taxes ( depends on the Region ) .. etc

Oracle

From Oracle , the portal is very simple to use, you can estimate everything using this portal here from Infrastcure cost, database, Application, … etc .

Cheers

Osama

Monitoring Azure services

Azure provides two primary services to monitor the health of your apps and resources.

  • Azure Monitor
  • Azure Service Health

Azure Monitor

maximizes the availability and performance of your applications by delivering a comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments. It helps you understand how your applications are performing and proactively identifies issues affecting them and the resources they depend on.

also it can collect data from different source such as application, operating system, and platform .. etc. for example the following will explain what i mean :-

What it’s monitorDescription
Application monitoring datacollect data about performance and functionality of the code you have written, regardless of its platform.
 OS monitoring dataData about the operating system on which your application is running.
resource monitoring dataData about the operation of an Azure resource.

subscription monitoring dataData about the operation and management of an Azure subscription
tenant monitoring dataData about the operation of tenant-level Azure services such azure AD.
Table explain what is the monitor in Azure Monitor

Azure Monitor starts collecting data. Activity Logs record when resources are created or modified and Metrics tell you how the resource is performing and the resources that it’s consuming, You can extend the data you’re collecting into the actual operation of the resources by enabling diagnostics and adding an agent to compute resources, also with Azure Monitor you can extend the monitoring to be for example :-

  • Application Insights is a service that monitors the availability, performance, and usage of your web applications, whether they’re hosted in the cloud or on-premises.
  • Azure Monitor for containers is a service that is designed to monitor the performance of container workload
  • Azure Monitor for VMs is a service that monitors your Azure VMs at scale, by analyzing the performance and health of your Windows and Linux VMs

How it’s works ?

Alert :- from the name it will notify the admin to about the erros or thereshould to take corrective actions,Alert rules based on metrics can provide alerts in almost real-time, based on numeric values. Alert rules based on logs allow for complex logic across data, from multiple sources.

AutoScale :- uses Autoscale to ensure that you have the right amount of resources running to manage the load on your application effectively.

Azure Service Health

provide personalized guidance and support when issues with Azure services affect you. It can notify you, help you understand the impact of issues, and keep you updated as the issue is resolved.

What is the plus about Azure Service health :-

  • Azure Status provides a global view of the health state of Azure services
  • customizable dashboard that tracks the state of your Azure services in the regions where you use them.
  • Helps you diagnose and obtain support when an Azure service issue affects your resources

Cheers

And Enjoy the Cloud

Osama

Encryption on Azure

What is encryption?

Encryption is the process of making data unreadable and unusable to unauthorized viewers. To use or read the encrypted data, it must be decrypted, which requires the use of a secret key. 

There are two different type :-

  • Symmetric encryption :– Which mean you will use same key  to encrypt and decrypt the data
  • Asymmetric encryption :– Which mean you will use different key , for example Private and public key.

both of these two type having two different ways :-

  • Encryption at rest which mean data stored in a database, or data stored in a storage account.
  • Encryption in transit which means  data actively moving from one location to another.

So, there are different type of Encryption provided by Azure:-

  • Encrypt raw storage
    • Azure Storage Service Encryption :-  encrypts your data before persisting it to Azure Managed Disks, Azure Blob storage, Azure Files, or Azure Queue storage, and decrypts the data before retrieval.
    • Encrypt virtual machine disks low-level encryption protection for data written to physical disk
  • Azure Disk Encryption : this method helps you to encruypt the actually windows or Linux disk, the best way to do this is h Azure Key Vault.
  • Encrypt databases
    • Transparent data encryption :- helps protect Azure SQL Database and Azure Data Warehouse against the threat of malicious activity. It performs real-time encryption and decryption of the database.

The best way to do this which is Azure Key Vault,  cloud service for storing your application secrets. Key Vault helps you control your applications’ secrets by keeping them in a single, why should i use it :-

  • Centralizing the solutions.
  • Securely stored secrets and keys.
  • Monitor access and use.
  • Simplified administration of application secrets.

There are also two different kind of certificate in Azure which will helps you to encrypt for example the website or application, you need to know that Certificates used in Azure are x.509 v3 and can be signed by a trusted certificate authority, or they can be self-signed.

Types of certificates

  • Service certificates are used for cloud services
  • Management certificates are used for authenticating with the management API

Service certificates

which is attached to cloud services and enable secure communication to and from the service. For example, if you deploy a web site, you would want to supply a certificate that can authenticate an exposed HTTPS endpoint. Service certificates, which are defined in your service definition, are automatically deployed to the VM that is running an instance of your role.

Management certificates

allow you to authenticate with the classic deployment model. Many programs and tools (such as Visual Studio or the Azure SDK) use these certificates to automate configuration and deployment of various Azure services. However, these types of certificates are not related to cloud services.

Be noted that you can use Azure Key Vault to store your certificates.

Cheers

Osama

What storage i should go with Azure data storage ? what my business needs ?

Azure provides several storage options that accommodate specific types of data storage needs.

  • Azure SQL Database ( Amazon RDS).

Azure SQL Database is a relational database as a service (DaaS) based on the latest stable version of the Microsoft SQL Server database engine. SQL Database is a high-performance, reliable, fully managed and secure database. You can use it to build data-driven applications and websites in the programming language of your choice without needing to manage infrastructure.

You can migrate your existing SQL Server databases with minimal downtime using the Azure Database Migration Service. The service uses the Microsoft Data Migration Assistant to generate assessment reports that provide recommendations to help guide you through required changes prior to performing a migration. Once you assess and perform any remediation required, you’re ready to begin the migration process. The Azure Database Migration Service performs all of the required steps. You just change the connection string in your apps.

  • Azure Cosmos DB (In Amazon DynamoDB)

Azure Cosmos DB is a globally distributed database service. It supports schema-less data that lets you build highly responsive and Always On applications to support constantly changing data. You can use this feature to store data that is updated and maintained by users around the world. The following illustration shows a sample Azure Cosmos DB database that’s used to store data that’s accessed by people located across the globe.

  • Azure Blob storage (In Amazon it will be Amazon S3)

Azure Blob Storage is unstructured, meaning that there are no restrictions on the kinds of data it can hold. Blobs are highly scalable and apps work with blobs in much the same way as they would work with files on a disk, such as reading and writing data. Blob Storage can manage thousands of simultaneous uploads, massive amounts of video data, constantly growing log files, and can be reached from anywhere with an internet connection.

Blobs aren’t limited to common file formats. A blob could contain gigabytes of binary data streamed from a scientific instrument, an encrypted message for another application, or data in a custom format for an app you’re developing.

  • Azure Data Lake Storage (In Amazon Kinesis Analytics).

The Data Lake feature allows you to perform analytics on your data usage and prepare reports. Data Lake is a large repository that stores both structured and unstructured data.

Azure Data Lake Storage combines the scalability and cost benefits of object storage with the reliability and performance of the Big Data file system capabilities. The following illustration shows how Azure Data Lake stores all your business data and makes it available for analysis.

  • Azure Files (In Amazon Elastic File System)

Azure Files offers fully managed file shares in the cloud that are accessible via the industry standard Server Message Block (SMB) protocol. Azure file shares can be mounted concurrently by cloud or on-premises deployments of Windows, Linux, and macOS. Applications running in Azure virtual machines or cloud services can mount a file storage share to access file data, just as a desktop application would mount a typical SMB share. Any number of Azure virtual machines or roles can mount and access the file storage share simultaneously. Typical usage scenarios would be to share files anywhere in the world, diagnostic data, or application data sharing.

  • Azure Queue (In Amazon Simple Queue Service (SQS))

Azure Queue storage is a service for storing large numbers of messages that can be accessed from anywhere in the world.

Azure Queue Storage can be used to help build flexible applications and separate functions for better durability across large workloads. When application components are decoupled, they can scale independently. Queue storage provides asynchronous message queueing for communication between application components, whether they are running in the cloud, on the desktop, on-premises, or on mobile devices, Typically, there are one or more sender components and one or more receiver components. Sender components add messages to the queue, while receiver components retrieve messages from the front of the queue for processing.

  • Disk Storage (In Amazon Elastic Block Store (EBS))

Disk storage provides disks for virtual machines, applications, and other services to access and use as they need, similar to how they would in on-premises scenarios. Disk storage allows data to be persistently stored and accessed from an attached virtual hard disk. The disks can be managed or unmanaged by Azure, and therefore managed and configured by the user. Typical scenarios for using disk storage are if you want to lift and shift applications that read and write data to persistent disks, or if you are storing data that is not required to be accessed from outside the virtual machine to which the disk is attached, Disks come in many different sizes and performance levels, from solid-state drives (SSDs) to traditional spinning hard disk drives (HDDs), with varying performance abilities.

  • Storage type :-
    • Hot storage tier: optimized for storing data that is accessed frequently. –> Default in Amazon
    • Cool storage tier: optimized for data that are infrequently accessed and stored for at least 30 days. –> S3 Infrequent Access (IA)
    • Archive storage tier: for data that are rarely accessed and stored for at least 180 days with flexible latency requirements. –> In Amazon S3 Glacier

Cheers

Osama