Migrating from MongoDB to Azure cosmos db, Using Mongorestore and mongodump manual/offline

In this post, i will discuss how to migrate from mongoDB (in my case the database was hosted on AWS) to Azure CosmosDB, i searched online about different articles how to do that, the problem i faced most of them were discussing the same way which is Online and using 3rd party software which is not applicable for me due to security reason, thefore i decided to post about it maybe it will useful for someone else.

Usually the easiet way which is use Azure Database Migration Service to perform an offline/online migration of databases from an on-premises or cloud instance of MongoDB to Azure Cosmos DB’s API for MongoDB.

There are some prerequisite before start the migration to know more about it read here, the same link explained different ways for migrations, however before you start you should create an instance for Azure Cosmos DB.

Preparation of target Cosmos DB account

Create an Azure Cosmos DB account and select MongoDB as the API. Pre-create your databases through the Azure portal

The home page for azure Cloud

from the search bar just search for “Azure Cosmos DB”

Azure Cosomo DB

You have add new account for the new migration Since we are migrating from MongoDB then The API should be “Azure CosmosDB for MongoDB API”

Create cosmos db

The target is ready for migration but we have to check the connection string so we can use them in our migration from AWS to Azure.

Get the MongoDB connection string to customize

  • the Azure Cosmos DB blade, select the API.
  •  the left pane of the account blade, click Connection String.
  • The Connection String blade opens. It has all the information necessary to connect to the account by using a driver for MongoDB, including a preconstructed connection string.
Connection string

From MongoDB (Source server) you have to take backup for the database, now after the backup is completed, no need to move the backup for another server , mongo providing two way of backup either mongodump (dump) or mongoexport and will generate JSON file.

For example using monogdump

mongodump --host <hostname:port> --db <Databasename that you want to backup > --collection <collectionname> --gzip --out /u01/user/

For mongoexport

mongoexport --host<hostname:port> --db <Databasename that you want to backup > --collection <collectionname> --out=<Location for JSON file>

After the the above command will be finished, in my advice run them in the background specially if the database size is big and generate a log for the background process so you can check it frequently.

Run the restore/import command from the source server , do you remember the connection string, now we will use them to connect to Azure Cosmos DB using the following, if you used mongodump then to restore you have to use mongorestore like the below :-

mongorestore --host testserver.mongo.cosmos.azure.com --port 10255 -u testserver -p  w3KQ5ZtJbjPwTmxa8nDzWhVYRuSe0BEOF8dROH6IUXq7rJgiinM3DCDeSWeEdcOIgyDuo4EQbrSngFS7kzVWlg== --db test --collection test /u01/user/notifications_service/user_notifications.bson.gz  --gzip --ssl --sslAllowInvalidCertificates

notice the follwing :-

  • host : From Azure portal/connection string.
  • Port : From Azure portal/connection string.
  • Password : From Azure portal/connection string.
  • DB : The name of the database you want to be created in azure cosmo,this name will be created during the migration to azure.
  • Collection : The name of the collection you want to be created in azure cosmo,this name will be created during the migration to azure.
  • Location for the backup.
  • gzip because i compressed the backup
  • Migration required to use ssl authication otherwise it will fail.

using mongoimport.

mongoimport --host testserver.mongo.cosmos.azure.com:10255 -u testserver -p w3KQ5ZtJbjPwTmxa8nDzWhVYRuSe0BEOF8dROH6IUXq7rJgiinM3DCDeSWeEdcOIgyDuo4EQbrSngFS7kzVWlg== --db test --collection test --ssl --sslAllowInvalidCertificates --type json --file /u01/dump/users_notifications/service_notifications.json

Once you run the command

Note: if you migrating huge or big databases you need to increase the cosmosdb throughout and database level after the migration will be finished return everything to the normal situation because of the cost.

Cheers

Osama

How to be Azure Solutions Architect Expert

Many of you knows that i have been working on different cloud vendor, oracle cloud infrastructure , Amazon AWS, and MS Azure, and I had chance to work on many of them with hands-on experience and implement projects on all of them.

Now i am working on 2nd book that will include different topics about the 3 of them, DevOps, and comparison between all the three cloud vendor and more.

During the Lockdown, i was working to sharp my skills and test them in the cloud, therefore i decided to go for azure first and trust me when i say “it’s on of the hardest exam i ever did”.

The exam itself it’s totally different from what i used to, real case scenario that you should be aware of azure features, all of them, and configure them.

To be “Azure Solutions Architect Expert”, there are some of the conditions you should go thru, first you need to apply for two exams, AZ-301 & AZ-300

  • AZ-301 Microsoft Azure Architect Design
  • AZ-300 Microsoft Azure Architect Technologies

Both are Part of the requirements for: Microsoft Certified: Azure Solutions Architect Expert, the first exam which AZ-301, disccused the following secure, scalable, and reliable solutions. Candidates should have advanced experience and knowledge across various aspects of IT operations, including networking, virtualization, identity, security, business continuity, disaster recovery, data management, budgeting, and governance. This role requires managing how decisions in each area affects an overall solution. Candidates must be proficient in Azure administration, Azure development, and DevOps, and have expert-level skills in at least one of those domains.

Learning Objectives

  • Determine workload requirements
  • Design for identity and security
  • Design a data platform solution
  • Design a business continuity strategy
  • Design for deployment, migration, and integration
  • Design an infrastructure strategy

For the AZ-300

Learning Objectives

  • Deploy and configure Azure infrastructure
  • Implement workloads and security on Azure
  • Create and deploy apps on Azure
  • Implement Azure authentication and secure data
  • Develop for the cloud

After you completed the both exams successfully you will receive your badge for the three exams, durtation for the exams around 3 hours and trust me you will need it.

Enjoy

Osama

Moving resource to another subscription or another resource group

In this post i will discuss how to move your Azure resource to another account, subscruiption or even another resource group.

when we are talking about resource we mean by that, disk, vm, IP, interface .. etc let’s say everything you create it, it’s consider as resource.

Sometimes you asked to moved your resource to another subscruiption but in my case i need to orgazine the infrastrcutre and make it much more easier to manage by create different resource group, do this it’s very simple step, you can do it either using Azure CLI, Powershell or GUI.

To do this just follow the screenshot :-

  • Login to Azure portal here.
  • From the Dashboard as you see below choose from the left panel –> Resource group

As you see from the below

  • new windows will be opened which the resource group that you have, just choose which one you want by pressing on it, choose from the list , export template
  • Once you choose this option, Azure will start Generating the template, but it will take some time depends on your resource group and how much resource it has, once it’s finished press Download.
  • Congrats, now you are having your resource as JSON file.

Thanks

Osama

Free Online Learning and Certifications for Oracle Cloud Infrastructure and Oracle Autonomous Database

Yes what you read it’s true …

Oracle now providing Free Online courses from Oracle University and not only this After complete the course you will be able to take the certficate for free,

Thank you Oracle for providing this and allow people to learn something new during this hard time for the whole world #CoronaVirus and quarantine.

Try to use your time wisely during these times, it will not come again and trust me when i am saying you will regret it.

The tracks :-

Note: Don’t Send me a message here or Linkedin Asking me about Dumps, if you ready to have the certificate apply for the exam otherwise don’t cheat.

Enjoy

Thanks

Osama