Data migration from SQL to NoSQL using snapshot- Livestream migration

Abstract

The process of moving data from a source database to a destination database is known as data migration. For a variety of reasons, including higher data handling capacity, improved speed, and scalability, many businesses are choosing to convert their databases from one kind (e.g., RDBMS) to another (e.g., NoSQL). Sqoop [3], mongoimport [2], and mongify [1] are a few techniques and technologies that have been developed to help with this transition from RDBMS to NoSQL databases. NoSQL databases use different models, as opposed to the relational model employed by RDBMS, including document, graph, and key-value. Large data volumes were the main focus of the design of NoSQL databases. The database migration model we provide in this paper can effectively transfer both real-time and historical data in parallel. Our Java-based model focuses on transferring data from MongoDB, a document-oriented NoSQL database, to MySQL, an RDBMS. The prototype we created can migrate both live data and a snapshot of the database at a particular moment in time simultaneously. Our experimental evaluation shows that, in terms of performance for both snapshot and live data migration, our model beats competing approaches.