Microsoft Azure Data Box is the best service to migrate large amount of data to Microsoft Azure storage. For small to medium size of cold data you can use Azcopy and Azure Import/Export service. Data Box is more secure than Azure Import/Export because it provides end-to-end data migration capabilities with the help of partners.
Let me explain you about Azure Data Box through visualization.
This service can be easily requested from Azure portal. Here is how it work:
Request for Azure Data Box service through Azure portal
Microsoft ships it to your address
You receive, connect and fill the data
You return it to the Microsoft
Microsoft upload the data based on your requirements
Microsoft erase the Data Box and wipe it as per NIST SP 800-88r1 standards
It is very fast process and doesn’t waste your time. You get only 10 days of time by default to copy your data to the Azure Data Box. These 10 days exclude the day you receive and the day carrier scans your package. For any reason, you couldn’t copy your data within 10 days then you need to pay fee extra on the daily basis. Here is a pricing details (Courtesy: Microsoft) for one full round-trip of this service:
Device Standard Shipping
Import Service FeePREVIEW
Extra Day FeePREVIEW
If this device is lost or damaged due to any reason, you will have to pay USD $40,000 as a recovery cost. At present this service is in preview and the above pricing is only applicable to preview and may change any point of time. Here is the list of Azure regions, where this service is available at present (31/May/2018).
Central US, East US, East US2, North Central US, South Central US, West Central US, West US, West US2
North Europe, South Europe
Now, look at the list of Azure partners for this service.
All these details are applicable to preview and may change at the time of GA.
Cloud has become a prominent option for all kind of organizations. When any medium to large organization moves to the cloud, data transfer becomes a biggest challenge. To address this concern, Microsoft provides different types of data transfer options to the customers. Before you get into the details of data transfer options, answer the following questions:
How much data, we need to migrate?
What is going to be the frequency of data transfer?
Data source and destination locations, and respective data regulations?
Find the bottlenecks that may arise at the time of migration?
Do the cost, time and effort comparison between possible data migration types?
If you look from the Microsoft point of view, they have divided the data transfer into four major categories:
Physical data transfer
Data transfer using command line tools and APIs
Data transfer using graphical user interface
Let me explain you briefly about each one of the data transfer methodology:
Physical data transfer: Widely used when you have large data sets to migrate. It could be leveraged for either one-time data migration activity or for less frequent data migration activity. For physical data transfer you can choose one data methodology based on the size.
Azure Import/Export: The Azure Import/Export can be used to transfer large amount of data using internal SATA HDDs or SSDs. By using this service, you can securely transfer data from on-premises to the cloud blob or file storage and vice-versa. When procuring drives for this service, don’t get confuse between SATA and SAS drives. Order SATA III drives as it is faster than older version of SATA drives and support speed of 6 Gbps.
Azure Data Box: Azure Data Box is an option to transfer very large amount of data, it is very similar to Azure Import/Export but avoids the hurdles of procuring, writing and sending multiple data disks. In this service, Microsoft provides you secure and reliable appliance to transfer data between on-premises and cloud blob and file storage. It is much easier than Azure Import/Export service as Microsoft takes the responsibility of end-to-end logistics.
Data transfer using command line tools and APIs: used when you have enough bandwidth available to migrate limited amount of data between on-premises and cloud blob and file storage. There are multiple tools available to perform this activity.
AzCopy: It is command line to tool to transfer data to and from Azure blob, file and table storage in fast, secure and reliable manner. You can install this tool on Windows or Linux machine to transfer data. It supports parallelism and the ability to resume copy operation when interrupted.
Azure CLI: It is a command line tool to manage Azure services and to upload data to Azure storage. Azure CLI doesn’t need any installation and configuration as it is available through Azure portal itself.
PowerShell: PowerShell is an alternative option for windows administrators to transfer data.
Data transfer using graphical user interface: is a most simpler way of transferring data between cloud on-premises. You have two options available to transfer data using graphical tools.
Azure Portal: Simplest way of exploring and uploading files to the Azure blob storage and data lake store but it has a limitation of exploring and uploading only file at a time.
Azure Storage Explorer: Azure storage explorer is a great option for GUI lovers, it provides a capability to manage, upload and download files through interactive interface for blobs, files, queues, tables and Azure Cosmos DBs objects. It also allows to manage data between blob storage, and between storage accounts.
Data Pipeline: used when you need to transfer data regularly.
Azure Data Factory: It is an option to transfer and transform data using data-driven workflows (a.k.a data pipeline) on a regular basis by leveraging orchestration or automation processes. It is a managed service to transfer data between Azure services, on-premises, or a combination of the two. The workflow can be created and scheduled based on your requirements. It can process and transform the data by leveraging compute services such as Azure HDInsight Hadoop, Spark, Azure Data Lake Analytics, and Azure Machine Learning.
Apart from the above core data transfer options, following are the list of tools that can be leveraged to transfer data within specific Azure services.