The company announced this week that it is now easier to manage Amazon Web Services S3 data with Microsoft’s AzCopy tool. This tool is currently in preview.
AzCopy, a command-line tool that IT pros use to “copy data between file systems and storage accounts or between storage accounts,” according Microsoft’s document description. It is typically used with the Azure File or Azure Blob services for copying data.
AzCopy preview 10.0.8 was released in March. It now offers a better buckets-to blobs transfer capability to allow organizations to move AWS S3 data into Microsoft’s Azure Blob storage service. You can copy an entire AWS S3 bucket. Multiple buckets can be copied.
Microsoft explained that earlier AzCopy versions were less efficient because the data had first been copied from AWS S3 storage onto the client machine running AzCopy, before being transferred to an Azure Storage account. However, the AzCopy preview 10.0.8 tool will transfer data directly into an Azure Storage account.
Microsoft described AzCopy preview version 10.0.08 as its “next generation data transport utility for Azure Storage.” To address scale and resiliency issues, the tool was “redesigned completely.” Microsoft “rewrote its sync commands” and made memory management improvements. Microsoft’s “new Azure Storage API operation Put Block From URL” is what allows the tool to copy data directly.
Microsoft claimed that the data copying rate was 50 Gbps within the same region when moving data between AWS S3 storage and a Microsoft Storage account using AzCopy preview. According to Microsoft, the tool can move terabytes worth of data.
According to Microsoft’s document, the AzCopy preview is also available as an option in GUI-based Azure Storage Explorer tool. According to Microsoft’s description, Azure Storage Explorer can be used with Azure services, Azure Stack, or cloud services to upload/download “blobs and files, queues and tables, and Cosmos DB entities”, according to Microsoft.
Azure Data Factory is another Microsoft option to move AWS S3 data into Azure Storage. This service is more advanced and can be used with hybrid scenarios (data on-premises and in the cloud). Azure Data Factory offers a “dashboard”, which allows you to monitor, move data over the internet or Azure ExpressRoute. You can also schedule incremental loads. Integration with Azure Key Vault is possible for credential management and support for multiple connectors.
