While the government uses the cloud to store everything from genomic sequences to climate data, the transfer speeds in or out of any cloud are extremely slow compared to sharing data between more traditional storage media.
And those speeds won’t change anytime soon, according to Jay Migliaccio, director of cloud services at Aspera, unless agencies start investing in new transit protocols.
Migliaccio, speaking at the FOSE 2014 conference presented by 1105 Media, explained that the problem is inherent in how WANs handle storage compared to how it is done in the cloud.
WANs can use the Internet’s basic Transmission Control Protocol (TCP) when transferring files, as local hard drives, network attached storage devices, and just about all non-cloud-based storage media use the same method of linear data storage. However, the cloud uses an object-based storage scheme. When moving a file to the cloud, the transfer should be done using Hypertext Transfer Protocol (HTTP), which is inherently difficult with very large files.
“To move a 1 terabyte file to the cloud, you basically have to break it down into thousands of parts to be compatible with the cloud object-based storage system. And it’s neither fast nor efficient.”
To solve this problem, Aspera, which is owned by IBM, has created a protocol that can bridge the gap between normal storage and the cloud. Built on the Fast and Secure Protocol (FASP), which is already used by many agencies for large data transfers, the new service, called Aspera On Demand, creates a fast lane to the cloud. âWe can transfer files directly to the cloud object storage area as if the file lands on a disk,â Migliaccio said.
And it’s not just about raw speed. The new protocol is built on an open architecture which can be integrated into any program or application. Developers can then control the speed of transfers, giving more bandwidth to higher-level clients or reducing the speed when multiple transfers take place. Migliaccio compared it to a car, in which control is as important as speed.
Government users will likely be interested in the fact that transfers with the new protocol can be done with encrypted files. âYou can have the file unencrypted on the other end if you want, while still protecting it during transport,â Migliaccio said. âOr you can have it stay encrypted so it’s protected in the cloud,â he added, which could be an attractive option for agencies using public clouds for storage.
In terms of the speed differences between TCP and Aspera On Demand, Migliaccio said the best that can be hoped for with a typical transfer of a large file to the cloud is around 100 megabytes per second. But with Aspera On Demand, getting speeds of up to 1 gigabit / s is pretty standard.
Agencies interested in using Aspera On Demand to speed up their data transfers to the cloud can find it through Amazon Web Services, where it’s available for rent by the hour or gigabyte. It is also available through Microsoft Azure as a full service. It is FIPS-140-2 certified and uses AES-128-bit encryption, although Migliaccio has said it will be upgraded to 256-bit encryption soon.