About
As more users adopt AARNet's CloudStor Plus offering within Australia, interim solutions deployed to overcome failures of various distributed replicated storage technologies haven't kept pace with the growth in data volume. AARNet's original design goal of user proximal data storage, combined with national and even international data replication for redundancy reasons continues to be a key driver for design choices.
AARNet's national network is over 90ms from end to end, and accommodating this has been a key issue with numerous software solutions, hindering attempts to provide both original design goals in a reliable real-time manner.
With the addition of features to the ownCloud software allowing primary data storage on OpenStack Swift, AARNet has chosen to deploy Swift in a nation spanning multi-region ring to take advantage of Swift's eventual consistency capabilities and the local region quorum functionality for fast writes.
The scaling capability of Swift resolves the twin problems of geographic redundancy, and user proximal access while scaling into the petabyte range.
Significantly, the ring and policy capabilities allow overflow into short or medium term use secondary storage, such as the Australian RDSI project nodes, or Amazon S3. This helps deliver predictable near linear growth of capital and operational expenditures while allowing a higher rate of growth of data volume.
Additionally, the policy capabilities within Swift combined with the ability to grant a user multiple storage targets within ownCloud, allow us to honour data sovereignty rules with respect to the physical location of the data on a per top level folder basis.
Finally, using the combined read and write affinity features of the Swift proxy, AARNet is presently experimenting with deployment of flash cache backed site local application nodes, providing the perception to the user of near instant data ingestion, while the node trickle uploads data to the redundant bulk storage ring…
The talk
Credits and licence
- Institution
- CERN
- Department
- HEP Computing
- Speaker
- David Mr. JERICHO
- Year
- 2014
- Language
- English
- Subjects
- Computer science
- Source
- CERN
- Licence
- CERN, as CERN states it. CERN's terms
- Attribution
- © CERN. CERN provides its audiovisual media free of charge for educational and informational use, credited to CERN; this lecture is shown in CERN's own player.
- Who can watch
- CERN gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed