A Fine-grained Load Balanced File Placement Algorithm for Distributed Storage Systems

LIU Shuo , XIN Gang

Computer & Telecommunication ›› 2018, Vol. 1 ›› Issue (1-2) : 41-43.

Computer & Telecommunication ›› 2018, Vol. 1 ›› Issue (1-2) : 41-43.

A Fine-grained Load Balanced File Placement Algorithm for Distributed Storage Systems

  • LIU Shuo,XIN Gang
Author information +
History +

Abstract

To gain a balanced distribution of access load, existing distributed storage systems often make file placement decisions based on popularity information. However, on one hand, popularity information is not known at the moment when the file is initially stored in the system. Therefore, placement optimization algorithms which depend on global popularity information are not practical. On the other hand, popularity information changes over time. File storage locations are required to make frequent adjustment,resulting in high data migration cost. In this paper, a new kind of distributed file placement algorithm is proposed, which does not depend on popularity information. It makes use of the correlation between the file's popularity and its creation time. Through a fine-grained similarity of different storage nodes in data amount created in each time interval, the algorithm achieves ideal load balance.Moreover, the algorithm is very practical and causes little migration cost, since it only depends on file creation time information,which is known and static. Experimental results show that the algorithm achieves a more balanced distribution of access load compared to the random placement algorithm adopted in HDFS.

Key words

placement algorithm / distributed file storage system / file popularity / load balance / fine-grained similarity

Cite this article

Download Citations
LIU Shuo , XIN Gang. A Fine-grained Load Balanced File Placement Algorithm for Distributed Storage Systems[J]. Computer & Telecommunication. 2018, 1(1-2): 41-43

Accesses

Citation

Detail

Sections
Recommended

/