|
|
|
| A Fine-grained Load Balanced File Placement Algorithm for Distributed Storage Systems |
| LIU Shuo,XIN Gang |
| AVIC Computing Technique Research Institute |
|
|
|
|
Abstract To gain a balanced distribution of access load, existing distributed storage systems often make file placement decisions based on popularity information. However, on one hand, popularity information is not known at the moment when the file is initially stored in the system. Therefore, placement optimization algorithms which depend on global popularity information are not practical. On the other hand, popularity information changes over time. File storage locations are required to make frequent adjustment,resulting in high data migration cost. In this paper, a new kind of distributed file placement algorithm is proposed, which does not depend on popularity information. It makes use of the correlation between the file's popularity and its creation time. Through a fine-grained similarity of different storage nodes in data amount created in each time interval, the algorithm achieves ideal load balance.Moreover, the algorithm is very practical and causes little migration cost, since it only depends on file creation time information,which is known and static. Experimental results show that the algorithm achieves a more balanced distribution of access load compared to the random placement algorithm adopted in HDFS.
|
|
|
|
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
