Please use this identifier to cite or link to this item: http://hdl.handle.net/11455/9054
標題: 一個具備自主負載平衡的可擴充分散式檔案系統
A Scalable Distributed File System with Autonomous Load-balancing
作者: 馮星叡
Fong, Sing-Ruei
關鍵字: 雲端運算
cloud computing
分散式檔案系統
元數據管理
負載平衡
B+Tree
distributed file system
metadata management
load-balancing
B+Tree
MySQL database
出版社: 電機工程學系所
引用: [1] S. A. Weil, K. T. Pollack, S. A. Brandt, and E. L. Miller, “Dynamic metadata management for petabyte-scale file systems,” In Proceedings of the 2004 ACM, IEEE Conference on Supercomputing (SC ’04). ACM, Nov. 2004 [2] C. M. Wang, C.C Huang, and H.M Liang, ”ASDF: An Autonomous and Scalable Distributed File System”, Cluster, Cloud and Grid Computing (CCGrid), 2011 11th IEEE/ACM International Symposium on, pp. 485-493, 2011 [3] S. Ghemawat, H. Gobioff, and S.T. Leung, ”The Google File System,” SOSP ''03 Proceedings of the nineteenth ACM symposium on Operating systems principles, pp. 29-43, 2003 [4] Y. Zhu and H. Jiang, “CEFT: A Cost-Effective, Fault-Tolerant Parallel Virtual File System,” J. Parallel and Distributed Computing, vol. 66, no. 2, pp. 291-306, Feb. 2006. [5] S.A. Brandt, L. Xue, and E.L. Miller, “Efficient Metadata Management in Large Distributed File System,” Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings, 20th IEEE/11th NASA Goddard Conference on, pp. 290-298, 2003 [6] Y. Zhu, H. Jiang, and J. Wang, “Hierarchical Bloom filter arrays (HBA): A novel, scalable metadata management system for large cluster-based storage,” Cluster Computing, 2004 IEEE International Conference on, pp. 165–174, 2004 [7] R. Buyya, R. Ranjan, and R. N. Calheiros, “Modeling and simulation of scalable Cloud computing environments and the CloudSim toolkit: Challenges and opportunities,” High Performance Computing & Simulation, 2009. HPCS ''09. International Conference on, pp. 1–11, 2009 [8] S. A. Weil, S. A. Brandt, E. L. Miller, and D. D. E. Long, “Ceph: A Scalable, High-Performance Distributed File System”, OSDI ’06: 7th USENIX Symposium on Operating Systems Design and Implementation, pp. 307-320, 2006 [9] D. Roselli, J. Lorch, and T. Anderson. “A comparison of file system workloads,” In Proceedings of the 2000 USENIX Annual Technical Conference, pp. 41-54, 2000 [10] H. Bin and P. Yuxing, “A Novel Metadata Management Scheme in Cloud Computing,” Software Technology and Engineering (ICSTE), 2010 2nd International Conference on, pp. V1-433-V1-438, 2010 [11] M. J. Hwang, D. G. Kim, and H. Y. Youn, “Enhancing the Performance of Metadata Service for Cloud Computing,” 2010 [12] MySQL :: The world''s most popular open source database, http://www.mysql.com/ [13] Dr. Monge''s B+ Tree index notes http://www.cecs.csulb.edu/~monge/classes/share/B+TreeIndexes.html [14] Apache Hadoop Project, Available :http://hadoop.apache.org/ [15] M. A. Vouk, "Cloud computing — Issues, research and implementations," Information Technology Interfaces, 2008. ITI 2008. 30th International Conference on, pp.31-40, 23-26 June 2008. [16] Oracle, Technology, Network, Java, Java SE, Overview http://www.oracle.com/technetwork/java/javase/overview/index.html [17] 蔡明志, “資料結構 : 使用Java = Data Structures Using Java,” 碁峯資訊, 2008 [18] 張真誠, “資料結構概論 : JAVA語言實作 = Fundamentals of data structures using java,” 全華, 2005
摘要: 隨著近年來雲端科技的發展,雲端運算的服務可以提供了廣大的數據中心跟資源可利用。所以,針對用戶需要設計出最適合、高效能、可擴充的私有雲端資料儲存服務系統,使用戶可以獲得輕鬆且快速上傳下載檔案便是一大重點。 對分散式檔案系統中的元數據伺服器(MDS)群組來說,有效率的元數據管理深刻影響著系統的效能及可擴充性的實現。我們需要元數據管理能夠負載平衡,避免個別伺服器負載過高,導致效能下降甚至出現錯誤。而為了具備可擴充性,我們也需要將被管理的伺服器建構成有分級制度的樹狀結構。在這篇論文中,我們設計一個小型家用雲端分散式檔案系統。我們為了有良好的負載平衡管理,設計了以標記(token)-佇列的元數據負載平衡管理,並搭配分散式MySQL資料庫來分離元數據的讀寫操作。此外,可擴充性的實現我們採用基於B+Tree結構的的元數據管理。當有元數據伺服器加入整個伺服器群組,伺服器群組頭會以伺服器計算出的效能權重值建入B+tree。伺服器群組便利用這個樹狀結構來管理伺服器及偵錯。
With the development of cloud technology in recent years, the cloud environment can provide an extensive data center with available resources. With the attention to design a more suitable, high-performance, scalable private cloud data storage system for users, we hope at the same time allow users to upload and download files easily and quickly. To design an efficient metadata management is a critical issue for system performance and particularly the most important part of metadata management is the scalability for distributed file system in metadata server (MDS) cluster. In this thesis, we design a home-cloud distributed file system that can separate read and write metadata operation by distributed MySQL database. A token-queuing scheme is used to balance a workload. To achieve scalability a hierarchical B+Tree structure is adopted to perform the management for metadata servers. The MDS cluster head can automatically calculate a weight value to construct B+Tree to manage MDSs and execute fault detection. Finally, we use CloudSim, an extensible simulation toolkit, to simulate our file system.
URI: http://hdl.handle.net/11455/9054
其他識別: U0005-2308201214112100
文章連結: http://www.airitilibrary.com/Publication/alDetailedMesh1?DocID=U0005-2308201214112100
Appears in Collections:電機工程學系所

文件中的檔案:

取得全文請前往華藝線上圖書館



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.