Please use this identifier to cite or link to this item: http://hdl.handle.net/11455/9206
標題: 基於Kinect多媒體系統之即時手勢追蹤演算法及其硬體架構設計
Real-Time Gesture Tracking Algorithm and Hardware Architecture Design in Kinect-Based Multimedia Systems
作者: 郭民偉
Kuo, Ming-Wei
關鍵字: 手部追蹤;Hardware architecture;深度圖;3D深度感應器;人機介面;硬體架構;Kinect;Depth map;Hand tracking;3-D depth sensor
出版社: 電機工程學系所
引用: [1] http://en.wikipedia.org/wiki/Human-computer_interaction [2] Feng Tang,Michael Harville,Hai Tao,Ian N. Robinson "Fusion of Local Appearance with Stereo Depth for Object Tracking" , Computer Vision and Pattern Recognition Workshops, 2008. CVPRW ''08. IEEE Computer Society Conference,pp1-8,June 2008 [3] Mahmoud Elmezain, Ayoub Al-Hamadi, Bernd Michaelis, "A Robust Method for Hand Gesture Segmentation and Recognition Using Forward Spotting Scheme in Conditional Random Fields" ,Pattern Recognition (ICPR), 2010 20th International Conference,pp.3850-3853, Aug. 2010 [4] Mohd Shahrimie Mohd Asaari,Shahrel Azmin Suandi, "Hand Gesture Tracking System Using Adaptive Kalman Filter" ,Intelligent Systems Design and Applications (ISDA), 2010 10th International Conference,pp.166-171, Nov. 29 2010-Dec. 1 2010 [5] Zheng Yi,Fan Liangzhong "Moving Object Detection Based on Running Average Background and Temporal Difference" ,Intelligent Systems and Knowledge Engineering (ISKE), 2010 International Conference,pp.270-272, Nov. 2010 [6] http://cnx.org/content/m19013/latest/ [7]http://140.130.15.147/半導體及光電/課程教材/自動化光學檢測/ch04_二值化影像處理與分析.pdf [8]Hee-Sung Kim, Gregorij Kurillo, Ruzena Bajcsy "Hand Tracking and Motion Detection from theSequence of Stereo Color Image Frames" ,Industrial Technology, 2008. ICIT 2008. IEEE International Conference,pp1-6,2008 [9]Sung-il Kand, Annah Roh, Hyunki Hong "Using Depth and Skin Color for Hand Gesture Classification" ,Consumer Electronics (ICCE), 2011 IEEE International Conference ,pp155-156,9-12 Jan. 2011 [10]Ehsan Parvizi and Q.M. Jonathan Wu, "Multiple Object Tracking Based on Adaptive Depth Segmentation" ,Computer and Robot Vision, 2008. CRV ''08. Canadian Conference ,pp273-277,28-30 May 2008 [11]Michael Van den Bergh,Luc Van Gool "Combining RGB and ToF cameras for real-time 3D hand gesture interaction" ,Applications of Computer Vision (WACV), 2011 IEEE Workshop ,pp66-72,5-7 Jan. 2011 [12]Chia-Ping Chen,Yu-Ting Chen,Ping-Han Lee,Yu-Pao Tsai,Shawmin Lei, "Real-time Hand Tracking on Depth Images" ,Visual Communications and Image Processing (VCIP), 2011 IEEE,pp1-4, 6-9 Nov. 2011 [13]Manders, C. Farbiz, F. Chong, J.H,Tang, K.Y. Chua, G.G,Loke, M.H.Yuan, M.L. ,"Robust Hand Tracking Using a Skin Tone and Depth Joint Probability Model", Automatic Face & Gesture Recognition, 2008. FG ''08. 8th IEEE International Conference ,pp1-6, 17-19 Sept. 2008 [14]Harsh Nanda ,Kikuo Fujimura "Visual Tracking Using Depth data" ,Computer Vision and Pattern Recognition Workshop, 2004. CVPRW ''04. Conference ,27-02 June 2004 [15]http://viml.nchc.org.tw/blog/sub_class.php?SUB_ID=1&CLASS_ID=1 [16]http://75.98.78.94/default.aspx [17]Takanori Yokoyama, Toshiki Iwasaki, and Toshinori Watanabe," Motion Vector Based Moving Object Detection and Tracking in the MPEG Compressed Domain", Content-Based Multimedia Indexing, 2009. CBMI ''09. Seventh International Workshop,pp201-206,3-5 June 2009 [18]M. Hu, S. Worrall, A. H. Sadka, A. M. Kondoz, "Face Feature Detection and Model Design For 2-D Scalable Model-Based Video Coding" ,Visual Information Engineering, 2003. VIE 2003. International Conference,pp125-128,7-9 July 2003 [19]蘇芳生,"人臉表情辨識系統",國立中正大學通訊工程學系碩士論文,民國九十三年 [20]曾郁展,"DSP-Based 之即時人臉辨識系統",國立中山大學電機工程學系碩士論文,民國九十四年 [21]蔡嵩陽,"即時手型辨識系統及其於家電控制之應用",民國一百年 [22]Soriano, M., B. Martinkauppi, S. Huovinen, and M. Laaksonen, “Using the Skin Locus to Cope with Changing Illumination Conditions in Color-based Face Tracking,” in Proc. IEEE Nordic Signal Processing Symposium, Kolmar den, Sweden, pp. 383-386, Jul. 13-15, 2000. [23]R. Vijayanandh,Dr. G. Balakrishnan"Human Face Detection Using Color Spaces and Region Property Measures" ,Control Automation Robotics & Vision (ICARCV), 2010 11th International Conference,pp1605-1610, 7-10 Dec. 2010. [24]M. Van den Bergh, F. Bosche, E. Koller-Meier, and L. Van Gool. “Haarlet-based Hand Gesture Recognition for 3D Interaction” ,Workshop on Applications of Computer Vision (WACV),pp1-8 ,December 7-8, 2009. [25] I. J. Ko and H. I. Choi. "Extracting the Hand Region with the Aid of a Tracking Facility", Electronic Letters, vol. 32, no. 17, pp.1561- 1563, 1996. [26] H. Francke, J. Ruiz-del-Solar, and R. Verschae, “Real-time Hand Gesture Detection and Recognition Using Boosted Classifiers and Active Learning,” Lecture Notes in Computer Science, vol. 4872, pp. 533-547, 2007. [27] E.-J. Ong and R. Bowden.”A Boosted Classifier Tree For Hand Shape Detection.“ ,6th IEEE International Conf. on Automatic Face and Gesture Recognition, pp. 889–894, May 2004. [28] S. Ongkittikul, S. Worrall, and A. Kondoz, “Enhanced Hand Tracking Using the k-means Embedded Particle Filter with Mean-shift Vector Resampling” , Visual Information Engineering, 2008. VIE 2008. 5th ,International Conference, pp. 23 –28, Aug. 1 2008. [29] M. Asaari and S. Suandi, “Hand Gesture Tracking System Using Adaptive Kalman Filter”, Intelligent Systems Design and Applications (ISDA), 2010 10th International Conference, pp. 166 –171, Dec 29 2010. [30] A. Argyros and M. Lourakis, “Binocular Hand Tracking and Reconstruction Based on 2d Shape Matching,” ,ICPR, vol. 1, pp. 207–210, 2006.
摘要: 
這幾年手勢辨識的趨勢已經逐漸取代一些普通的控制介面,現今已漸漸使用在各種裝置上面,如Samsung的Smart TV、remote control、Slide show control等。稱之為人機介面(Human-Computer interface(HCI)。過去有許多類似的研究用複雜的演算法達到手部追蹤的效果。但我們現在有更好的選擇在此應用上面。我們提出的演算法,在於我們同時擁有深度圖(Depth Map Image)以及彩度圖(Color Image),透過 OpenNI從Microsoft Kinect for XBOX 360即時截取下來,以及透過我們的演算法即時處理將手部位置計算出來。
我們所提出了一個以硬體為導向的演算法,主要是針對單人以及單手使用,我們的演算法主要分成兩部分。在我們的第一部分我們希望使用者在使用範圍內揮手對我們的系統做一個初始化的動作,藉由此動作我們可以大概估算出我們使用者的深度值(即對攝影機之距離)。
在我們的第二部分在這邊使用者可以在我們的使用範圍裏面移動並去做一個指令辨識的動作去操作我們的多媒體系統。
一開始先將我們的一般2D彩色影像分隔成一個一個8*8的區塊,然後再去對我們的前一張彩度影像去做一個相減的動作,得出來的差異值在去做直方圖統計,用此方式可以得到我們移動物體的邊緣,然後得到移動物件的區塊,再去對我們的移動區塊去做一個深度值的累加最後得到移動物件的深度值去對我們的物件深度去做一個更新的動作。然後再去對我們的物件經過皮膚過濾之後得到人臉以及人手再去根據人手的深度值做二元化,最後再經過一個改良快速中值濾波器將一些零星雜訊去除,最後得到我們的手部二元圖。在辨識指令方面,我們使用一個手掌向前快速推的動作去驅動指令辨識的初始化來避免錯誤偵測。軟體展示視窗部分我是使用OpenCV 函式庫來做為我們的GUI介面。
因軟體處理時間過長,故本論文欲使用硬體加速達成即時處理。硬體架構設計方面,本論文對我們提出的演算法再做簡化,減少所需暫存的資料量與複雜的運算以利硬體實作;我們提出規格為每秒30張畫面,640x480解析度,一點平行度之手勢追蹤架構電路,運作頻率為47.6MHz。

In recent years, hand gesture recognization is the newest interface for many devices, such as Samsung’s SmartTV, Slide show, remote control. It is called HCI (Human-Computer Interface). In the past, there were many complex algorithms. Now we have a new choice in this application. We propose a new algorithm which has depth and color images. Capturing the raw data from XBOX 360 kinect, we calculate the hand position in real-time.
We propose a hardware-oriented algorithm. It is designed for personal user. Our algorithm is separated into two parts. First, we asked the user to shake his/her hand in the detectable region to initiate the system. Our proposed algorithm can convert the captured image to generate user’s depth map. Second, user can move in the region and make a command to operate our multimedia system. The current and the previous luminance frames which are block-based color images are subtracted. After the absolutes of differences were calculated out, their histogram was generated. In this method, the moving pixels can be got. The depth values of the moving blocks in histogram were counted. The largest number of histogram was the user’s depth value. The user’s depth value was used to extract user’s depth region. A skin-tone filter was used to get user’s head region, hand region and depth value. Hand depth region was defined to transfer into a binary image. A modified median filter was used to remove noise from the binary image. Finally, the hand’s depth value and hand position were used to track hand and to recognize command. We used an Command-Trigger-Event (CTE) to trigger the hand tracking. In software, we used OpenCV to be our GUI interface.
Because the software processing time can’t reach real-time requirements, we need to design hardware to achieve real-time processing. We proposed a spec for 30 frames per second, 640x480 resolutions, and the operation frequency is 47.6MHz.
URI: http://hdl.handle.net/11455/9206
其他識別: U0005-2208201213501400
Appears in Collections:電機工程學系所

Show full item record
 

Google ScholarTM

Check


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.