Please use this identifier to cite or link to this item:
Stereo Camera-based Real-Time 3D Virtual Human Construction with Interactive Entertainment Applications
|關鍵字:||電腦視覺;computer vision;立體視覺;姿態預估;物體切割;stereo vision;posture estimation;object segmentation||出版社:||電機工程學系所||引用:|| T. Noma, I. Oishi, H. Futsuhara, H. Baba, T. Ohashi, and T. Ejima, “Motion generator approach to translating human motion from video to animation,” Proc. 7th Pacific Conf. Computer Graphics and Applications, pp. 50-58, Oct. 1999.  K. Watanabe and M. Hokari, “Kinematical analysis and measurement of sports form,” IEEE Trans. Syst., Man, and Cyber., Part A: Systems and Humans, vol. 36, no. 3, pp. 549- 557, 2006.  Xbox- Kinect. http://www.xbox.com/en-US/kinect  Virtools software. http://www.3ds.com/products/3dvia/3dvia-virtools/  I. Haritaoglu, D. Harwood and L. S. Davis, “ real-time surveillance of people and their activities,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 809 – 830, Aug. 2000.  S. Y. Chen, S. Y. Ma, and L. G. Chen, “Efficient moving object segmentation algorithm using background registration technique,” IEEE Trans. Circuits and Systems for Video Technique, vol. 12, no. 7, pp. 577-586, July 2002.  C. Kim, J. Cho, and Y. Lee, “The relational properties among results of background subtraction”, Proc. of Int. Conf. on Advanced Communication Technology, vol. 3, pp. 1887-1890, 2008.  C. F. Juang, C. M. Chang, J. R. Wu, and D. M. Lee, “Computer vision-based human body segmentation and posture estimation,” IEEE Trans. Syst., Man, and Cyber., Part A: Systems and Humans, vol. 39, no. 1, pp. 119-133, Jan. 2009.  H.D. Cheng, X.H. Jiang, Y. Sun and J. Wang, “Color image segmentation: advances and prospects”, Pattern Recognition, vol. 34, pp. 2259–2281, 2001.  Q. Zhou and J.K. Aggarwal, “Tracking and classifying moving objects from video”, Proc. IEEE Int. Workshop Performance Evaluation of Tracking and Surveillance, pp. 52-59, Dec. 2001.  Razali M.T. and Adznan B.J., “Detection and classification of moving object for smart vision sensor,” Proc. Information and Communication Technologies Conf., vol. 1, pp. 733-737, 2006.  T. Horprasert, D. Harwood, and L.S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” Proc. IEEE Int. Conf. Computer Vision, Frame-Rate Workshop, pp. 1-19, Greece, Sept. 1999.  W. C. Du, Vision-Based Real-Time 3D Human Body Segmentation And 3D Virtual Human Model Construction, Master Thesis, National Chung-Hsing University, Taiwan, July 2011.  H. J. Jiang, T. C. Chen, and C. F. Juang, “Stereo camera-based real-time 3D character construction and human behavior following with interactive entertainment application,” Proc. Nat. Symp. System Science and Engineering, pp. 501-506, Taiwan, R.O.C., June 2012.  D. Anderson, R. H. Luke, J. M. Keller, M. Skubic, M. J. Rantz, and M. A. Aud, “Modeling human activity from voxel person using fuzzy logic,” IEEE Trans. Fuzzy Systems, vol. 17, no. 1, pp. 39-49, Feb. 2009.  R. Rosales, M. Siddiqui, J. Alon, and S. Sclaroff, “Estimating 3D body pose using uncalibrated cameras,” Proc. of IEEE Computer Society Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 821 – 827, 2001.  I. Mikic, M. Trivedi, E. Hunter, and P. Cpsman, “Articulated body posture estimation from multi-camera voxel data,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 455 – 460, 2001.  T. Matsuyama, X. Wu, T. Takai, and S. Nobuhara, “Real-time dynamic 3D object shape reconstruction and high-fidelity texture mapping for 3-D video,” IEEE Trans. Circuits and Systems for Video Technology, vol. 14, no. 3, pp. 357-369, March 2004.  K. Takahashi, Y. Nagasawa, and M. Hashimoto, “Remarks on 3D human posture estimation system using simple multi-camera system”, Proc. IEEE Int. Conf. Syst., Man, and Cyber., pp. 1962-1967, 2006.  J. W. Hsieh, C. H. Chuang, S. Y. Chen et al., “Segmentation of Human Body Parts Using Deformable Triangulation”, IEEE Trans. Syst., Man, and Cyber., Part A: Systems and Humans, vol. 40, no. 3, pp. 596-610, May 2010.  L.A. Schwarz, D. Mateus, V. Castaneda, and N. Navab, “ Manifold Learning for ToF-based Human Body Tracking and Activity Recognition”, British Machine Vision Conference (BMVC), 2010.  C. Plagemann, V. Ganapathi, and D. Koller, “Real-time identiﬁcation and localization of body parts from depth images”, IEEE Int. Conf. on Robotics and Automation (ICRA), 2010.  M. Mortara, G. Patane, and M. Spagnuolo, “From geometric to semantic human body models”, Computers & Graphs, vol. 30, no. 2, pp. 185-196 Apr. 2006.  L.A. Schwarz, A. Mkhitaryan, D. Mateus, and N. Navab, “Human skeleton tracking from depth data using geodesic distances and optical flow”, Image and Vision Computing, vol. 30, no. 3, pp. 217-226, Mar. 2012.  T. Y. Zhang and C. Y. Suen, “A fast parallel algorithm for thinning digital patterns,” Communications of the ACM, Vol. 27, No. 3, pp. 236-239, March 1984.  C. Arcelli, G. Sanniti di Baja, “ A One-Pass Two-Operation Process to Detect the Skeletal Pixels on the 4-Distance Transform”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, no. 4, pp. 411-414, April 1989.  C. K. Chui and G. Chen, Kalman Filtering with Real-Time Applications, Springer Series in Information Sciences, Vol. 17 (4th ed.). Springer, New York, 2009.  M. Potmesil, “Generating octree models of 3D objects from their silhouette in a sequence of images”, Computer Vision, Graphics, and Image Processing, vol. 40, pp. 1-29, 1987.  P. Strivasan, P. Lang, and S. Hackwood, “Computational geometric methods in volumetric intersections for 3D reconstruction”, Patter Recognition, vol. 28, no. 8, pp. 843-857, 1990.  S. Iwasawa, K. Ebihara, J. Ohya, and S. Morishima, “Real-time estimation of human body posture from monocular thermal images,” Proc. of IEEE Computer Society Conf. Computer Vision and Pattern Recognition, pp. 15-20, 1997.  Razali M.T. and Asznan B.J., “Detection and classification of moving object for smart vision sensor,” Proc. Information and communication Technologies, vol. 1, pp. 733-737, 2006.  T. Horprasert, D. Harwood, and L.S. Davis, “A statistical approach for real-time robust background subtraction and shadow detection,” Proc. IEEE Int. Conf. Computer Vision, Frame-Rate Workshop, pp. 1-19, Freece, Sept. 1999||摘要:||
本論文提出一種利用在二維所擷取的人體輪廓資訊以即時估測三維人體重要點的方法。三維的人體重要點包含頭，身體的中心點，雙手及雙腳的末端點，左右肩膀以及手肘和膝蓋。本論文利用一台立體攝影機同時擷取左右兩組連續的影像。首先，利用一種角度補償的色彩三原色分割演算法(AC-RGB segmentation)來從背景中分離出移動的物體，並且降低陰影的影響。然後在兩張各別影像上，利用人體輪廓的凸點、人體幾何的特性、膚色、與時序資訊，來求取頭、雙手及雙腳的末端點的二維位置。手肘和膝蓋點的位置主要利用二維減影求取的骨幹來獲得。最後，重要點的深度資訊是利用同一個點在左右影像得到的二維位置來求取。對於那些左右影像所得到二維位置無法匹配的重要點，本論文利用Kalman Filter 演算法來進行預估與修正。最後，將得到的三維特徵點的座標資訊與所預估的人體方向傳入Virtools這套軟體，建立出虛擬的三維人物模型。實驗結果顯示，相較於其他不同色彩模式的影像分割方法，AC-RGB人體分割的方法，有較佳的分割效果。再者，本論文亦建構出一個即時虛擬三維人體模型系統，來驗證所提出方法得可行性。為了顯示本系統的潛力，亦將其應在一些三維互動式娛樂性遊戲上。
This thesis proposes a real-time three-dimensional (3D) human body significant point estimation method by using two-dimensional (2D) extracted body contours. The located 3D body significant points include the head, center of the body, tips of the feet and the hands, the shoulders, the elbows, and the knees. This thesis uses one stereo vision camera to capture both the left and right sets of image sequences at the same time. An Angle-Compensated RGB (AC-RGB) segmentation method is used to segment a moving object from the background in RGB color space and reduce the influence of shadow. In each of the left and right images, 2D locations of the head and tips of the feet and the hands are obtained based on 2D contour convex points, body geometrical characteristics, skin color, and temporal information. Locations of the elbows and the knees are obtained by using the skeleton pixels of a 2D body silhouette. The depth of each significant point is obtained by using the 2D locations of the same point found in the left and right images. The Kalman filter algorithm is used to correct locations of the 3D significant body points and predict the points with mismatching locations from the left and right images. The 3D locations of the significant points and the estimated orientation of the human body are sent to the Virtools software to reconstruct a virtual 3D human model. Experimental results show that the AC-RGB human body segmentation outperforms several segmentation methods used for comparison. This thesis sets up a real-time virtual 3D human model system to verify the effectiveness of the proposed approach. To show the potential of the system, the system is also applied to some 3D interactive entertainment games.
|Appears in Collections:||電機工程學系所|
Show full item record
TAIR Related Article
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.