Please use this identifier to cite or link to this item: http://hdl.handle.net/11455/8690
標題: 利用模糊分群與支持向量機設計模糊分類及迴歸模型
Fuzzy Classification and Regression Model Design Using Fuzzy Clustering and Support Vector Machine
作者: 謝承達
Hsieh, Cheng-Da
關鍵字: Support Vector Machine;支持向量機;fuzzy classification;recurrent regression;模糊分類;遞迴式回歸
出版社: 電機工程學系所
引用: [1] C. Cortes and V. Vapnik. “Support vector networks,” Int. J. Machine Learning, vol. 20, pp.1-25, 1995. [2] V. Vapnik, The Nature of Statistical Learning Theory. New York : Springer - Verlag, 1995. [3] N. Cristianini and J. Shawe-Tayor. An Introduction to Support Vector Machines, Cambridge University Press, 2000. [4] O. Chapelle, P. Haffner, and V. N. Vapnik. “Support vector machines for histogram-based image classification,” IEEE Trans. Neural Networks, vol. 10, pp. 1055 - 1064, 1999. [5] C. J. C. Burges. “A tutorial on support vector machines for pattern recognition,” Data Mining and knowledge Discovery, vol. 2, no. 2, pp. 121-167, 1998. [6] H. Eghbalnia and A. Assadi, “An application of support vector machines and symmetry to computational modeling of perception through visual attention,” Neurocomputing, no. 38-40, pp. 1193-1201, 2001. [7] G. Gomez-Perez, G.. Camps-Valls, J. Gutierrez, and J. Malo, “Perceptual adaptive insensitivity for support vector machine image coding,” IEEE Trans. Neural Networks, vol.16, pp. 1574 - 1581, 2005. [8] F. Rossi and N. Villa, “Support vector machine for functional data classification,” Neurocomputing; vol. 69, no. 7-9, pp. 730-742, 2006. [9] B. Sch lkopf, K. Sung, C. Burges, F. Girosi, P. Niyogi, T. Poggio, and V. Vapnik, “Comparing support vector machines with Gaussian kernels to radial basis function classifiers,” IEEE Trans Signal Processing, vol. 45, pp. 2758-2765, 1997. [10] Y. Chen and J. Z. Wang, “Support vector learning for fuzzy rule-based classification systems,” IEEE Trans. Fuzzy Systems, vol. 11, pp. 716-728, 2003. [11] C. F. Lin and S. D. Wang, “Fuzzy support vector machines,” IEEE Trans. Neural Networks, vol.13, pp. 464-471, 2002. [12] J. H. Chiang and P. Y. Hao, “Supprot vector learning mechanism for fuzzy rule-based modeling: a new approach,” IEEE Trans. Fuzzy Systems, vol.12, pp. 1-11, 2004. [13] A.J. Smola and B. Scholkopf, “A tutorial on Support Vector Regression,” Statistics and Computing, vol. 14, no. 3, pp. 199-222, 2004. [14] C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems: A Neural-Fuzzy Synergism to Intelligent Systems, Prentice Hall, May, 1996. [15] X. Li, B. Hu, and R. Du, “Predicting the parts weight in plastic injection molding using least squares support vector regression,” IEEE Trans. Systems, Man and Cybernetics, Part C: Applications and Reviews, vol. 38, no. 6, pp. 827-833, Nov. 2008. [16] C. C. Chuang, “Fuzzy weighted support vector regression with a fuzzy partition,” IEEE Trans. Syst., Man, and Cybern., Part B: Cybernetics, vol. 37, no. 3, pp. 630-640, June 2007. [17] J.T. Jeng and T.T. Lee, “Support vector machines for the fuzzy neural networks,” in: Proc. IEEE Int. Conf. on System, Man and Cybernetics, 1999, pp.115-120. [18] C.T. Lin, S.F. Liang, C.M. Yeh, and K.W. Fan, “Fuzzy neural network design using support vector regression for function approximation with outliers,” in: Proc. IEEE Int. Conf. on System, Man and Cybernetics, vol. 3, 2005, pp. 2763-2768. [19] C.F. Juang and C.T. Lin, “An on-line self-constructing neural fuzzy inference network and its applications,” IEEE Trans. Fuzzy Systems, vol. 6, no. 1, pp. 12-32 , 1998. [20] S. L. Chiu. “Fuzzy model identification based on cluster estimation,” J. Intell. Fuzzy Syst., vol. 2, pp. 267-278, 1994. [21] S. Q. Wu and M. J. Er, “Dynamic fuzzy neural networks - a novel approach to function approximation,” IEEE Trans. Syst., Man, Cybern., Part B: Cybernetics, vol. 30, no. 2, pp. 358- 364, Apr. 2000. [22] N. K. Kasabov and Q. Song, “DENFIS: Dynamic evolving neural-fuzzy inference system and its application for time-series prediction,” IEEE Trans. Fuzzy Systems, vol. 10, no. 2, pp. 144 -154, April 2002. [23] D. Kukolj and E. Levi, “Identification of complex systems based on neural and Takagi-Sugeno fuzzy model,” IEEE Trans. Syst., Man and Cybern., Part B: Cybernetics, vol. 34, no. 1, pp. 272-282, Feb. 2004. [24] C. S. Ouyang, W. J. Lee, and S. J. Lee, “A TSK-type neurofuzzy network approach to system modeling problems,” IEEE Trans. Syst., Man, Cybern., Part B: Cybernetics, vol. 35, no. 4, pp. 751- 767, 2005. [25] G. Leng, T. M. McGinnity, and G. Prasad, “An approach for on-line extraction of fuzzy rules using a self-organizing fuzzy neural network,” Fuzzy Sets and Systems, vol. 150, no. 2, pp. 211-243, 2005. [26] C. F. Hsu, “Self-organizing adaptive fuzzy neural control for a class of nonlinear systems,” IEEE Trans. Neural Networks, vol.18, no. 4, pp. 1232 - 1241, July 2007. [27] C. F. Juang and Y. W. Tsao, “A type-2 self-organizing neural fuzzy system and its FPGA implementation,” IEEE Trans. Syst., Man and Cybern., Part B: Cybernetics, vol. 38, no. 6, pp. 1537-1548, Dec. 2008. [28] N. R. Pal and S. Saha, “Simultaneous structure identification and fuzzy rule generation for Takagi-Sugeno models,” IEEE Trans. Syst., Man, and Cybern., Part B: Cybernetics, vol. 38, no. 6, pp. 1626-1638, Dec., 2008. [29] Q. Liu and J. Wang, “A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming,” IEEE Trans. Neural Networks, vol. 19, no. 4, pp. 558-570, 2008. [30] L. Wang, R. Zhang, Z. Xu, and J. Peng, “Some characterizations of global exponential stability of a generic class of continuous-time recurrent neural networks,” IEEE Trans. Syst., Man and Cybern., Part B: Cybernetics, vol. 39, no. 3, pp. 763-772, June 2009. [31] C. F. Juang and C.T. Lin, “A recurrent self-organizing neural fuzzy inference network,” IEEE Trans. Neural Networks, vol.10, no. 4, pp. 828-845, Jul. 1999. [32] J. Zhang and A. J. Morris, “Recurrent neuro-fuzzy networks for nonlinear process modeling,” IEEE Trans. Neural Networks, vol. 10, no. 2, pp. 313-326, Feb. 1999. [33] C. H. Lee and C. C. Teng, “Identification and control of dynamic systems using recurrent fuzzy neural networks,” IEEE Trans. Fuzzy Systems, vol. 8, no. 4, pp. 349-366, Aug. 2000. [34] C. F. Juang, “A TSK-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms,” IEEE Trans. Fuzzy Systems, vol. 10, no. 2, pp. 155-170, April 2002. [35] P. A. Mastorocostas and J. B. Theocharis, “A recurrent fuzzy-neural model for dynamic system identification,” IEEE Trans. Syst., Man and Cybern., Part B: Cybernetics, vol. 32, no. 2, pp. 176-190, April 2002. [36] C. J. Lin and C. C. Chin, “Prediction and identification using wavelet-based recurrent fuzzy neural networks,” IEEE Trans. Syst., Man and Cybern., Part B: Cybernetics, vol. 34, no. 5, pp. 2144-2154, Oct. 2004. [37] J. S. Wang and Y. P. Chen, “A Hammerstein recurrent neurofuzzy network with an online minimal realization learning algorithm,” IEEE Trans. Fuzzy Systems, vol. 16, no. 6, pp. 1597-1612, Dec. 2008. [38] P.P. Angelov and D. Filev, “An approach to online identification of Takagi-Sugeno fuzzy models,” IEEE Trans. Syst., Man and Cybern., Part B: Cybernetics, vol. 34, no. 1, pp. 484-498, Feb. 2004. [39] G. Leng, T. M. McGinnity, and G. Prasad, “An approach for on-line extraction of fuzzy rules using a self-organizing fuzzy neural network,” Fuzzy Sets and Systems, vol. 150, no. 2, pp. 211-243, 2005. [40] F. H ppner, F. Klawonn, R. Kruse, and T. Runkler, Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition, Wiley, 1999. [41] S. Chen, G.. J. Gibson, C. F. N. Cowan, and P. M. Grant, “Adaptive equalization of finite non-Linear channels using multilayer perceptrons,” Signal Processing, 20, pp. 107-119, 1990. [42] G. J. Gibson, S. Siu, and C. F. N. Cowan, “The application of nonlinear structures to the reconstruction of binary signals,” IEEE Trans. Signal Processing, 39, pp. 1877-1884, 1991. [43] K. A. Al-Mashouq and I. S. Reed, “The use of neural nets to combine equalization with decoding for severe intersymbol interference channels,” IEEE Trans. Neural Networks, 5, pp. 982-988, 1994. [44] S. Chen, G.. J. Gibson, C. F. N. Cowan, and P. M. Grant, “Reconstruction of binary signals using an adaptive radial-basis-function equalizer,” Signal Processing, 22, pp. 77-93, 1999. [45] D. J. Sebald and J. A. Bucklew, “Support Vector Machine Techniques for Nonlinear Equalization,” IEEE Trans. Signal Processing, 28, pp.3217-3226, 2000. [46] C. Xu and J. L. Prince. “Snakes, Shapes, and Gradient Vector Flow,” IEEE Transaction on Image Proceeding, 7, pp. 359-369, 1998. [47] Z. Yu and C. Bajaj. “Normalized Gradient Vector Diffusion and Image Segmentation,” Proceeding of the 7th European Conference on Computer Vision (ECCV'02), 3, pp. 517-530, 2002. [48] J. Platt, Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: B.Scholkopf, C. Burges, and A. J. Smola, eds. Advances in kernel methods - support vector learning. Cambridge, MA: MIT Press, 185-208, 1999. [49] J. Platt, Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines. Microsoft Research Technical Report MSR-TR-98-14, 1998. [50] J.S.R. Jang and C.T. Sun, “Functional equivalence between radial basis function networks and fuzzy inference system,” IEEE Trans. Neural Networks, vol. 4, no.1, pp. 156-159, 1993. [51] B. Sch lkopf and A.J. Smola, Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, Chap. 13, MIT Press, Cambridge, MA, 2002. [52] LIBSVM: A Library for Support Vector Machines, C. C. Chang and C. J. Lin. [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm/index.html [53] B. Ribeiro, “Kernelized based functions with minkovsky's norm for SVM regression,” in: Proc. IEEE Internat. Joint Conf. on Neural Networks, Honolulu, Hawaii, vol. 3, 2002, pp. 2198-2203. [54] Y. Chen and J.Z. Wang, “Kernel Machines and Additive Fuzzy Systems: Classification and Function Approximation,” in: 12th IEEE Int. Conf. on Fuzzy Systems, ST. Louis MO, USA, vol. 2, 2003, pp. 789-795. [55] C.J. Lin and C.T. Lin, “An ART-based fuzzy adaptive learning control network,” IEEE Trans. Fuzzy Systems, vol. 5, no. 4, pp. 477-496, 1997. [56] L.X. Wang and J.M. Mendel, “Generating fuzzy rules by learning from examples,” IEEE Trans. Systems Man Cybernet., vol. 22, no.6, pp. 1414-1427, 1992. [57] J. S. Zhang and Y. W. Leung, “Robust clustering by pruning outliers,” IEEE Trans. Syst. Man and Cybern., Part B: Cybernetics, vol. 33, no. 6, pp. 993-998, 2003. [58] P. D''Urso, “Fuzzy clustering for data time arrays with inlier and outlier time trajectories,” IEEE Trans. Fuzzy Systems, vol.13, no. 5, pp. 583 - 604, 2005. [59] G. Chen, Y. Chen, and H. Ogmen, “Identifying chaotic system via a wiener-type cascade model,” IEEE Trans. Contr. Syst., vol. 17, no. 5, pp. 29-36, Oct. 1997. [60] J. B. Theocharis, “A high-order recurrent neuron-fuzzy system with internal dynamics: application to the adaptive noise cancellation,” Fuzzy Sets and Systems, vol. 157, no. 4, pp. 471-500, 2006. [61] A. S. Weigend and N. A. Gersehnfield, Time Series Prediction: Forecasting the Future and Understanding the Past, Reading, MA: Addison-Wesley, 1994. [62] S. Singh, “Noise impact on time-series forecasting using an intelligent pattern matching technique,” Pattern Recognition, vol. 32, no. 8, pp. 1389-1398, Aug. 1999. [63] D. Coyle, G. Prasad, and T. M. McGinnity, “ Faster self-organizing fuzzy neural network training and a hyperparameter analysis for a brain-computer interface,” IEEE Trans. Syst., Man, and Cybern., Part: Cybernetics, vol. 39, no. 6, pp. 1458-1471, Dec. 2009. [64] J. M. Leski, “TSK-fuzzy modeling based on -insensitive learning,” IEEE Trans. Fuzzy Systems, vol. 13, no. 2, pp. 181-193, April 2005. [65] C. F. Juang and C. D Hsieh, “TS-fuzzy system-based support vector regression,” Fuzzy Sets and Systems, vol. 160, no. 17, pp. 2486-2504, Sep. 2009. [66] J. M. Mendel, “Computing derivatives in interval type-2 fuzzy logic system,” IEEE Trans. Fuzzy Systems, vol. 12. no. 1, pp. 84-98, Feb. 2004. [67] L. Wang, Z. C. Mu, and H. Guo, “Fuzzy rule-based support vector regression system,” Journal of Control Theory and Applications, vol. 3, no. 3, pp. 230-234, Aug. 2005.
摘要: 
本論文的目的是採用支持向量機(Support Vector Machine)設計模糊分類(fuzzy classification)、前饋式(feedforward)及遞迴式回歸模型(recurrent regression models)。支持向量機(SVM)對模糊模型的訓練可具有降低雜訊的影響(noise effects)及達到良好的推廣能力(generalization ability)。這些設計的模糊模型可應用在不同的分類和回歸問題,其包含通道等化器(channel equalization)、 函數近似(function approximation)、系統鑑別(system identification)及 序列估測(sequence prediction).
文中設計四種新的模糊模型,第一是模糊分類模糊模型即是模糊C平均-支持向量機(Fuzzy C-means based Support Vector Machine),此模型的輸出是輸入資料對各群歸屬值的權重和,利用線性核心支持向量機學習權重參數,使此模糊模型具有良好的分類推廣能力,模擬結果顯示模糊C平均-支持向量機在通道等化的問題上呈現出不錯的雜訊抑制效果。第二和第三均是前饋式模糊回歸模型。第二是TS模糊系統-支持向量回歸(Takagi-Sugeno Fuzzy System based Support Vector Regression),採用一次通行的模糊分群演算法進行訓練資料的分群,一個新的TS-核心是依據TS-模式的模糊規則而來,其架構是群(cluster)的輸出和輸入變數線性組合的乘積,所以此模型的輸出是此TS-核心的線性權重和,使用線性支持向量迴歸學習權重參數。第三是利用自我分裂產生規則數和疊代支持向量回歸建構TS型式模糊系統(Takagi-Sugeno (TS)-type Fuzzy System Constructed by self-splitting Rule Generation and iterative Linear Support Vector Regression),此模型可自動產生規則數是引入自我分裂的技術到K-平均分群演算法中。每條規則的前件部(Antecedent)及後件部(Consequent)被表示成由輸入資料轉換後向量的線性組合係數,可使用線性支持向量回歸作參數學習。第四是一個遞迴式回歸模型,即區域遞迴式模糊類神經網路-支持向量回歸(Locally Recurrent Fuzzy Neural Network with Support Vector Regression)。此模型的目的是處理具有時間特性的問題。遞迴式結構是區域性地將各規則的激發量迴傳給各規則本身,採用一次通行的模糊分群演算法進行訓練資料的分群並決定網路隱藏層的節點(node)數量。使用疊代式線性支持向量回歸對回授路徑及後件部作參數學習。經由不同的模擬範例與其他的分類、回歸模型作比較,證明本論文所提的所有模型有雜訊抑制能力及良好的推廣能力。

This dissertation presents the design of fuzzy classification, feedforward, and recurrent regression models using support vector machine (SVM). The use of SVM for fuzzy model training helps reduce noise effects and achieve high generalization ability. The designed fuzzy models are applied to different classification and regression problems, including channel equalization, function approximation, system identification, and sequence prediction.
Four novel fuzzy models are proposed in this dissertation. The first one is a fuzzy classification model and is called Fuzzy C-means based Support Vector Machine (FCM-SVM). In FCM-SVM, input training data is clustered by fuzzy c-means. The output of FCM-SVM is a weighted sum of the degrees where each input data belongs to the clusters. To achieve high generalization ability, FCM-SVM weights are learned through linear SVM. Simulation results on channel equalization problems show that the FCM-SVM performance is good in reducing noise influence. The second and third are feedforward fuzzy regression models. The second one is the Takagi-Sugeno (TS) Fuzzy System-based Support Vector Regression (TSFS-SVR). In TSFS-SVR, a one-pass clustering algorithm clusters the input training data. A new TS-kernel, which corresponds to a TS-type fuzzy rule, is then constructed by the product of a cluster output and a linear combination of input variables. The TSFS-SVR output is a linear weighted sum of the TS kernels. TSFS-SVR weights are learned through linear SVR. The third one is the TS-type Fuzzy System constructed by self-splitting Rule Generation and iterative Linear Support Vector Regression (FS-RGLSVR). The rules in the FS-RGLSSVR are automatically generated by introducing the self-splitting technique to the K-means clustering algorithm. Each of the consequent and antecedent part parameters is expressed as a linear combination coefficient in a transformed input space so that the linear SVR is applicable. The fourth one is a recurrent fuzzy regression model and is called Locally Recurrent Fuzzy Neural Network with Support Vector Regression (LRFNN-SVR). The LRFNN-SVR is proposed for handling problems with temporal properties. The recurrent structure in a LRFNN-SVR comes from locally feeding the firing strength of each fuzzy rule back to itself. A one-pass clustering algorithm clusters the input training data and determines the number of network nodes in hidden layers. An iterative linear support vector regression (SVR) algorithm is proposed to tune free parameters in the rule consequent part and feedback loops. Comparisons with other classification and regression models in different simulation examples demonstrate the noise robustness and generalization abilities of the proposed fuzzy models.
URI: http://hdl.handle.net/11455/8690
其他識別: U0005-2801201014421500
Appears in Collections:電機工程學系所

Show full item record
 

Google ScholarTM

Check


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.