Please use this identifier to cite or link to this item:
The study on applying the visualization of feature map and convolutional neural network to pollen grains classification
|關鍵字:||深度學習;卷積神經網路;資料擴充;特徵圖可視化;Deep learning;convolutional neural network;data augmentation;visualization of feature map||引用:|| A. B. Gonçalves, J. S. Souza, G. G. da Silva, M. P. Cereda, A. Pott, M. H. Naka, et al., 'Feature Extraction and Machine Learning for the Classification of Brazilian Savannah Pollen Grains,' PloS one, vol. 11, p. e0157044, 2016.  A. Daood, E. Ribeiro, and M. Bush, 'Pollen Grain Recognition Using Deep Learning,' in International Symposium on Visual Computing, 2016, pp. 321-330.  P. Li and J. R. Flenley, 'Pollen texture identification using neural networks,' Grana, vol. 38, pp. 59-64, 1999.  E. Stillman and J. R. Flenley, 'The needs and prospects for automation in palynology,' Quaternary Science Reviews, vol. 15, pp. 1-5, 1996.  I. France, A. Duller, G. Duller, and H. Lamb, 'A new approach to automated pollen analysis,' Quaternary Science Reviews, vol. 19, pp. 537-546, 2000.  M. Rodriguez-Damian, E. Cernadas, A. Formella, and R. Sa-Otero, 'Pollen classification using brightness-based and shape-based descriptors,' in Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on, 2004, pp. 212-215.  A. Corbi, C. Cortes, J. Bousquet, A. Basomba, A. Cistero, J. Garcia-Selles, et al., 'Allergenic cross-reactivity among pollens of Urticaceae,' International Archives of Allergy and Immunology, vol. 77, pp. 377-383, 1985.  S. H. Landsmeer, E. A. Hendriks, L. A. De Weger, J. H. Reiber, and B. C. Stoel, 'Detection of pollen grains in multifocal optical microscopy images of air samples,' Microscopy research and technique, vol. 72, pp. 424-430, 2009.  W. Treloar, G. Taylor, and J. Flenley, 'Towards automation of palynology 1: analysis of pollen shape and ornamentation using simple geometric measures, derived from scanning electron microscope images,' Journal of quaternary science, vol. 19, pp. 745-754, 2004.  N. M. García, V. A. E. Chaves, J. C. Briceño, and C. M. Travieso, 'Pollen grains contour analysis on verification approach,' in International Conference on Hybrid Artificial Intelligence Systems, 2012, pp. 521-532.  M. del Pozo-Baños, J. R. Ticay-Rivas, J. B. Alonso, and C. M. Travieso, 'Features extraction techniques for pollen grain classification,' Neurocomputing, vol. 150, pp. 377-391, 2015.  D. S. da Silva, L. N. B. Quinta, A. B. Gonçalves, H. Pistori, and M. R. Borth, 'Application of wavelet transform in the classification of pollen grains,' African Journal of Agricultural Research, vol. 9, pp. 908-913, 2014.  M. Langford, G. Taylor, and J. Flenley, 'Computerized identification of pollen grains by texture analysis,' Review of Palaeobotany and Palynology, vol. 64, pp. 197-203, 1990.  M. del Pozo-Baños, J. R. Ticay-Rivas, J. Cabrera-Falcón, J. Arroyo, C. M. Travieso-González, L. Sánchez-Chavez, et al., 'Image processing for pollen classification,' in Biodiversity Enrichment in a Diverse World, ed: Intech, 2012.  Y. Zhang, D. Fountain, R. Hodgson, J. Flenley, and S. Gunetileke, 'Towards automation of palynology 3: pollen pattern recognition using Gabor transforms and digital moments,' Journal of quaternary science, vol. 19, pp. 763-768, 2004.  S. W. Punyasena, D. K. Tcheng, C. Wesseln, and P. G. Mueller, 'Classifying black and white spruce pollen using layered machine learning,' New Phytologist, vol. 196, pp. 937-944, 2012.  M. Rodriguez-Damian, E. Cernadas, A. Formella, M. Fernández-Delgado, and P. De Sa-Otero, 'Automatic detection and classification of grains of pollen based on shape and texture,' IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 36, pp. 531-542, 2006.  O. Ronneberger, H. Burkhardt, and E. Schultz, 'General-purpose object recognition in 3D volume data sets using gray-scale invariants-classification of airborne pollen-grains recorded with a confocal laser scanning microscope,' in Pattern Recognition, 2002. Proceedings. 16th International Conference on, 2002, pp. 290-295.  M. Reisert and H. Burkhardt, 'Invariant features for 3d-data based on group integration using directional information and spherical harmonic expansion,' in Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, 2006, pp. 206-209.  Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, 'Gradient-based learning applied to document recognition,' Proceedings of the IEEE, vol. 86, pp. 2278-2324, 1998.  A. Krizhevsky, I. Sutskever, and G. E. Hinton, 'Imagenet classification with deep convolutional neural networks,' in Advances in neural information processing systems, 2012, pp. 1097-1105.  N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, 'Dropout: A simple way to prevent neural networks from overfitting,' The Journal of Machine Learning Research, vol. 15, pp. 1929-1958, 2014.  D. P. Kingma and J. Ba, 'Adam: A method for stochastic optimization,' arXiv preprint arXiv:1412.6980, 2014.  D. Klep, 'Data augmentation of a handwritten character dataset for a Convolutional Neural Network and integration into a Bayesian Linear Framework,' 2016.  L. Perez and J. Wang, 'The Effectiveness of Data Augmentation in Image Classification using Deep Learning,' arXiv preprint arXiv:1712.04621, 2017.  M. D. Zeiler and R. Fergus, 'Visualizing and understanding convolutional networks,' in European conference on computer vision, 2014, pp. 818-833.  L. M. Zintgraf, T. S. Cohen, T. Adel, and M. Welling, 'Visualizing deep neural network decisions: Prediction difference analysis,' arXiv preprint arXiv:1702.04595, 2017.||摘要:||
花粉粒在許多領域的應用非常廣泛，如蜂花粉含有高營養價值，有助於人類的美麗和健康，對其辨識並分類能使產品品質得到保證，花粉可能引發花粉過敏，藉由分類花粉可以協助醫生的診斷，花粉化石可用來研究古環境與古氣候的重建。花粉粒的分類傳統採人工方式，然而以人工分類不僅要由專家來執行，顯示花粉粒的分類需要高熟練度，而且當資料量龐大時，處理會非常耗時，因此產生了自動分類的概念。一般對於花粉粒的分類會先用影像處理方法提取出特徵，再以機器學習方法分類，準確率大概落在64%至95%，但人工提取特徵耗時，更重要的是，假如沒有提取到合適的特徵，分類的準確率會不符預期。近年來卷積神經網路(Convolution neural network, CNN)也被應用在花粉粒的研究上，CNN的卷積層具有特徵提取的功能，能夠省去繁雜的前處理步驟，因此Daood在2016年以CNN對花粉粒分類，達到約90%的準確率。本研究使用的資料集包含805張、23類的花粉粒圖像，實驗結果分為兩階段，第一階段將805張影像分為灰階與彩色以三種不同架構的CNN分類，其中彩色圖的準確率皆比灰階圖高，而彩色圖以Simple CNN所得準確率81.55%為最高，相較於使用同資料集的準確率64%有顯著的提升。第二階段以資料擴充將圖像經旋轉、縮放、裁切等方式擴充到8714張，再以Simple CNN建立預測模型，實驗結果達到了95.95%的準確率，相較於同樣以深度學習，包含30類資料集的所得89.95%的差距不大，本研究也額外收集了9類花粉與原本23類資料合併成一32類的資料集，以同樣的實驗步驟得到95.67%的準確率，說明了Simple CNN對於花粉粒分類的泛化性。由於傳統以特徵提取方式能夠對花粉粒特徵作觀察，本研究也透過卷積層的特徵圖可視化觀察花粉粒的特徵，發現同類的花粉粒其所得的特徵數以及過濾器的位置會相似，不同類的花粉粒可由顏色特徵、紋理特徵或形狀特徵去分辨其差異。
Pollen grains is widely used in many fields. For example, bee pollen has high nutritional value and contributes to the beauty and health of human beings, pollen may cause pollen allergies, pollen fossils can be used to study the reconstruction of paleoenvironment and present a variety of paleoclimate features. The above application shows that the pollen grains have a very high research value.The classification of pollen grains is traditionally artificial. However, manual classification not only requires experts to classify pollen grains, showing that the classification of pollen grains requires high proficiency and cost, but also when the dataset is more than 1000 images, the classification task will become time-consuming. Therefore, the concept of automatic classification of pollen grains is generated. Generally, for the classification of pollen grains, the image processing method is used to extract features at first, after obtaining features, the next step is to classify with machine learning methods, the accuracy rate is approximately between 64% and 95%. As we mentioned, feature extraction is time consuming. Above all, if the appropriate features are not extracted, the accuracy of the classification will be affected. In recent years, due to the vigorous development of deep learning, Convolutional neural network(CNN) has also been applied to the study of pollen grains. Because the convolutional layer of the CNN has the feature extraction function, it is able to eliminate the need for complex pre-processing steps and achieving high classification accuracy. Daood et al. classified the pollen grains with CNN in 2016, reaching an accuracy of 89.95% after data augmentation and transfer learning. This study used a dataset containing a total of 805 images, 23 classes. The results of the experiment include two phases. In the first phase, 805 images were divided into grayscale and RGB with CNN classification of three different architectures. The best accuracy is RGB images with Simple CNN achieving 81.55%, which is a significant improvement over  using the same data set. The second part using general data augmentation augmented the images to 8714 by rotating, resizing, and shearing, and then using Simple CNN to establish a predictive model, the experimental results have achieved an accuracy of 95.95% and a precision of 96.09%. Compared with the research using deep learning, there is not much difference in the accuracy of 89.95% obtained from  containing 30 types of dataset, also, our research collected 9 classes of pollen grains and combined with original dataset, following the research flowchart, the 32 classes dataset reached 95.67% of accuracy, proving the generalization of Simple CNN. Because the traditional feature extraction method can explain the importance of the extracted pollen features, this study also observed the features of the pollen grains through visualization of the convolution layer, and found that the same type of pollen grains, the locations of filters will be similar and different types of pollen grains can be distinguished by their different color features, texture features or shape features.
|Appears in Collections:||資訊管理學系|
Show full item record
Files in This Item:
TAIR Related Article
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.