甲蟲王

論文名稱	Tree-structured multilayer neural network for classification
發表日期	2019-02-01
論文收錄分類	SCI
所有作者	Shiueng-Bien Yang
作者順序	第一作者
通訊作者	是
刊物名稱	Neural Computing and Applications
發表卷數
是否具有審稿制度	是
發表期數
期刊或學報出版地國別/地區	NATGBR-英國
發表年份	2019
發表月份	2
發表形式	電子期刊
所屬計劃案	無
可公開文檔
可公開文檔
可公開文檔

[英文摘要] :
In traditional neural trees (NTs), each internal node is designed as a neural network (NN), such as single- or two-layer
neural networks, to determine which branch should be followed for an input sample. Because each NN contained in the
internal nodes is designed separately, the produced NT does not consider overall effectiveness. Thus, the designed NT is
usually not an optimal NT. In this study, the tree-structured multilayer neural network (TSMLNN) is proposed for
classification. The TSMLNN is similar to an NT, which is the result of dividing a deep multilayer NN into many small subnetworks. The TSMLNN has the advantages of both a multilayer NN and an NT. In addition, the split method is proposed
to determine how to split the network in the TSMLNN. The genetic algorithm is proposed to automatically search for the
weights, activation threshold of each node and the proper number of nodes at each layer according to both the computing
complexity and classification error rate in the TSMLNN, and the proposed TSMLNN tends to be optimal. A heuristic
method is also proposed to help users to decide which TSMLNN is the best within the classification error rate range.
Finally, the performance of the proposed TSMLNN is compared with that of state-of-the-art neural networks in
experiments.

[參考文獻] :
References
1. Reza T, Mohammadreza K (2018) A method for handwritten
word spotting based on particle swarm optimization and multilayer perceptron. IET Softw 12(2):152–159
2. Sudholt S, Fink GA (2016) A deep convolutional neural network
for word spotting in handwritten documents. In: 15th international conference frontiers in handwriting recognition (ICFHR),
Shenzhen, China, pp 277–282
3. Khayyat M, Lam L, Suen CY (2014) Learning-based word
spotting system for Arabic handwritten documents. Pattern
Recogn 47(3):1021–1030
4. Xu Y et al (2015) A regression approach to speech enhancement
based on deep neural networks. IEEE Trans Audio Speech Lang
Process 23(1):7–19
5. Zagoris K, Pratikakis I, Gatos B (2017) Unsupervised word
spotting in historical handwritten document images using document-oriented local features. IEEE Trans Image Process
26(8):4032–4041
Fig. 10 Probability of input samples falling at each level of nodes and
the relative recognition time in TSMLNN for the ImageNet dataset
Table 5 Classification error rates of methods on ImageNet dataset
Methods Top 1 err. Top 5 err.
TSMLNN (depth = 28) 20.49 5.21
VGG [35] (ILSVRC’14) – 8.43
GoogLeNet [36] (ILSVRC’14) – 7.89
PReLU-net [38] 21.59 5.71
BN-inception [39] 21.99 5.81
ResNet-50 [37] 20.74 5.25
ResNet-101 [37] 19.87 4.60
ResNet-152 [37] 19.38 4.49
Neural Computing and Applications
123
Author's personal copy
6. Toselli AH, Puigcerver J, Vidal E (2016) Two methods to
improve confidence scores for Lexicon-free word spotting in
handwritten text. In: 15th international conference frontiers in
handwriting recognition (ICFHR), Shenzhen, China, pp 349–354
7. Siniscalchi SM et al (2014) An artificial neural network approach to
automatic speech processing. Neurocomputing 140(22):326–338
8. Luca B, Henriques JF, Valmadre J, Torr P, Vedaldi A (2016)
Learning feed-forward one-shot learners. In Advances in NIPS,
pp 523–531
9. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution
using deep convolutional networks. IEEE Trans Pattern Anal
Mach Intell 38(2):295–307
10. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image superresolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern
recognition, pp 1646–1654
11. Zang F, Du B, Zhang L, Xu M (2016) Weakly supervised
learning based on coupled convolutional neural networks for
aircraft detection. IEEE Trans Geosci Remote Sens 54(9):53–63
12. Larochelle H (2009) Exploring strategies for training deep neural
networks. Mach Learn 1:1–40
13. Hinton G et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research
groups. IEEE Signal Process Mag 29(6):82–97
14. Lu X et al (2013) Speech enhancement based on deep denoising
autoencoder. In: Proceedings of Interspeech, pp 436–440
15. Simard PY et al (2003) Best practices for convolutional neural
networks applied to visual document analysis. In: Proceedings of
international conference on document analysis and recognition,
pp 958–963
16. Ciresan D et al (2012) Multi-column deep neural networks for
image classification. In: Proceedings of annual conference on
computer vision and pattern recognition, pp 3642–3649
17. Chen TE, Yang SI, Ho LT, Tsai KH, Chen YH, Chang YF, Lai
YH, Wang SS, Tsao Y, Wu CC (2017) S1 and S2 heart sound
recognition using deep neural networks. IEEE Trans Biomed Eng
64(2):372–380
18. Zhang H, Chow TWS, Jonathan Wu QM (2016) Organizing
books and authors by multilayer SOM. IEEE Trans Neural Netw
Learn Syst 27(12):2537–2550
19. Zhang H, Wang S, Xu X, Chow TWS, Jonathan Wu QM (2018)
Tree2Vector: learning a vectorial representation for tree-structured
data. IEEE Trans Neural Netw Learn Syst 29(11):5304–5318
20. Zhang H, Wang S, Zhao M, Xu X, Ye Y (2018) Locality
reconstruction models for book representation. IEEE Trans
Knowl Data Eng 30(10):1873–1886
21. Guo H, Gelfand SB (1992) Classification trees with neural networks feature extraction. IEEE Trans Neural Netw 3(6):923–933
22. Foresti GL, Micheloni C (2002) Generalized neural trees for
pattern classification. IEEE Trans Neural Netw 13(6):1540–1547
23. Foresti GL (2004) An adaptive high-order neural tree for pattern
recognition. IEEE Trans Syst Man Cybern Part B Cybern
34(2):988–996
24. Maji P (2008) Efficient design of neural network tree using a
single splitting criterion. Nerocomputing 71:787–800
25. Micheloni C, Rani A, Kumarb S, Foresti GL (2012) A balanced
neural tree for pattern classification. Neural Netw 27:81–90
26. Balestriero R (2017) Neural decision trees, 1–11. arXiv:1702.
07360v2
27. Yang SB (2018) Constrained-storage variable-branch neural tree
for classification. Neural Comput Appl. https://doi.org/10.1007/
s00521-017-3315-y
28. Mahmoudabadi H, Izadi M, Menhaj MB (2009) A hybrid method
for grade estimation using genetic algorithm and neural networks.
Comput Geosci 13:91–101
29. Samanta B, Bandopadhyay S, Ganguli R (2004) Data segmentation and genetic algorithms for sparse data division in Nome
placer gold grade estimation using neural network and geostatistics. Min Explor Geol 11(1):69–76
30. Chatterjee S, Bandopadhyay S, Machuca D (2010) Ore grade
prediction using a genetic algorithm and clustering based
ensemble neural network model. Math Geosci 42(3):309–326
31. Tahmasebi P, Hezarkhani A (2009) Application of optimized
neural network by genetic algorithm. IAMG09. Stanford
University; California, 2(2):15–23
32. David OE, Greental I (2014) Genetic algorithms for evolving
deep neural networks. GECCO’14, pp 12–16
33. Le Borgne H, Gue´rin-Dugue´ A, O’Connor NE (2007) Learning
midlevel image features for natural scene and texture classification. IEEE Trans Circuits Syst Video Technol 17(3):286–297
34. Gonzalez RC, Woods RE (1992) Digital image processing.
Addison-Wesley, Boston
35. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
36. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D,
Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with
convolutions. arXiv:1409.4842
37. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for
image recognition. arXiv:1512.03385
38. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers:
Surpassing human-level performance on imagenet classification.
arXiv:1502.01852
39. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep
network training by reducing internal covariate shift. arXiv:1502.
03167
40. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S,
Huang Z, Karpathy A, Khosla A, Bernstein M et al (2014)
Imagenet large scale visual recognition challenge. arXiv:1409.
0575