Back تابع التفعيل Arabic Funció d'activació Catalan Aktivační funkce Czech Aktivierungsfunktion German Función de activación Spanish تابع فعالسازی Persian Fonction d'activation French פונקציית אקטיבציה HE 活性化関数 Japanese 활성화 함수 Korean
| Part of a series on |
| Machine learning and data mining |
|---|
The activation function of a node in an artificial neural network is a function that calculates the output of the node based on its individual inputs and their weights. Nontrivial problems can be solved using only a few nodes if the activation function is nonlinear.[1] Modern activation functions include the smooth version of the ReLU, the GELU, which was used in the 2018 BERT model,[2] the logistic (sigmoid) function used in the 2012 speech recognition model developed by Hinton et al,[3] the ReLU used in the 2012 AlexNet computer vision model[4][5] and in the 2015 ResNet model.
- ^ Hinkelmann, Knut. "Neural Networks, p. 7" (PDF). University of Applied Sciences Northwestern Switzerland. Archived from the original (PDF) on 2018-10-06. Retrieved 2018-10-06.
- ^ Hendrycks, Dan; Gimpel, Kevin (2016). "Gaussian Error Linear Units (GELUs)". arXiv:1606.08415 [cs.LG].
- ^ Hinton, Geoffrey; Deng, Li; Deng, Li; Yu, Dong; Dahl, George; Mohamed, Abdel-rahman; Jaitly, Navdeep; Senior, Andrew; Vanhoucke, Vincent; Nguyen, Patrick; Sainath, Tara; Kingsbury, Brian (2012). "Deep Neural Networks for Acoustic Modeling in Speech Recognition". IEEE Signal Processing Magazine. 29 (6): 82–97. doi:10.1109/MSP.2012.2205597. S2CID 206485943.
- ^ Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E. (2017-05-24). "ImageNet classification with deep convolutional neural networks". Communications of the ACM. 60 (6): 84–90. doi:10.1145/3065386. ISSN 0001-0782.
- ^ King Abdulaziz University; Al-johania, Norah; Elrefaei, Lamiaa; Benha University (2019-06-30). "Dorsal Hand Vein Recognition by Convolutional Neural Networks: Feature Learning and Transfer Learning Approaches" (PDF). International Journal of Intelligent Engineering and Systems. 12 (3): 178–191. doi:10.22266/ijies2019.0630.19.