Performance Analysis of Sigmoid and Relu Activation Functions in Deep Neural Network

Abstract

Now in the modern era of technology, everyone wants to get accurate and relevant results within a minimum period from a system. At the time of deciding to develop a deep neural network for generating the desired result, the model must be robust and efficient. This manuscript contains an original research work where two activations, namely sigmoid and Relu, are critically analyzed and studied for understanding the comparative performance of these two activations-based models. There are three deep learning libraries used for loading and preprocessing datasets such as Keras, NumPy, and Scikit-learn. The most popular MNIST dataset is used in this model. Further, the loaded dataset is divided into two parts for training and testing, and it has been done with a total number of ten iterations. The defined model is a multiclass network model having ten output classes, so that softmax activation is used in the output layer, which performs well with multiclass classification.