Derivation of Weibull Distributions From Spike-like Inputs in Artificial Neural Networks

Abstract

The idea that artificial neural network based at perceptron can be expressed as a family of Weibull functions is explored. Basically, it is assumed that “spike” inputs produce a kind of deformation on the resulting Sigmoid function or also called activation function. Thus, one would obtain a kind of polynomials so that a relationship to family of polynomials can be well-established. In this paper, it is found that from the fundamental definition of perceptron, the Weibull functions emerge as a family of polynomials that would replace systematically the traditional Sigmoid function. With this, one can conclude that activation function would pass from a binary definition to one continue presumably dictated by distributions of probabilities.