Skip to content

DA6. Artificial Neural Networks – Part

Statement

Conduct research in the University of the People Library and the internet on the functioning of an artificial neuron and the concept of the activation function. Report your findings and include a description of the activation function and why the sigsmoid function is important in artificial neurons and how this relates to a biological neuron.

Solution

A network is a collection of nodes connected to each other in different ways, where each node may connect to one or more other nodes using uni or bi directional connection (Greshenson, 2003).

A biological neuron is connected to other neurons through synapses, to receive the neural impulses from other neurons, which passes through the cell body where it is being processed, and if the sum of the impulses is greater than a certain threshold, the neuron outputs a signal through the axon to other neurons (Greshenson, 2003).

An artificial neuron is a computational model that’s mimicking the biological neuron. It receives inputs (impulses from synapses), and theses inputs are processed (usually multiplied) according to a certain waitings, then it generates an output (signal) to other neurons (Greshenson, 2003).

The weights adjust the inputs (increase or decrease) and generate an output that inhibits or excites the next neuron (Greshenson, 2003).

According to Jain (2019), the activation function is a function that takes outputs from the previous neuron and process it further before sending to the next neuron, that is, decides which neurons should be activated and which should not from all neurons that are connected to the current neuron. The concept is similar to the the threshold discussed in biological neurons.

Jain (2019) also states that the activation function should have certain properties, such as: it needs to be differentiable, and suitable so that the gradient descent can be used to find the weights, symmetric around the origin, and has low cost of computation(Jain, 2019).

sigmoid function is an activation function (Jain, 2019) which is a function takes any real value as input and outputs values in the range of 0 to 1 (Baheti, 2021). It is a non-linear function denoted by the formula: f(x) = 1 / (1 + e^-x) (Baheti, 2021). It is commonly used in models that predict probabilities (since it outputs values between 0 and 1), but it is not symmetric around the origin, and it is computationally expensive (Jain, 2019).

We saw how activation functions (including sigmoid) and other components of neural networks mimic the biological neurons, where the incoming impulse represent inputs, the cell itself processes the inputs (using weights and activation function), and decides if the output should be passed to which of the next neurons (Greshenson, 2003).

References