Brain and Perceptrons:
Biological learning systems are built from a very complex web of interconnected neurons.
The human brain has an immensely connected network of approximately 100 billion neurons, each connected on average to 1,000 other neurons.
Even though the neuron transmission speeds are much slower than computer processing speeds, the brain’s massive parallelism makes it extremely powerful for learning and decision-making tasks.
Artificial Neurons:
An artificial neuron is a mathematical function conceived as a model of a biological neuron.
Artificial neurons are the elementary units of an artificial neural network (ANN).
This artificial neuron receives one or more inputs and sums them to produce an output.
Each input is individually weighted, and the sum is passed through a function known as the activation function or transfer function.
📌 Terminology:
- : Input signal
- : Weights associated with inputs
- : Input signal taking a constant value of 1
- : Weight associated with x₀ (called bias)
- : Summation of input signals
- : Activation function (produces the output)
- : Output signal
⚡ Activation Function
In artificial neural networks, the activation function takes the incoming signals as input and produces the output signal. It determines whether a neuron should be activated or not by calculating a weighted sum and applying a transformation.
🧠 Common Activation Functions:
1. Threshold activation function
The threshold activation function is defined as
2. Unit step function
Some time threshold activation function is also defined as unite step function in which case it is called unit step activation functions.
Defined as
3. Sigmoid activation function (logistic function)
one of the most commonly used activation functions is the sigmoidal activation function.
it is defined as:
4. Linear activation function
The Linear activation function is defined by
5. Piecewise saturated linear activation function
It is defined as:
$
6. Gaussian activation function
This is defined as
Explanation:
- is the standard deviation.
- is the mathematical constant Pi.
- ensures the Gaussian shape.
This function is commonly used in radial basis function networks and probabilistic models.
7. Hyperbolic tangent (tanh) activation function
This function is defined as
