Activation Function
A mathematical function that determines the output of a neuron based on its weighted input sum.
In-depth explanation
Activation functions introduce non-linearity, enabling neural networks to learn complex patterns. Without them, a network would just be a linear transformation. Common functions include ReLU (max(0, x)), sigmoid (1/(1+e^-x)), tanh, and softmax (for multi-class output). Choice of activation function affects training dynamics and model capabilities.
Examples
More in Neural Networks
Neural Network
A computing system inspired by biological neural networks, consisting of interconnected nodes (neurons).
Neuron
A basic computational unit in a neural network that receives inputs, applies weights and activation, and produces output.
Backpropagation
The algorithm for calculating gradients of the loss function with respect to network weights.
Epoch
One complete pass through the entire training dataset during model training.
Batch Size
The number of training examples used in one iteration of model training.
Master Activation Function.
Learn how to apply this concept with hands-on projects in our comprehensive AI programs.