Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
What is Backpropagation Algorithm?
Backpropagation defines the whole process encompassing both the calculation of the gradient and its need in the stochastic gradient descent. Technically, backpropagation is used to calculate the gradient of the error of the network concerning the network’s modifiable weights.
The characteristics of Backpropagation are the iterative, recursive and effective approach through which it computes the updated weight to increase the network until it is not able to implement the service for which it is being trained. Derivatives of the activation service to be known at network design time are needed for Backpropagation.
Backpropagation is widely used in neural network training and calculates the loss function for the weights of the network. Its service with a multi-layer neural network and discover the internal description of input-output mapping.
It is a standard form of artificial network training, which supports computing gradient loss function concerning all weights in the network. The backpropagation algorithm is used to train a neural network more effectively through a chain rule method.
This gradient is used in a simple stochastic gradient descent algorithm to find weights that minimize the error. The error propagates backward from the output nodes to the inner nodes.
The training algorithm of backpropagation involves four stages which are as follows −
Initialization of weights − There are some small random values are assigned.
Feed-forward − Each unit X receives an input signal and transmits this signal to each of the hidden unit Z1, Z2,... Zn. Each hidden unit calculates the activation function and sends its signal Z1 to each output unit. The output unit calculates the activation function to form the response of the given input pattern.
Backpropagation of errors − Each output unit compares activation Yk with the target value Tk to determine the associated error for that unit. It is based on the error, the factor $\delta$k(K = 1, ... . m) is computed and is used to distribute the error at the output unit Yk back to all units in the previous layer. Similarly the factor $\delta$j(j = 1, ... . p) is compared for each hidden unit Zj.
It can update the weights and biases.
Types of Backpropagation
There are two types of Backpropagation which are as follows −
Static Back Propagation − In this type of backpropagation, the static output is created because of the mapping of static input. It is used to resolve static classification problems like optical character recognition.
Recurrent Backpropagation − The Recurrent Propagation is directed forward or directed until a specific determined value or threshold value is acquired. After the certain value, the error is evaluated and propagated backward.
4K+ Views
To Continue Learning Please Login