Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
How to measure the Binary Cross Entropy between the target and the input probabilities in PyTorch?
We apply the BCELoss() method to compute the binary cross entropy loss between the input and target (predicted and actual) probabilities. BCELoss() is accessed from the torch.nn module. It creates a criterion that measures the binary cross entropy loss. It is a type of loss function provided by the torch.nn module.
The loss functions are used to optimize a deep neural network by minimizing the loss. Both the input and target should be torch tensors having the class probabilities. Make sure that the target is between 0 and 1. Both the input and target tensors may have any number of dimensions. BCELoss() is used for measuring the error of a reconstruction in, for example, an auto-encoder.
Syntax
torch.nn.BCELoss()
Steps
To compute the binary cross entropy loss, one could follow the steps given below −
Import the required library. In all the following examples, the required Python library is torch. Make sure you have already installed it.
import torch
Create the input and target tensors and print them.
input = torch.rand(3, 5) target = torch.randn(3, 5).softmax(dim=1)
Create a criterion to measure the binary cross entropy loss.
bce_loss = nn.BCELoss()
Compute the binary cross entropy loss and print it.
output = bce_loss(input, target)
print('Binary Cross Entropy Loss:
', output)Note − In the following examples, we are using random numbers to generate input and target tensors. So, you may get different values of these tensors.
Example 1
In the following Python program, we compute the binary cross entropy loss between the input and target probabilities.
import torch
import torch.nn as nn
input = torch.rand(6, requires_grad=True)
target = torch.rand(6)
# create a criterion to measure binary cross entropy
bce_loss = nn.BCELoss()
# compute the binary cross entropy
output = bce_loss(input, target)
output.backward()
print('input:
', input)
print('target:\ n ', target)
print('Binary Cross Entropy Loss:
', output)Output
input: tensor([0.3440, 0.7944, 0.8919, 0.3551, 0.9817, 0.8871], requires_grad=True) target: tensor([0.1639, 0.4745, 0.1537, 0.5444, 0.6933, 0.1129]) Binary Cross Entropy Loss: tensor(1.2200, grad_fn=<BinaryCrossEntropyBackward>)
Notice that both the input and target tensor elements are in between 0 and 1.
Example 2 −
In this program, we compute the BCE loss between the input and target tensors. Both the tensors are 2D. Notice that for the target tensor, we use softmax() function to make its elements between 0 and 1
import torch
import torch.nn as nn
input = torch.rand(3, 5, requires_grad=True)
target = torch.randn(3, 5).softmax(dim=1)
loss = nn.BCELoss()
output = loss(input, target)
output.backward()
print("Input:
",input)
print("Target:
",target)
print("Binary Cross Entropy Loss:
",output)Output
Input: tensor([[0.5080, 0.5674, 0.1960, 0.7617, 0.9675], [0.8497, 0.4167, 0.4464, 0.6646, 0.7448], [0.4477, 0.6700, 0.0358, 0.8317, 0.9484]], requires_grad=True) Target: tensor([[0.0821, 0.2900, 0.1864, 0.1480, 0.2935], [0.1719, 0.3426, 0.0729, 0.3616, 0.0510], [0.1284, 0.1542, 0.1338, 0.1779, 0.4057]]) Cross Entropy Loss: tensor(1.0689, grad_fn=<BinaryCrossEntropyBackward>)
Notice that the elements of the input and target tensors are in between 0 and 1.
1K+ Views
