How to measure the Binary Cross Entropy between the target and the input probabilities in PyTorch?

PyTorchServer Side ProgrammingProgramming

We apply the BCELoss() method to compute the binary cross entropy loss between the input and target (predicted and actual) probabilities. BCELoss() is accessed from the torch.nn module. It creates a criterion that measures the binary cross entropy loss. It is a type of loss function provided by the torch.nn module.

The loss functions are used to optimize a deep neural network by minimizing the loss. Both the input and target should be torch tensors having the class probabilities. Make sure that the target is between 0 and 1. Both the input and target tensors may have any number of dimensions. BCELoss() is used for measuring the error of a reconstruction in, for example, an auto-encoder.

Syntax

torch.nn.BCELoss()

Steps

To compute the binary cross entropy loss, one could follow the steps given below −

  • Import the required library. In all the following examples, the required Python library is torch. Make sure you have already installed it.

import torch
  • Create the input and target tensors and print them.

input = torch.rand(3, 5)
target = torch.randn(3, 5).softmax(dim=1)
  • Create a criterion to measure the binary cross entropy loss.

bce_loss = nn.BCELoss()
  • Compute the binary cross entropy loss and print it.

output = bce_loss(input, target)
print('Binary Cross Entropy Loss: \n', output)

Note − In the following examples, we are using random numbers to generate input and target tensors. So, you may get different values of these tensors.

Example 1

In the following Python program, we compute the binary cross entropy loss between the input and target probabilities.

import torch
import torch.nn as nn

input = torch.rand(6, requires_grad=True)
target = torch.rand(6)

# create a criterion to measure binary cross entropy
bce_loss = nn.BCELoss()

# compute the binary cross entropy
output = bce_loss(input, target)
output.backward()
print('input:\n ', input)
print('target:\ n ', target)
print('Binary Cross Entropy Loss: \n', output)

Output

input:
   tensor([0.3440, 0.7944, 0.8919, 0.3551, 0.9817, 0.8871], requires_grad=True)
target:
   tensor([0.1639, 0.4745, 0.1537, 0.5444, 0.6933, 0.1129])
Binary Cross Entropy Loss:
   tensor(1.2200, grad_fn=<BinaryCrossEntropyBackward>)

Notice that both the input and target tensor elements are in between 0 and 1.

Example 2 −

In this program, we compute the BCE loss between the input and target tensors. Both the tensors are 2D. Notice that for the target tensor, we use softmax() function to make its elements between 0 and 1

import torch
import torch.nn as nn

input = torch.rand(3, 5, requires_grad=True)
target = torch.randn(3, 5).softmax(dim=1)

loss = nn.BCELoss()

output = loss(input, target)
output.backward()

print("Input:\n",input)
print("Target:\n",target)
print("Binary Cross Entropy Loss:\n",output)

Output

Input:
   tensor([[0.5080, 0.5674, 0.1960, 0.7617, 0.9675],
      [0.8497, 0.4167, 0.4464, 0.6646, 0.7448],
      [0.4477, 0.6700, 0.0358, 0.8317, 0.9484]],
   requires_grad=True)
Target:
   tensor([[0.0821, 0.2900, 0.1864, 0.1480, 0.2935],
      [0.1719, 0.3426, 0.0729, 0.3616, 0.0510],
      [0.1284, 0.1542, 0.1338, 0.1779, 0.4057]])
Cross Entropy Loss:
   tensor(1.0689, grad_fn=<BinaryCrossEntropyBackward>)

Notice that the elements of the input and target tensors are in between 0 and 1.

raja
Updated on 25-Jan-2022 08:13:43

Advertisements