# PyTorch – How to compute element-wise entropy of an input tensor?

PyTorchServer Side ProgrammingProgramming

To compute the element-wise entropy of an input tensor, we use torch.special.entr() method. It returns a new tensor with entropy computed element-wise.

• If the element of tensor is negative, the entropy is negative infinity.

• If the element of the tensor is a zero, the entropy is zero.

• The entropy for a positive number element is computed as the negative value of the element multiplied by its natural logarithm. It accepts torch tensor of any dimension.

## Steps

We could use the following steps to compute the entropy on a tensor element-wise −

• Import the required library. In all the following examples, the required Python library is torch. Make sure you have already installed it.

import torch
• Define a torch tensor. Here we define a 2D tensor of random numbers.

tensor = torch.randn(2,3,3)
• Compute the entropy of the above-defined tensor using torch.special.entr(tensor). Optionally assign this value to a new variable.

ent = torch.special.entr(tensor)
• Print the computed entropy.

print("Entropy:", ent)

## Example 1

In this example, we compute the entropy of a 1D user-defined tensor.

# import necessary libraries
import torch

# define a 1D tensor
tensor1 = torch.tensor([-1,1,2,0,.4])

# print above created tensor
print("Tensor:", tensor1)

# compute the entropy on input tensor element wise
ent = torch.special.entr(tensor1)

# Display the computed entropies
print("Entropy:", ent)

## Output

It will produce the following output −

Tensor: tensor([-1.0000, 1.0000, 2.0000, 0.0000, 0.4000])
Entropy: tensor([ -inf, -0.0000, -1.3863, 0.0000, 0.3665])

Notice that the entropy of negative number is -inf, of zero is zero.

## Example 2

In this example, we compute the entropy of a 2D torch tensor element-wise.

# import necessary libraries
import torch

# define a tensor of random numbers
tensor1 = torch.randn(2,3,3)

# print above created tensor
print("Tensor:\n", tensor1)

# compute the entropy on input tensor element wise
ent = torch.special.entr(tensor1)

# Display the computed entropies
print("Entropy:\n", ent)

## Output

It will produce the following output −

Tensor:
tensor([[[ 0.5996, -0.7526, -1.0233],
[-0.9907, -0.0358, 0.6433],
[ 0.4527, -0.1434, 0.3338]],
[[ 0.0521, -0.3729, -0.1162],
[ 0.2417, 0.7732, -0.6362],
[-0.7942, -0.2582, 1.0860]]])
Entropy:
tensor([[[ 0.3067, -inf, -inf],
[ -inf, -inf, 0.2838],
[ 0.3588, -inf, 0.3663]],

[[ 0.1539, -inf, -inf],
[ 0.3432, 0.1989, -inf],
[ -inf, -inf, -0.0896]]])