Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
PyTorch – How to compute element-wise entropy of an input tensor?
To compute the element-wise entropy of an input tensor, we use torch.special.entr() method. It returns a new tensor with entropy computed element-wise.
If the element of tensor is negative, the entropy is negative infinity.
If the element of the tensor is a zero, the entropy is zero.
The entropy for a positive number element is computed as the negative value of the element multiplied by its natural logarithm. It accepts torch tensor of any dimension.
Steps
We could use the following steps to compute the entropy on a tensor element-wise −
Import the required library. In all the following examples, the required Python library is torch. Make sure you have already installed it.
import torch
Define a torch tensor. Here we define a 2D tensor of random numbers.
tensor = torch.randn(2,3,3)
Compute the entropy of the above-defined tensor using torch.special.entr(tensor). Optionally assign this value to a new variable.
ent = torch.special.entr(tensor)
Print the computed entropy.
print("Entropy:", ent)Example 1
In this example, we compute the entropy of a 1D user-defined tensor.
# import necessary libraries
import torch
# define a 1D tensor
tensor1 = torch.tensor([-1,1,2,0,.4])
# print above created tensor
print("Tensor:", tensor1)
# compute the entropy on input tensor element wise
ent = torch.special.entr(tensor1)
# Display the computed entropies
print("Entropy:", ent)Output
It will produce the following output −
Tensor: tensor([-1.0000, 1.0000, 2.0000, 0.0000, 0.4000]) Entropy: tensor([ -inf, -0.0000, -1.3863, 0.0000, 0.3665])
Notice that the entropy of negative number is -inf, of zero is zero.
Example 2
In this example, we compute the entropy of a 2D torch tensor element-wise.
# import necessary libraries
import torch
# define a tensor of random numbers
tensor1 = torch.randn(2,3,3)
# print above created tensor
print("Tensor:
", tensor1)
# compute the entropy on input tensor element wise
ent = torch.special.entr(tensor1)
# Display the computed entropies
print("Entropy:
", ent)Output
It will produce the following output −
Tensor: tensor([[[ 0.5996, -0.7526, -1.0233], [-0.9907, -0.0358, 0.6433], [ 0.4527, -0.1434, 0.3338]], [[ 0.0521, -0.3729, -0.1162], [ 0.2417, 0.7732, -0.6362], [-0.7942, -0.2582, 1.0860]]]) Entropy: tensor([[[ 0.3067, -inf, -inf], [ -inf, -inf, 0.2838], [ 0.3588, -inf, 0.3663]], [[ 0.1539, -inf, -inf], [ 0.3432, 0.1989, -inf], [ -inf, -inf, -0.0896]]])
2K+ Views
