How to normalize a tensor in PyTorch?

PyTorchServer Side ProgrammingProgramming

A tensor in PyTorch can be normalized using the normalize() function provided in the torch.nn.functional module. This is a non-linear activation function.

  • It performs Lp normalization of a given tensor over a specified dimension.

  • It returns a tensor of normalized value of the elements of original tensor.

  • A 1D tensor can be normalized over dimension 0, whereas a 2D tensor can be normalized over both dimensions 0 and 1, i.e., column-wise or row-wise.

  • An n-dimensional tensor can be normalized over dimensions (0,1, 2,..., n-1).

Syntax

torch.nn.functional.normalize(input, p=2.0, dim = 1)

Parameters

  • Input – Input tensor

  • p – Power (exponent) value in norm formulation

  • dim – Dimension over which the elements are normalized.

Steps

We could use the following steps to normalize a tensor −

  • Import the torch library. Make sure you have it already installed.

import torch
from torch.nn.functional import normalize
  • Create a tensor and print it.

t = torch.tensor([[1.,2.,3.],[4.,5.,6.]])
print("Tensor:", t)
  • Normalize the tensor using different p values and over different dimensions. The above defined tensor is a 2D tensor, so we can normalize it over two dimensions.

t1 = normalize(t, p=1.0, dim = 1)
t2 = normalize(t, p=2.0, dim = 0)
  • Print the above computed normalized tensor.

print("Normalized tensor:\n", t1)
print("Normalized tensor:\n", t2)

Example 1

# import torch library
import torch
from torch.nn.functional import normalize

# define a torch tensor
t = torch.tensor([1., 2., 3., -2., -5.])

# print the above tensor
print("Tensor:\n", t)

# normalize the tensor
t1 = normalize(t, p=1.0, dim = 0)
t2 = normalize(t, p=2.0, dim = 0)

# print normalized tensor
print("Normalized tensor with p=1:\n", t1)
print("Normalized tensor with p=2:\n", t2)

Output

Tensor:
 tensor([ 1., 2., 3., -2., -5.])
Normalized tensor with p=1:
 tensor([ 0.0769, 0.1538, 0.2308, -0.1538, -0.3846])
Normalized tensor with p=2:
 tensor([ 0.1525, 0.3050, 0.4575, -0.3050, -0.7625])

Example 2

# import torch library
import torch
from torch.nn.functional import normalize

# define a 2D tensor
t = torch.tensor([[1.,2.,3.],[4.,5.,6.]])

# print the above tensor
print("Tensor:\n", t)

# normalize the tensor
t0 = normalize(t, p=2.0)

# print the normalized tensor
print("Normalized tensor:\n", t0)

# normalize the tensor in dim 0 or column-wise
tc = normalize(t, p=2.0, dim = 0)

# print the normalized tensor
print("Column-wise Normalized tensor:\n", tc)

# normalize the tensor in dim 1 or row-wise
tr = normalize(t, p=2.0, dim = 1)

# print the normalized tensor
print("Row-wise Normalized tensor:\n", tr)

Output

Tensor:
 tensor([[1., 2., 3.],
   [4., 5., 6.]])
Normalized tensor:
 tensor([[0.2673, 0.5345, 0.8018],
   [0.4558, 0.5698, 0.6838]])
Column-wise Normalized tensor:
 tensor([[0.2425, 0.3714, 0.4472],
   [0.9701, 0.9285, 0.8944]])
Row-wise Normalized tensor:
 tensor([[0.2673, 0.5345, 0.8018],
   [0.4558, 0.5698, 0.6838]])
raja
Published on 06-Dec-2021 11:37:23

Advertisements