How to apply linear transformation to the input data in PyTorch?

We can apply a linear transformation to the input data using the torch.nn.Linear() module. It supports input data of type TensorFloat32. This is applied as a layer in the deep neural networks to perform linear transformation. The linear transform used −

y = x * W ^ T + b

Here x is the input data, y is the output data after linear transform. W is the weight matrix and b is biases. The weights W have shape (out_features, in_features) and biases b have shape (out_features). They are initialized randomly and updated during the training of a Neural Network.

Syntax

torch.nn.Linear(in_features, out_features)

Parameters

• in_features - It is the size of the input sample.

• out_features - It is the size of output sample.

Steps

You could use the following steps to apply linear transformation to input data −

• Import the required library. In all the following examples, the required Python library is torch. Make sure you have already installed it.

import torch

• Define input tensor and print it.

input = torch.randn(2,3)
print("Input Tensor:
",input)

• Define the Linear transform using suitable in_features and out_features.

linear = nn.Linear(in_features, out_features)

• Apply the above-defined linear transform to the input data. And optionally, assign the output to a new variable.

output = linear(input)

• Print the linear transformed tensor.

print("Transformed Tensor:
",output)

Example 1

# Import the required library
import torch
import torch.nn as nn

# torch.nn.Linear(in_features, out_features, bias=True,
device=None, dtype=None)
in_features = 3
out_features = 5
linear = nn.Linear(in_features, out_features)
input = torch.randn(2,3)
print("Input Tensor:
",input)
print("Size of Input Tensor:
",input.size())

# Compute the linear transformation
output = linear(input)
print("Transformed Tensor:
",output)
print("Size of Transformed Tensor:
",output.size())

Output

Input Tensor:
   tensor([[-0.0921, 0.1992, -1.4930],
      [-0.0827, 0.0926, -0.1487]])
Size of Input Tensor:
   torch.Size([2, 3])
Transformed Tensor:
   tensor([[-0.5778, -0.3074, -0.5339, -0.3443, -0.1688],
      [-0.3149, -0.2189, -0.1795, -0.3617, -0.0839]],
      grad_fn=<AddmmBackward>)
Size of Transformed Tensor:
   torch.Size([2, 5])

Notice that the size has changed after the linear transformation.

Example 2

# Import the required library
import torch
import torch.nn as nn

# torch.nn.Linear(in_features, out_features, bias=True,
device=None, dtype=None)
in_features = 3
out_features = 5
linear = nn.Linear(in_features, out_features, bias=False)
input = torch.randn(2,3)
print("Input Tensor:
",input)
print("Size of Input Tensor:
",input.size())

# Compute the linear transformation
output = linear(input)
print("Transformed Tensor:
",output)
print("Size of Transformed Tensor:
",output.size())

Output

Input Tensor:
   tensor([[-0.6691, -1.6172, -0.9707],
      [-0.4425, 1.4376, -0.8004]])
Size of Input Tensor:
   torch.Size([2, 3])
Transformed Tensor:
   tensor([[ 0.3554, 1.3869, 0.5136, -0.1801, -0.5858],
      [ 0.3262, 0.2073, -0.8602, -0.0161, 0.5233]],
      grad_fn=<MmBackward>)
Size of Transformed Tensor:
   torch.Size([2, 5])

Notice how the size changed after the linear transformation.