# How to apply rectified linear unit function element-wise in PyTorch?

To apply a rectified linear unit (ReLU) function element-wise on an input tensor, we use torch.nn.ReLU(). It replaces all the negative elements in the input tensor with 0 (zero), and all the non-negative elements are left unchanged. It supports only real-valued input tensors. ReLU is used as an activation function in neural networks.

### Syntax

relu = torch.nn.ReLU()
output = relu(input)

### Steps

You could use the following steps to apply rectified linear unit (ReLU) function element-wise −

• Import the required library. In all the following examples, the required Python library is torch. Make sure you have already installed it.

import torch
import torch.nn as nn
• Define input tensor and print it.

input = torch.randn(2,3)
print("Input Tensor:",input)
• Define a ReLU function relu using torch.nn.ReLU().

relu = torch.nn.ReLU()
• Apply the above-defined ReLU function relu on the input tensor. And optionally assign the output to a new variable

output = relu(input)
• Print the tensor containing ReLU function values.

print("ReLU Tensor:",output)

Let's take a couple of examples to have a better understanding of how it works.

## Example 1

# Import the required library
import torch
import torch.nn as nn
relu = torch.nn.ReLU()
input = torch.tensor([[-1., 8., 1., 13., 9.],
[ 0., 1., 0., 5., -5.],
[ 3., -5., 8., -1., 5.],
[ 0., 3., -1., 13., 12.]])
print("Input Tensor:",input)
print("Size of Input Tensor:",input.size())

# Compute the rectified linear unit (ReLU) function element-wise
output = relu(input)
print("ReLU Tensor:",output)
print("Size of ReLU Tensor:",output.size())

## Output

Input Tensor:
tensor([[-1., 8., 1., 13., 9.],
[ 0., 1., 0., 5., -5.],
[ 3., -5., 8., -1., 5.],
[ 0., 3., -1., 13., 12.]])
Size of Input Tensor:
torch.Size([4, 5])
ReLU Tensor:
tensor([[ 0., 8., 1., 13., 9.],
[ 0., 1., 0., 5., 0.],
[ 3., 0., 8., 0., 5.],
[ 0., 3., 0., 13., 12.]])
Size of ReLU Tensor:
torch.Size([4, 5])

In the above example, notice that the negative elements in the input tensor are replaced with zero in the output tensor.

## Example 2

# Import the required library
import torch
import torch.nn as nn
relu = torch.nn.ReLU(inplace=True)
input = torch.randn(4,5)
print("Input Tensor:",input)
print("Size of Input Tensor:",input.size())

# Compute the rectified linear unit (ReLU) function element-wise
output = relu(input)
print("ReLU Tensor:",output)
print("Size of ReLU Tensor:",output.size())

## Output

Input Tensor:
tensor([[ 0.4217, 0.4151, 1.3292, -1.3835, -0.0086],
[-0.7693, -1.7736, -0.3401, -0.7179, -0.0196],
[ 1.0918, -0.9426, 2.1496, -0.4809, -1.2254],
[-0.3198, -0.2231, 1.2043, 1.1222, 0.7905]])
Size of Input Tensor:
torch.Size([4, 5])
ReLU Tensor:
tensor([[0.4217, 0.4151, 1.3292, 0.0000, 0.0000],
[0.0000, 0.0000, 0.0000, 0.0000, 0.0000],
[1.0918, 0.0000, 2.1496, 0.0000, 0.0000],
[0.0000, 0.0000, 1.2043, 1.1222, 0.7905]])
Size of ReLU Tensor:
torch.Size([4, 5])