How to convert a PyTorch tensor with gradient to a numpy array?

PyTorchServer Side ProgrammingProgramming

To convert a Torch tensor with gradient to a Numpy array, first we have to detach the tensor from the current computing graph. To do it, we use the Tensor.detach() operation. This operation detaches the tensor from the current computational graph. Now we cannot compute the gradient with respect to this tensor. After the detach() operation, we use the .numpy() method to convert it to a Numpy array.

If a tensor with requires_grad=True is defined on GPU, then to convert this tensor to a Numpy array, we have to perform one more step. First we have to move the tensor to CPU, then we perform Tensor.detach() operation and finally use .numpy() method to convert it to a Numpy array.

Steps

  • Import the required library. The required library is torch.

  • Create a tensor with gradient on CPU. If a tensor with gradient is already defined on the GPU, then we have to move it to the CPU.

  • Detach the tensor from the current computational graph. You can use .detach() to perform this operation. After the detach() operation, the tensor is without gradient.

  • Next, convert the tensor without gradient to a Numpy array. You can use .numpy() to convert it to a Numpy array.

  • Print the Numpy array.

Example 1

# import torch library
import torch

# define a tensor with requires gradient true
x = torch.tensor([1.,2.], requires_grad = True)
print("x:", x)

# x.numpy()--> error
x = x.detach().numpy()
print("x:", x)
print("type of x:", type(x))

Output

x: tensor([1., 2.], requires_grad=True)
x: [1. 2.]
type of x: <class 'numpy.ndarray'>

Example 2

import torch

# define device
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

# define tensors with gradients on gpu
x = torch.tensor([1.,2.], requires_grad = True, device = device)
y = torch.tensor([1.,2., 3.], requires_grad = True, device = device)

print("x:", x)
print("y:", y)

# first move the tensor from gpu to cpu
x = x.to('cpu') # or x = x.cpu()

# then detach the requires_grad
x = x.detach()

# and then convert to numpy
x = x.numpy()
print("x:",x)
print("type of x:", type(x))

y = y.cpu().detach().numpy()
print("y:", y)
print("type of y:", type(y))

Output 1 − If GPU is not available

x: tensor([1., 2.], requires_grad=True)
y: tensor([1., 2., 3.], requires_grad=True)
x: [1. 2.]
type of x: <class 'numpy.ndarray'>
y: [1. 2. 3.]
type of y: <class 'numpy.ndarray'>

Output 2 − If GPU is available

x: tensor([1., 2.], device='cuda:0', requires_grad=True)
y: tensor([1., 2., 3.], device='cuda:0', requires_grad=True)
x: [1. 2.]
type of x: <class 'numpy.ndarray'>
y: [1. 2. 3.]
type of y: <class 'numpy.ndarray'>
raja
Published on 06-Jan-2022 09:34:56

Advertisements