Difference Between Tensor and Variable in Pytorch

Tensor is used to define an n-dimension matrix or multi-dimensional arrays which in turn, are used in mathematical operations. They represent many types of arrays such as scalar, vector and n-dimensional arrays. They share similar functionalities to NumPy arrays or Tensorflow tensors. PyTorch tensors support automatic differentiation, which is instrumental for backpropagation in neural networks to improve accuracy and reduce errors. They are used in storing and manipulating data in PyTorch.

Algorithm

• Import PyTorch.

• Define a list.

• Convert it into a tensor.

• Print the tensor.

Example

import torch
lst=[1,2,3,4,5]
tensor=torch.tensor(lst)
print(tensor)

Output

tensor([1,2,3,4,5])

Variable

They are the primary data types in PyTorch. Before updating to PyTorch 0.4.0, they were used to wrap tensors and provide automatic differentiation and tracking computational graphs. Variables were mostly to deploy gradient-based optimisation algorithms before being deprecated. After the PyTorch update to 0.4.0, most of its functionalities were merged with Tensors. Now, the tensors can perform all these functions without the Variable, hence it isn't used. The main reason for deprecation was to reduce the complexity of the API in PyTorch and reduce confusion between tensors and variables.

Algorithm

• Import PyTorch

• Import Variable from PyTorch.

• Create a simple list.

• Wrap the variable function after converting it into a tensor.

• Print the variable.

Example

import torch
from torch.autograd import Variable
lst=[1,2,3,4,5]
var=Variable(torch.tensor(lst))
print(var)

Output

tensor([1,2,3,4,5])

Let's go through the usage of tensors and variables in more complex examples and see for ourselves:

Example With Tensors

It is common knowledge that all programs are run through the CPU of the computer. In order to dramatically speed up computation time, we can harness the power of the GPU (Graphical Processing Unit) to run the program.

Note: If you are using Google Colab, change the runtime to GPU runtime or else the program will throw an error.

Algorithm

• Import tensors from PyTorch.

• Create two tensors of float data type.

• Find their sum and print it.

• Perform the GPU acceleration using the cuda() function.

• Print all the GPU accelerations.

Example

import torch
from torch import tensor, cuda
#to prevent the program from throwing error
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
t1=tensor(6.)
t2=tensor(9.)

res=t1+t2
print(res)

if cuda.is_available():
   t1=t1.cuda()
   print(t1)
   t2=t2.cuda()
   print(t2)
   res=t1+t2
   print(res)

Output

tensor(15.)
tensor(6., device='cuda:0')
tensor(9., device='cuda:0')
tensor(15., device='cuda:0')

Example with variables

To perform differentiation in a given function, we use the grad function. By calculating the computation graph of our input, returning a Python function as its derivative, unlike conventional differentiations where an algebraic derivative is returned.

Here, we return the function of our Variables, after performing a backward pass on the result variable, where the gradient is calculated, the gradient will not be calculated, resulting in the differentiation to return as 'None'

Algorithm

• Import PyTorch.

• Import Variable from PyTorch.

• Wrap two tensors using the Variable function.

• To find the gradient type, it is required to set 'requires_grad'=True

• Find the sum of both tensors.

• Pass the result backside in the network to calculate the gradient.

• Print the gradient value of both tensors.

Example

import torch
from torch.autograd import Variable
var1 = Variable(torch.tensor(6.), requires_grad=True)
var2 = Variable(torch.tensor(9.), requires_grad=True)
result = var1 + var2
print(result)
result.backward()
print(var1.grad)
print(var2.grad)

Output

tensor(15., grad_fn=<AddBackward07>)
tensor(1.)
tensor(1.)

Conclusion

Variables and Tensors used to have different functionalities, before deprecating the Variable function in PyTorch 0.4.0. The variable was used to wrap a multidimensional tensor, to perform differentiations. Since the update, there is no need to use variables in PyTorch, rendering them obsolete.