- Trending Categories
- Data Structure
- Operating System
- MS Excel
- C Programming
- Social Studies
- Fashion Studies
- Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
Difference Between Tensor and Variable in Pytorch
PyTorch is an open-source Python library used for machine learning, computer vision and deep learning. It is an excellent library to build neural networks, conduct complex computations and optimise gradient differentiation.
Developed by Facebook's Research Team (FAIR), it gained popularity due to its dynamic computing graphs, allowing it to change graphs in real time. This was revolutionary back in 2016 when models working in real-time just started to pop off.
There are two main variables which will be focused on, tensor and variable in PyTorch.
Tensor is used to define an n-dimension matrix or multi-dimensional arrays which in turn, are used in mathematical operations. They represent many types of arrays such as scalar, vector and n-dimensional arrays. They share similar functionalities to NumPy arrays or Tensorflow tensors. PyTorch tensors support automatic differentiation, which is instrumental for backpropagation in neural networks to improve accuracy and reduce errors. They are used in storing and manipulating data in PyTorch.
Define a list.
Convert it into a tensor.
Print the tensor.
import torch lst=[1,2,3,4,5] tensor=torch.tensor(lst) print(tensor)
They are the primary data types in PyTorch. Before updating to PyTorch 0.4.0, they were used to wrap tensors and provide automatic differentiation and tracking computational graphs. Variables were mostly to deploy gradient-based optimisation algorithms before being deprecated. After the PyTorch update to 0.4.0, most of its functionalities were merged with Tensors. Now, the tensors can perform all these functions without the Variable, hence it isn't used. The main reason for deprecation was to reduce the complexity of the API in PyTorch and reduce confusion between tensors and variables.
Import Variable from PyTorch.
Create a simple list.
Wrap the variable function after converting it into a tensor.
Print the variable.
import torch from torch.autograd import Variable lst=[1,2,3,4,5] var=Variable(torch.tensor(lst)) print(var)
Let's go through the usage of tensors and variables in more complex examples and see for ourselves:
Example With Tensors
It is common knowledge that all programs are run through the CPU of the computer. In order to dramatically speed up computation time, we can harness the power of the GPU (Graphical Processing Unit) to run the program.
Note: If you are using Google Colab, change the runtime to GPU runtime or else the program will throw an error.
Import tensors from PyTorch.
Create two tensors of float data type.
Find their sum and print it.
Perform the GPU acceleration using the cuda() function.
Print all the GPU accelerations.
import torch from torch import tensor, cuda #to prevent the program from throwing error device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') t1=tensor(6.) t2=tensor(9.) res=t1+t2 print(res) if cuda.is_available(): t1=t1.cuda() print(t1) t2=t2.cuda() print(t2) res=t1+t2 print(res)
tensor(15.) tensor(6., device='cuda:0') tensor(9., device='cuda:0') tensor(15., device='cuda:0')
Example with variables
To perform differentiation in a given function, we use the grad function. By calculating the computation graph of our input, returning a Python function as its derivative, unlike conventional differentiations where an algebraic derivative is returned.
Here, we return the function of our Variables, after performing a backward pass on the result variable, where the gradient is calculated, the gradient will not be calculated, resulting in the differentiation to return as 'None'
Import Variable from PyTorch.
Wrap two tensors using the Variable function.
To find the gradient type, it is required to set 'requires_grad'=True
Find the sum of both tensors.
Pass the result backside in the network to calculate the gradient.
Print the gradient value of both tensors.
import torch from torch.autograd import Variable var1 = Variable(torch.tensor(6.), requires_grad=True) var2 = Variable(torch.tensor(9.), requires_grad=True) result = var1 + var2 print(result) result.backward() print(var1.grad) print(var2.grad)
tensor(15., grad_fn=<AddBackward07>) tensor(1.) tensor(1.)
Variables and Tensors used to have different functionalities, before deprecating the Variable function in PyTorch 0.4.0. The variable was used to wrap a multidimensional tensor, to perform differentiations. Since the update, there is no need to use variables in PyTorch, rendering them obsolete.
Kickstart Your Career
Get certified by completing the courseGet Started