Torch - Introduction

Torch is an open source machine learning library primary to provide a scientific computing framework. Torch was originally built on top of Lua but it is mainly popularized by PyTorch, a very popular Python based deep learning framework which is built upon Torch principles. PyTorch is also known as an extension of Torch.

Tensor: A Basic Building Block

Torch provides a powerful N-dimensional array known as Tensor. A Tensor acts as a basic building block similar to NumPy array but is very important in machine learning.

• GPU Acceleration − A Tensor processing can be moved to GPUs(Graphical Processing Units), leading to faster computations crucial while training large neural networks.

• Automatic Differentiation − A Tensor can track the operations performed on it. This allows automatic computation of gradients, crucial to optimize neural network basis backpropagation.

nn Module: Neurak Network Module

Torch provides nn module, a flexible and powerful package to build a neural network. In case of PyTorch, same is available under torch.nn package.

• Modularized Approach − In Torch, we can build neural network layers as modules. For example, linear layers, convolutional layers, activation functions can be implemented as modules. To compute outputs, these modules will be using forward() methods and to compute gradients, backward() methods are used.

• Composability − Modules can be combined allowing to create complex networks. Tensor provides Sequential module for linear processing of layers and Parallel for parallel processing.

• Flexible − Tensor uses Dynamic Computational Graph technique which means a neural network structure can be defined and modified on the fly. This flexibility is very useful with models having variable input sizes.

Optimization

Torch was designed and developed while keeping performance and optimization in consideration. Following are optimization features that Torch provides.

• Optimizer Algorithms − Torch comes up with many optimizer algorithms like Stochastic Gradient Descent(SGD), Adam, RMSprop etc. We can use these optimizers to update the parameters of the models based on computed gradients. This can help in minimizing the loss function.

• Loss Functions − A Loss function determines the performance of a model how well or how worse it is performing. Tensor provides loss functions like Mean Squared Error(MSE) for regression or Cross-Entropy for classification. Their forward() and backward() functions can be used compute the loss and gradients respectively.

Key Features of Torch

Following are some of the salient key features of Torch.

• Designed for Efficiency − Torch is designed to improve performance especially when GPUs are available.

• Modular and Flexible − Torch being modules based, can be used to create different network architectures and algorithm. Its dynamic computational graph capability makes it highly flexible to create and modify algorithms on the fly.

• Easy to use − Torch, especially PyTorch is Python based and utilizes Python econsytem and is quite easy to use. With user friendly syntax, it is very popular among researchers and scientists.

• Actively maintained by Community − Torch, original Lua based community is smaller by PyTorch is having a very large base of active community users and provides a rich soruce of tutorials, pre-trained models etc.

• Extensibility − Torch is open source and its modular nature allows researcher to modify or customize Torch packages as per their custom needs.

Application of Torch

Torch, especially PyTorch are widely used in various machine learning domains.

• Computer Vision − Various domains like Image classification, detection of objects, semantic segmentation, generation of Images.

• Natural Language Processing (NLP) − modeling of Languages, sentiment analysis, language translation, chatbots.

• Speech and Audio Processing − Recognizing Speech, text-to-speech processing.

• Reinforcement Learning − We can train agents to make decisions in complex environments.

Conclusion

Torch provides a powerful and flexible framework to build, train and deploy machine learning models. PyTorch can be considered as modern incarnation of Torch and being Python based, it has increased the userbase to a quite high scale. Torch and PyTorch focuses primarily on deep neural networks and also used in research and rapid prototyping.