How to estimate the gradient of a function in one or more dimensions in PyTorch?

To estimate the gradient of a function, we can apply the torch.gradient() function. This function estimates the gradient using the second-order accurate central differences method. We can estimate the gradient in one or more dimensions. The function of which the gradient is to be estimated may be defined on a real or complex domain. In the process of estimating the gradients, the gradient is estimated by estimating each partial derivative of the function independently.

Syntax

torch.gradient(values)

where the parameter values is the tensor that represents the values of the function.

Steps

We could use the following steps to estimate the gradient of a function −

• Import the required library. In all the following examples, the required Python library is torch. Make sure you have already installed it.

import torch

• Define the function f and the points x.

x = torch.tensor([-1., -2., 3., 4.])
def f(x):
   return x**3

• Compute the values of the above defined function f for the given point x.

values = f(x)

• Now estimate the gradient of the function using torch.gradient(values). Here values is a tensor computed above as the values of the function f for the given point x.

grad = torch.gradient(values)

• Print the tensor containing estimated gradients.

print("Estimated Gradients:
", grad)

Now let's take a couple of examples to demonstrate how to estimate the gradient of a function.

Example 1

# Python program to estimate the gradient of
# f(x)=x^3 at points [-2, -1, 2, 4]

# Import the required library
import torch

# define the points
x = torch.tensor([-1., -2., 3., 4.])
print("Points
", x)

# define the function
def f(x):
   return x**3
# values of the function
values = f(x)
print("Function Value:
", values)

# estimate the gradients of the above function
grad = torch.gradient(values)

# print the gradients above estimated
print("Estimated Gradients:
", grad)

Output

Points
   tensor([-1., -2., 3., 4.])
Function Value:
   tensor([-1., -8., 27., 64.])
Estimated Gradients:
   (tensor([-7., 14., 36., 37.]),)

In the above example, we have estimated the gradient of the function f(x)=x^3 at points [-2, -1, 2, 4].

Example 2

# Python 3 program to estimates the gradient of f(x)=x^2+3
# Import the required library
import torch

# define the points
x = torch.randn(2,2)
print("Points
", x)

# define the function
def f(x):
   return x**2+3

# values of the function
values = f(x)
print("Function Value:
", values)

# estimate the gradients of the above function
grad = torch.gradient(values)

# print the gradients above estimated
print("Estimated Gradients:
", grad)

# estimate the gradients of the above function in dim 0
grad_dim0 = torch.gradient(values, dim=0)

# print the gradients above estimated
print("Estimated Gradients in dim 0:
", grad_dim0)

# estimate the gradients of the above function in dim 1
grad_dim1 = torch.gradient(values, dim=1)

# print the gradients above estimated
print("Estimated Gradients in dim 1:
", grad_dim1)

Output

Points
   tensor([[-1.7004, 1.5121],
      [-0.5974, -1.2117]])
Function Value:
   tensor([[5.8914, 5.2864],
      [3.3569, 4.4682]])
Estimated Gradients:
   (tensor([[-2.5345, -0.8182],
      [-2.5345, -0.8182]]), tensor([[-0.6050, -0.6050],
      [ 1.1113, 1.1113]]))
Estimated Gradients in dim 0:
   (tensor([[-2.5345, -0.8182],
      [-2.5345, -0.8182]]),)
Estimated Gradients in dim 1:
   (tensor([[-0.6050, -0.6050],
      [ 1.1113, 1.1113]]),)

In the above example, we have estimated the gradient of the function f(x)=x^2+3 at some random points in different dimensions.