How to compute the Hessian of a given scalar function in PyTorch?

PyTorchServer Side ProgrammingProgramming

The hessian() function computes the Hessian of a given function. The hessian() function can be accessed from the torch.autograd.functional module. The function whose Hessian is being computed takes a tensor as the input and returns a tuple of tensors or a tensor. The hessian() function returns a tensor with the Hessian values computed for a function with the given input.

Syntax

torch.autograd.functional.hessian(func, input)

Parameters

  • func − It's a Python function for which the Hessian is computed.

  • input − It’s input to the function, func.

Steps

We could use the following steps to compute the Hessian of a given function −

  • Import the required library. In all the following examples, the required Python libraries are torch. Make sure you have already installed it.

import torch
from torch.autograd.functional import hessian
  • Define a function func for which the Hessian is to be calculated. The input to this function is input.

def func(x):
   return x**3 + 4*x -10
  • Define the tensor input to the function func.

input = torch.tensor([2.,3.,4.])
  • Compute the Hessian of the function defined above for the given input input.

output = hessian(func, input)
  • Print the tensor containing the computed hessian.

print("Hessian Tensor:\n", output)

Example 1

# Import the required libraries
import torch
from torch.autograd.functional import hessian

# define a function
def func(x):
   return x**3 + 4*x -10
input = torch.tensor([2.])
output = hessian(func, input)
print("Hessian Tensor:\n",output)

Output

Hessian Tensor:
   tensor([[12.]])

In the above example, we have computed the Hessian of a function for a given input. The function here we have used is a univariate function.

Example 2

# Import the required libraries
import torch
from torch.autograd.functional import hessian
def func(x):
   return (x**3 + 4*x -10).sum()

# apply an input having more than one elements
input = torch.randn(2,2)
output = hessian(func, input)
print(output)

Output

tensor([[[[-1.4218, -0.0000],
   [ 0.0000, -0.0000]],

   [[-0.0000, -2.7878],
   [ 0.0000, -0.0000]]],

   [[[-0.0000, -0.0000],
   [ 1.9817, -0.0000]],

   [[-0.0000, -0.0000],
   [ 0.0000, -2.8517]]]])

In the above example, we have computed the Hessian of a function for a given 2×2 input tensor. The function here we have used is a univariate function.

Example 3

# Import the required libraries
import torch
from torch.autograd.functional import hessian

# define a function
def func(x, y):
   return x**3 + x*y
input = (torch.tensor([2.]),torch.tensor([4.]))
output = hessian(func, input)
print(output)

# to apply multi element tensor as input
def func(x, y):
   return (x**3 + x*y).sum()
input = (torch.tensor([2.,3.]),torch.tensor([4., 5.]))
output = hessian(func, input)
print(output)

Output

((tensor([[12.]]), tensor([[1.]])), (tensor([[1.]]),
   tensor([[0.]])))
((tensor([[12., 0.],
   [ 0., 18.]]), tensor([[1., 0.],
   [0., 1.]])), (tensor([[1., 0.],
   [0., 1.]]), tensor([[0., 0.],
   [0., 0.]])))

In the above example, we have computed the Hessian of a function for a given input tensor. The function here we have used is a bivariate function. So, we have defined the input as a tuple of two tensors.

raja
Updated on 27-Jan-2022 05:55:46

Advertisements