SciPy optimize.leastsq() Function

scipy.optimize.leastsq() is a function in SciPy's optimization module that is used for solving non-linear least squares problems. It aims to minimize the sum of squared residuals between observed data and a model's predictions. This function is commonly used in curve fitting and parameter estimation problems where the goal is to find the model parameters that best fit a given set of data points.

By using this function the olusers can define a residual function representing the difference between observed data and the model and then find the optimal parameters that minimize this difference. This function is versatile and supports both unconstrained and constrained optimization, offering flexibility for a variety of fitting problems.

Syntax

Following is the syntax of the function scipy.optimize.leastsq() which is used to solve non-linear least squares problems −

scipy.optimize.leastsq(func, x0, args=(), full_output=False, col_deriv=0, atol=0.0001, xtol=0.0001, gtol=0.0001, maxfev=10000, factor=100, diag=None)

Parameters

Following are the parameters of the scipy.optimize.leastsq() function −

• func: The residual function that takes a set of parameters and returns the difference between observed and predicted values.
• x0: The initial guess for the parameters of the model.
• args (optional): Additional arguments passed to the residual function.
• full_output: If True then the function returns additional information such as the success flag and the Jacobian matrix.
• col_deriv (optional): If set to 1 then it indicates that the residual function returns the derivatives of the residuals.
• atol, xtol, gtol: Tolerance parameters that control the convergence criteria for the optimization process.
• maxfev: The maximum number of function evaluations allowed.
• factor (optional): A scaling factor that is used to calculate the step size in the optimization process.
• diag (optional): Diagonal elements of the covariance matrix for scaling parameters.

Return Value

The scipy.optimize.leastsq() function returns a tuple containing the following −

• x: The optimal values of the model parameters.
• cov_x: The covariance of the optimal parameter estimates (if full_output=True).
• infodict: A dictionary containing additional information about the optimization process (if full_output=True).
• msg: A message indicating the termination status of the optimization (if full_output=True).

Fitting a Polynomial Curve

In this example we use scipy.optimize.leastsq() to fit a quadratic curve to a set of noisy data points. We define a residual function that represents the difference between the model and observed data points and then use leastsq() function to estimate the parameters of the quadratic model −

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import leastsq

# Define the true model and add noise
def model(x, a, b, c):
    return a*x**2 + b*x + c

# Generate noisy data
x_data = np.linspace(-5, 5, 100)
y_true = model(x_data, 2, -3, 5)
y_data = y_true + np.random.normal(0, 1, size=x_data.shape)

# Residual function
def residuals(params, x, y):
    return model(x, *params) - y

# Initial guess for parameters
initial_guess = [1, -1, 1]

# Perform the least squares optimization
params_opt, _ = leastsq(residuals, initial_guess, args=(x_data, y_data))

# Plot the results
plt.scatter(x_data, y_data, label='Noisy Data', color='r')
plt.plot(x_data, model(x_data, *params_opt), label='Fitted Model', color='b')
plt.legend()
plt.show()

print("Optimal parameters:", params_opt)

Here is the output of the scipy.optimize.leastsq() function used to fit a quadratic curve to noisy data −

Optimal parameters: [ 1.9922913  -3.0012294   4.98916841]

Fitting an Exponential Decay Curve

In this example we use scipy.optimize.leastsq() to fit an exponential decay model to a set of data points in which we define a residual function that represents the difference between the observed data and the predicted values from the exponential decay model. The goal is to find the parameters a, b and c that minimize these residuals −

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import leastsq

# Define the model function
def model(x, a, b, c):
    return a * np.exp(-b * x) + c

# Generate noisy data
x_data = np.linspace(0, 5, 50)
y_true = model(x_data, 2, 1.5, 0.5)
y_data = y_true + np.random.normal(0, 0.2, size=x_data.shape)

# Residual function
def residuals(params, x, y):
    return model(x, *params) - y

# Initial guess for parameters
initial_guess = [1, 1, 1]

# Perform the least squares optimization
params_opt, _ = leastsq(residuals, initial_guess, args=(x_data, y_data))

# Plot the results
plt.scatter(x_data, y_data, label='Noisy Data', color='r')
plt.plot(x_data, model(x_data, *params_opt), label='Fitted Model', color='b')
plt.legend()
plt.show()

print("Optimal parameters:", params_opt)

Here is the output of the scipy.optimize.leastsq() function used to fit an exponential decay model to the noisy data −

Sinusoidal Curve Fitting

In this example we use scipy.optimize.leastsq() to fit a sinusoidal model to a dataset, here we generate synthetic data with some noise and estimate the parameters a, b and c that minimize the difference between the observed data and the model predictions.

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import leastsq

# Define the sinusoidal model
def model(x, a, b, c):
    return a * np.sin(b * x + c)

# Generate noisy data
x_data = np.linspace(0, 2 * np.pi, 100)
y_true = model(x_data, 2, 1.5, 0.5)
y_data = y_true + np.random.normal(0, 0.2, size=x_data.shape)

# Residual function
def residuals(params, x, y):
    return model(x, *params) - y

# Initial guess for parameters
initial_guess = [1, 1, 0]

# Perform the least squares optimization
params_opt, _ = leastsq(residuals, initial_guess, args=(x_data, y_data))

# Plot the results
plt.scatter(x_data, y_data, label='Noisy Data', color='r', s=10)
plt.plot(x_data, model(x_data, *params_opt), label='Fitted Model', color='b')
plt.legend()
plt.show()

print("Optimal parameters:", params_opt)

Here is the output of the scipy.optimize.leastsq() function used for sinusoidal curve fitting: