Algorithms

The Levenberg-Marquardt is one of the most widely used non-linear curve fitting methods. It is very useful for finding solutions to complex fitting problems. Its utility is the way in which the fit method moves smoothly between the two extremes mentioned above, the steepest descent and the Hessian, for finding the next step size. The algorithm uses the method of steepest descent to determine the step size when the results are far from the minimum:

But as the solution approaches the minimum, the algorithm switches to the Hessian matrix for determining the step size in order to zero in on the best fit.

At this point the method does not know what to use for the constant value (const) in the steepest decent model or when to switch between the methods. When the fit process is initiated, the method of steepest decent should be used. Unfortunately, the choice for the value of the constant presents a problem: whereasis nondimensional, the other fit parameters in a, do have dimensions. In fact these dimensions can be very different from each other.

Marquardt observed that in order to obtain a dimensionless value for the step size, the constant of proportionality between andmust be of the order. Upon examining the equation, he found that the inverse of the diagonal elements , are of the order . Since this value may still be too large for a search step size, a "fudge factor" was invoked, where was >>1. By replacing the constant in theequation from above, the step size was redefined as:

Marquardt then made the realization that the two equations could be combined using the following definitions, forming the new matrix1in terms of:

When used as a fitting routine, Marquardt was thereby able to combine the best of both methods into one simple equation. The equation spans the full range of the fitting processes, from the method of steepest decent to the Hessian matrix method, automatically:

Marquardt was able to develop the equation by realizing the when  was large, an essentially diagonalized matrix 1(the steepest decent portion) resulted, and when approached 0, the equation reverted to the full matrix, (the inverse Hessian).

Since the Levenberg-Marquardt method is an iterative process that requires initial user input, any interpretations from the final results are only as good as the initial inputs. Therefore the use of incorrect function forms can fit the data fine, but using the "fit parameters" for further interpretation would be erroneous. For example, if the actual physical peak shapes (i.e. the fit function) for the system were Lorentzian, and the user chose to use Gaussian instead, using the final fit parameters for other calculations would be inaccurate. The algorithm only optimizes the information fed into it, it can not interpret it for the user.

Back to top