The Symmetric Rank 1 (SR1) method is a quasi-Newton method to update the second derivative (Hessian) based on the derivatives (gradients) calculated at two points. It is a generalization to the secant method for a multidimensional problem. This update maintains the symmetry of the matrix but does not guarantee the update to be a positive definite matrix. For this reason it is the method of choice for indefinite problems.

PropertyValue
dbpprop:abstract
  • The Symmetric Rank 1 (SR1) method is a quasi-Newton method to update the second derivative (Hessian) based on the derivatives (gradients) calculated at two points. It is a generalization to the secant method for a multidimensional problem. This update maintains the symmetry of the matrix but does not guarantee the update to be a positive definite matrix. For this reason it is the method of choice for indefinite problems. Given a function <math>f(x)</math>, its gradient (<math>\nabla f</math>), and Hessian matrix <math>B</math>, the Taylor series is: <math>f(x_0+\Delta x)=f(x_0)+\nabla f(x_0)^T \Delta x+\frac{1}{2} \Delta x^T {B} \Delta x </math>, and the Taylor series of the gradient itself: <math>\nabla f(x_0+\Delta x)=\nabla f(x_0)+B \Delta x</math>, is used to update <math>B</math>. Equation above (secant equation) can admit an infinite number of solutions to <math>B</math>. The SR1 formula finds a solution of rank 1 that is symmetric and closest to the current approximate value of <math>B_k</math>: <math>B_{k+1}=B_{k}+\frac {(y_k-B_k \Delta x_k) (y_k-B_k \Delta x_k)^T}{(y_k-B_k \Delta x_k)^T \Delta x_k}</math>, where <math>y=\nabla f(x_k+\Delta x_k)-\nabla f(x_k)</math>. The corresponding update to the inverse Hessian approximation <math>H_k=B_k^{-1}</math> is given by: <math>H_{k+1}=H_{k}+\frac {(\Delta x_k-H_k y_k)(\Delta x_k-H_k y_k)^T}{(\Delta x_k-H_k y_k)^T y_k}</math>. The derivation is simple, and the SR1 formula has been rediscovered a number of times. The main drawback is that the denominator can vanish. It is therefore a good idea to apply the SR1 update only if <math>|\Delta x_k^T (y_k-B_k \Delta x_k)|\geq r \|\Delta x_k\|\cdot \|y_k-B_k \Delta x_k\| </math>, where <math>r\in(0,1)</math> is a small number, e.g. <math>10^{-8}</math>.
dbpprop:hasPhotoCollection
rdf:type
rdfs:comment
  • The Symmetric Rank 1 (SR1) method is a quasi-Newton method to update the second derivative (Hessian) based on the derivatives (gradients) calculated at two points. It is a generalization to the secant method for a multidimensional problem. This update maintains the symmetry of the matrix but does not guarantee the update to be a positive definite matrix. For this reason it is the method of choice for indefinite problems.
rdfs:label
  • SR1 formula
owl:sameAs
skos:subject
foaf:page
is dbpprop:disambiguates of
is owl:sameAs of