Arnoldi decomposition, GMRES, and preconditioning for linear discrete ill-posed problems

GMRES is one of the most popular iterative methods for the solution of largelinear systems of equations that arise from the discretization of linear well-posed problems, such asboundary value problems for elliptic partial differential equations. The method is also applied tothe iterative solution of linear systems of equations that are obtained by discretizing linear ill-posedproblems, such as many inverse problems. However, GMRES does not always perform well whenapplied to the latter kind of problems. This paper seeks to shed some light on reasons for the poorperformance of GMRES in certain situations, and discusses some remedies based on specific kindsof preconditioning. The standard implementation of GMRES is based on the Arnoldi process, whichalso can be used to define a solution subspace for Tikhonov or TSVD regularization, giving rise tothe Arnoldi–Tikhonov and Arnoldi-TSVD methods, respectively. The performance of the GMRES,the Arnoldi–Tikhonov, and the Arnoldi-TSVD methods is discussed. Numerical examples illustrateproperties of these methods.