^{1}

The authors have declared that no competing interests exist.

Performed the experiments: NK. Analyzed the data: NK. Contributed reagents/materials/analysis tools: NK. Wrote the paper: NK.

The present paper introduces a condition number estimation method for preconditioned matrices. The newly developed method provides reasonable results, while the conventional method which is based on the Lanczos connection gives meaningless results. The Lanczos connection based method provides the condition numbers of coefficient matrices of systems of linear equations with information obtained through the preconditioned conjugate gradient method. Estimating the condition number of preconditioned matrices is sometimes important when describing the effectiveness of new preconditionerers or selecting adequate preconditioners. Operating a preconditioner on a coefficient matrix is the simplest method of estimation. However, this is not possible for large-scale computing, especially if computation is performed on distributed memory parallel computers. This is because, the preconditioned matrices become dense, even if the original matrices are sparse. Although the Lanczos connection method can be used to calculate the condition number of preconditioned matrices, it is not considered to be applicable to large-scale problems because of its weakness with respect to numerical errors. Therefore, we have developed a robust and parallelizable method based on Hager’s method. The feasibility studies are curried out for the diagonal scaling preconditioner and the SSOR preconditioner with a diagonal matrix, a tri-daigonal matrix and Pei’s matrix. As a result, the Lanczos connection method contains around 10% error in the results even with a simple problem. On the other hand, the new method contains negligible errors. In addition, the newly developed method returns reasonable solutions when the Lanczos connection method fails with Pei’s matrix, and matrices generated with the finite element method.

Solving the linear equation is considered to be the most time consuming component of simulation computation. As a result, a number of linear equation solvers have been developed in order to realize more efficient simulation. Linear equation solvers can be roughly categorized into two types: direct solvers and iterative solvers. Direct solvers have been used for ill conditioned problems, because of the robustness of the solution. However, the iterative method has become mainstream in recent years for the following reasons:

The iterative method requires less memory than the direct method, if the coefficient matrix of the equation is sparse. Most simulation methods, as is the case with the finite element method and the finite difference method, use sparse matrices.

The iterative method is more suitable for distributed memory type parallel computers than the direct method. Recently, the trend in super computers has been toward distributed memory computers.

Evaluating the convergence rate of an iterative solver is sometimes important in order to demonstrate the effectiveness of new methods or to select an adequate method. The convergence rate of the conjugate gradient method (CG) and other Krylov sub-space type solvers strongly depends on the eigenvalue distribution of coefficient matrix, that is to say, the convergence rate is better if the eigenvalues are concentrated. Complete knowledge of the eigenvalue distribution enables the complete prediction of the convergence behavior of the PCG. However, obtaining all of the eigenvalues is difficult, and so another simple indicator, called condition number, is used. Generally, the condition number is easy to calculate when the coefficient matrix is explicitly obtained. As described above, parallel computers, especially distributed memory type parallel computers, are becoming increasingly important for engineers or physicists. Obtaining preconditioned matrices using a parallel computer is extremely difficult and an evaluation method is required. One way to evaluate preconditioned matrices is by the Lanczos connection method [

In the present paper, we introduce a new evaluation method of preconditioned matrices based on Hager’s method, and verify the robustness of the new method by comparing their precisions.

When using iterative solvers, an acceleration method called preconditioning is performed:

Let the linear equation be written as

1: | |

2: | Let _{0} be the initial approximation |

3: | _{0} = _{0} |

4: | |

5: | solve _{i} = _{i} |

6: | _{i} = ⟨_{i}, _{i}⟩ |

7: | |

8: | _{i+1} = _{i} |

9: | |

10: | |

11: | _{i+1} = _{i} + _{i}_{i} |

12: | |

13: | _{i+1} = _{i+1} |

14: | |

15: | _{i+1} = _{i} + _{i}_{i+1} |

16: | _{i+1} = _{i} + _{i}_{i+1} |

17: | |

18: | return _{ilast}, where _{last} is the last iteration |

19: |

Pseudocode of the PCG. In the algorithm,⟨⋅, ⋅⟩ denotes vector inner product.

When the preconditioning matrix

Assuming that the coefficient matrix is symmetric and is decomposed as

The convergence rate of the CG method depends on the distribution of eigenvalues of a coefficient matrix

The Lanczos connection method is a method of solving the eigensystem and is strongly related to the CG method [_{k} as
_{i}, (

Next, we define _{k} as follows:
_{k} are the scalars shown in the PCG algorithm. Using the matrix _{k} = [_{0}, _{1}, …, _{k−1}],
_{i}, (_{1}, _{2}, …, _{k} are _{k} is a matrix having components that are given by
_{k},
_{k} is a _{i}‖_{2}, (_{k} are the Lanczos vectors, for which the associated projections of _{k} approximate those of matrix

Hager’s method gives the 1-norm of ^{−1} [^{T} =

1: | |

2: | Set |

3: | |

4: | Solve |

5: | _{1} ≤ |

6: | return ^{−1}‖_{1}, and exit |

7: | |

8: | _{1} |

9: | |

10: | |

11: | Solve ^{T} |

12: | _{max} = |

13: | _{max}]∣ < ^{T} |

14: | return ^{−1}‖_{1}, and exit |

15: | |

16: | |

17: | |

18: | |

19: |

Pseudocode of Hager’s method. In the algorithm, [

Here, we develop a new condition number estimation method which overcomes the downside of the Lanczos connection method. That is to say, new method must have such features:

Applicable for large system. Especially, it must work on distributed memory computers.

Robust for ill conditioned system.

As introduced in the previous section, we can obtain the 1-norm condition number using Hager’s method. There is a simple method of extending Hager’s method to preconditioned matrices without explicit matrix production. That is, in Hager’s method, we simply replace matrix

1: | |

2: | _{inv} = call procedure calc 1-norm of inverse preconditioned matrix ( |

3: | _{forward} = call procedure calc 1-norm of preconditioned matrix( |

4: | return _{inv} ⋅ _{forward} as the condition number of |

5: | |

6: | |

7: | Set _{inv} = 0 |

8: | |

9: | |

10: | _{1} ≤ _{inv} |

11: | return _{inv} as |

12: | |

13: | _{inv} = ‖_{1} |

14: | |

15: | |

16: | |

17: | _{max} = |

18: | _{max}]∣ < ^{T} |

19: | return _{inv} as |

20: | |

21: | |

22: | |

23: | |

24: | |

25: | |

26: | Set _{forwad} = 0 |

27: | |

28: | |

29: | _{1} ≤ _{forward} |

30: | return _{forward} as |

31: | |

32: | _{forward} = ‖_{1} |

33: | |

34: | |

35: | |

36: | _{max} = |

37: | _{max}]∣ < ^{T} |

38: | return _{forward} as |

39: | |

40: | |

41: | |

42: | |

43: |

Pseudocode of the modified Hager’s method. The modified Hager’s method consists of two major parts: one is for calculating

1: | |

2: | Let _{0} be the initial approximation |

3: | |

4: | |

5: | _{i} = ⟨_{i}, _{i}⟩ |

6: | |

7: | _{i+1} = _{i} |

8: | |

9: | |

10: | _{i+1} = _{i} + _{i}_{i} |

11: | |

12: | |

13: | |

14: | _{i+1} = _{i} + _{i}_{i+1} |

15: | _{i+1} = _{i} + _{i}_{i+1} |

16: | |

17: | return _{ilast}, where _{last} is the last iteration |

18: |

Pseudocode of the PCG in the modified Hager’s method. In the algorithm,⟨⋅, ⋅⟩ denotes vector inner product.

Since the Hager’s method gives us only

These developed algorithms for preconditioned matrices always converge if coefficient matrices and preconditioning matrices are positive definite. This is because, Hager’s method always converges for positive definite matrices [

In this section, we carry out the feasibility test of our new method by comparing the performance of both our method and the Lanczos connection method. In the following sections, the computational environment, and the matrices which we employ will be explained, and the performances are also described.

Here, we describe the computational environment in order to determine the precision of the numerical calculation. All of the code was written with Fortran90, and the eigenvalues of the tridiagonal matrix obtained from the Lanczos connection was calculated using the Intel Math Kernel library. The machine architecture and compiler environment are listed in

Architecture | Intel Pentium4 2.8 GHz |

Operating System | Vine linux 3.1 |

Compiler | Intel Fortran Compiler 8.1 |

Compiler options | -O3 |

Math library | Intel Math Kernel library |

Float number precision | real(8) |

Details of the computational environment used for the precision verification.

In order to verify the precision of both of these methods, we first employ a simple diagonal matrix for a test problem. The test matrix _{diag} is given by
_{diag} are 1.0.

The

We employ Pei’s matrix [_{Pei}, which becomes increasingly ill conditioned as _{Pei} is singular.

The above sample matrices were artificially created and have simple structures. In this section, we will introduce matrices that are created with the finite element method (FEM) in order to investigate the feasibility of both methods with realistic problems. The equation which employed was Poission’s equation. The analysis domain was a cube with edge lengths of 1.0. In the present study, three types of finite element meshes were prepared and each of three types had a different degree of freedom (DOF) on

First we discuss the feasibility of the Lanczos connection. The eigenvalues of _{diag} calculated with the Lanczos connection is listed in the first row in

Ideal | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|

Lanczos connection | 1.00 | 2.02 | 3.04 | 4.08 | 5.13 | 6.17 | 7.28 | 8.37 | 9.51 | 3.0×10^{6} |

Without 1st step | 1.00 | 2.02 | 3.04 | 4.08 | 5.13 | 6.20 | 7.28 | 8.37 | 9.51 | NA |

In the table, the eigenvalues of a 10 × 10 diagonal matrix calculated using the Lanczos connection method are listed. The first row indicates the eigenvalues with all information, and the second row indicates the eigenvalues without the first CG step information. In the full information case, the condition number becomes unreasonable, while the second one shows a reasonable result. In the table, NA means “Not Available”.

Next, we check the feasibility of both the Lanczos connection and Hager’s method with _{diag}. The condition numbers calculated using both methids are listed in

Matrix Size | 10 | 100 | 200 | 500 |
---|---|---|---|---|

Lanczos connection | 9.49 | 99.74 | 199.78 | 499.81 |

Octave (2-norm cond. num.) | 10 | 100 | 200 | 500 |

Hager’s method | 10.00 | 100.00 | 200.00 | 500.00 |

Octave (1-norm cond. num.) | 10 | 100 | 200 | 500 |

In the table, condition numbers calculated using both the Lanczos connection method and Hager’s method are listed with numbers with Octave. Both method show good performances, but the Lanczos method includes some error, although the test matrix is simple.

The condition numbers of _{Tri} are listed in

Matrix Size | 10 | 100 | 200 | 500 |
---|---|---|---|---|

Lanczos connection | 41.8 | 4.13 × 10^{3} |
1.64 × 10^{4} |
1.02 × 10^{5} |

Octave (2-norm cond. num.) | 48.0 | 4.13 × 10^{3} |
1.64 × 10^{4} |
1.02 × 10^{5} |

Hager’s method | 60.0 | 5.10 × 10^{3} |
2.02 × 10^{6} |
1.26 × 10^{6} |

Octave (1-norm cond. num.) | 60.0 | 5.10 × 10^{3} |
2.02 × 10^{6} |
1.26 × 10^{6} |

The condition numbers of _{Tri} using the Lanczos connection method, the modified Hager’s method, and Octave are listed.

Matrix Size | 10 | 100 | 200 | 500 |
---|---|---|---|---|

Lanczos connection | 41.8 | 4.13 × 10^{3} |
1.64 × 10^{4} |
1.02 × 10^{5} |

Octave (2-norm cond. num.) | 48.0 | 4.13 × 10^{3} |
1.64 × 10^{4} |
1.02 × 10^{5} |

Modified Hager’s method | 60.0 | 5.10 × 10^{3} |
2.02 × 10^{6} |
1.26 × 10^{6} |

Octave (1-norm cond. num.) | 60.0 | 5.10 × 10^{3} |
2.02 × 10^{6} |
1.26 × 10^{6} |

The condition numbers of diagonal scaled _{Tri} using the Lanczos connection method, the modified Hager’s method, and Octave are listed.

In the present paper, we fixed the matrix size as 100 × 100 and set

Lanczos | Octave | Error(%) | Modified Hager | Octave | Error (%) | |
---|---|---|---|---|---|---|

64,851 | 1,365.6 | 4,648.9 | 1,684.08 | 1,684.1 | 0.0012 | |

178,699 | 3,259.8 | 5,387.0 | 4,020.75 | 4,020.8 | 0.0012 | |

338,463 | 7,224.7 | 4,584.8 | 8,911.86 | 8,911.9 | 0.0004 |

The condition numbers of SSOR preconditioned _{Pei} using the Lanczos connection method, the modified Hager’s method, and Octave are listed. In this table, error values defined as

The condition numbers of preconditioned matrices of Poisson’s equation are listed in ^{27}% error. Reasonable results can be obtained by removing the last CG step information from the Lanczos connection procedure. The condition number moves from 2.35 × 10^{28} to 6.94 × 10^{2}. However, such procedures can only be done when the true result is already known, and therefore, the Lanczos connection cannot be used with ease. On the other hand, using the modified Hager’s method gives the results with an error rate under 6.46%.

Lanczos | Octave | Error(%) | Modified Hager | Octave | Error(%) | |
---|---|---|---|---|---|---|

20 × 20 × 20 | 2.35 × 10^{28} |
7.75 × 10^{2} |
3.03 × 10^{27} |
2.98 × 10^{2} |
2.98 × 10^{2} |
0.00 |

10 × 10 × 80 | 5.82 × 10^{4} |
2.79 × 10^{3} |
1.98 × 10^{3} |
3.63 × 10^{3} |
3.87 × 10^{3} |
6.43 |

5 × 5 × 320 | 5.98 × 10^{5} |
4.73 × 10^{4} |
1.16 × 10^{3} |
6.16 × 10^{4} |
6.59 × 10^{4} |
6.46 |

The condition numbers of the diagonal scaled Poisson–FEM matrices using the Lanczos connection method, the modified Hager’s method, and Octave are listed. In this table, error values defined as

Lanczos | Octave | Error(%) | Modified Hager | Octave | Error(%) | |
---|---|---|---|---|---|---|

20 × 20 × 20 | 4.12 × 10^{2} |
8.22 × 10^{2} |
49.84 | 1.05 × 10^{3} |
1.07 × 10^{3} |
2.37 |

10 × 10 × 80 | 4.82 × 10^{2} |
2.47 × 10^{3} |
80.49 | 8.48 × 10^{3} |
8.53 × 10^{3} |
0.64 |

5 × 5 × 320 | 1.22 × 10^{5} |
7.61 × 10^{5} |
84.03 | 1.50 × 10^{6} |
1.50 × 10^{6} |
0.00 |

The condition numbers of SSOR preconditioned Poisson–FEM matrices using the Lanczos connection method, the modified Hager’s method, and Octave are listed. In this table, error values defined as

In SSOR preconditioned cases, the Lanczos connection gives more accurate results than diagonal scaled cases, however, the results still include over 50% errors. In addition, when the condition number of the original matrix becomes large, the error becomes more extreme. Estimations may become unreasonable in cases where problems are ill-conditioned. The modified Hager’s method gives results with under 3% errors. The results in this section show that the modified Hager’s method can be used to estimate the condition numbers of practical problems. Finally, for reference, the condition numbers of each problem based on each norm without preconditioning are indicated below: 1-norm 20 × 20 × 20: 1.02 × 10^{3}, 10 × 10 × 80: 3.73 × 10^{3}, and 5 × 5 × 320: 7.22 × 10^{4}, 2-norm 20 × 20 × 20: 7.75 × 10^{2}, 10 × 10 × 80: 2.79 × 10^{3}, and 5 × 5 × 320: 4.73 × 10^{4}. The condition numbers without preconditioning were calculated with Octave. Sample programs of both the Lanczos connection and the modified Hager’s method will be uploaded on the Zenodo site for readers’ convenience [

In this section, the research examines how to implement the PCG, and the modified Hager’s method. First, the algorithm of the PCG (

Hager’s method, and therefore the modified Hager’s method as well, can be implemented with the memory allocation techniques explained above. There are several additional operations to PCG (e.g. lines 10, 12, and 16 in

Finally, the parallel version of the modified Hager’s method was implemented with the above technique. In addition, it was confirmed that the same results as the sequential version were obtained with the diagonal scaling preconditioning.

Here, the research discusses the computational effort and the memory usage of the Lanczos connection, the modified Hager’s method, and the naive implementation that computes preconditioned matrices explicitely. First, the computational effort, and the memory usage of the naive implementation are proportional to ^{3}, and ^{2} respectively, where

Second, the computational effort and the memory usage of the Lanczos connection are proportional to

Finally, we consider the modified Hager’s method. The memory space requirement of the modified Hager’s method is almost the same as the PCG, but additional four vectors must be stored. Experiments in this study indicate that Hager’s method converges within four iterations. In addition, Higham also reported that Hager’s method converges within four iterations [

In present paper, we developed a condition number estimation method for the preconditioned matrix and verified the accuracy of the newly developed method. Although the Lanczos connection method can be considered as a method by which to estimate the condition number of the preconditioned matrix, this method is weak with respect to numerical errors. Therefore, we compared the condition numbers estimated using both the Lanczos connection method and the newly developed method with Octave’s result. The newly developed method provided better results, whereas the Lanczos connection method failed. In addition, the newly developed method can be applied without difficulty for parallel computing and large-scale problem because this new method does not require explicit matrix operation.

I would like to thank the CTBTO for giving me the opportunity to publish the article. However, the views expressed are those of the author and do not necessarily reflect the view of CTBTO Preparatory Commission.