Memory requirements of lobpcg: Matlab and hypre implementation differences

Hi,

until recently i used lobpcg.m from http://www.mathworks.com/matlabcentral/fileexchange/48-lobpcg-m to solve an eigenvalue problem A x = lambda*x. A is large, symmetric, complex and sparse. To compute several eigenpairs (lambda, x) inside the spectrum i use the folded spectrum method: folding the original matrix around the desired value AA = ( A -value)^2, solve the resulting eigenproblem for AA and backtransform the eigenvalues.

This worked fine until i added some small modifications to the off-diagonal entries of the original matrix A. These modifications resulted in very poor convergence.

To solve this problem i call lobpcg via mpirun in the ij-interface of hypre, where i can use the BoomerAMG preconditioner. With this the convergence is okay again, but:

Compared to the lobpcg.m-version, the memory requirements using hypre are several times higher (lets say by a factor of 4 or 5 approximately). In both cases i use the same number of processors. Of course, with mpirun several small parts of the matrix to diagonalize are additionally stored locally, but this is not the origin of the high memory requirements.

Maybe somebody can help me with this.

Thanks

Answer 1

[Andrew Knyazev]

The lobpcg code in hypre is very similar to the MATLAB code, and uses about the same memory. The BoomerAMG preconditioner requires extra memory. You can check this by tracing the memory use while running the code and looking at the screen output. BoomerAMG preconditioner is constructed first. The lobpcg starts when you see the actual screen output from the iterations.

You can alternatively check this by running the lobpcg in hypre without the BoomerAMG preconditioner - these runs should be very similar to MATLAB runs in all respects.

You may be able to reduce the memory use in BoomerAMG preconditioner without slowing it down too much, by using some non-default options. The BoomerAMG preconditioner is a part of hypre, which is not related to , although can be used by, lobpcg. Please contact hypre technical support if you have any questions concerning the BoomerAMG preconditioner.

One thing surprises me in your post: "A is complex" - to my recollection, hypre currently does not support complex arithmetic.

Answer 2

[Elias]

Thanks for the quick answer. I can confirm that the high memory requirement stems from the preconditioner, so i will concentrate on this part.

A being complex apparently does not make any problems in hypre. The documentation only says:

"The hypre library currently does not directly support complex-valued systems.",

whatever that means. At least in the hypre SStruct interface there is a function for declaring a matrix to be complex.

Answer 3

[Andrew Knyazev]

Please try adding the following options to the ij driver: "-hmis -agg_nl 1 -interptype 6 -Pmx 4" These are proposed by a member of the hypre team. They make the preconditioner a bit worse, but, at the same time, not so memory-hungry.

I am still uneasy about your use of a complex matrix in the ij driver. You may want to send a email to hypre support to ask about it. If you do, please let me know their answer.

Since you actually use BLOPEX in hypre, rather then MATLAB, you may want to post your further comments at our BLOPEX site at http://code.google.com/p/blopex/issues/list

Answer 4

[Elias]

Hey, thank you for still worrying about my problem.

I investigated the situation about complex matrices and found the trivial solution: hypre doesnt care about my complex matrix because the complex parts are simply cut off by using matlab2hypreIJ.m for creating the ij-formatted input files from the matlab matrix. Maybe there should be a warning about it.

Since i found out about this i am no longer using hypre, but SLEPc instead for the diagonalization of my complex matrix. As it turned out, the default method there (krylov-schur) is the fastest and less memory-conserving way for the diagonalization, without any need for a preconditioner.

Anyway, out of curiosity i would like to investigate about the ij-options you provided. What are they doing exactly? An interpolation? Sadly hypre isnt that well documented, so its hard to find out about options like these. In a free minute i will try the options and report about the outcome.

Regards

Answer 5

[Andrew Knyazev]

The development version of SLEPc incorporates BLOPEX now, see http://code.google.com/p/blopex/wiki/PETScInstallDevel so that you can easily give it a try (without preconditioning). If you do, please let me know how it runs. We have not yet tested it for complex matrices.

Answer 6

[Elias]

Yes i've seen that and tried using blopex as a solver. It gives an error message for using complex input matrices but works well for real matrices.

Answer 7

[Andrew Knyazev]

Have you actually tried the development version of SLEPc? The one that one gets by

svn checkout http://www.grycap.upv.es/slepc/svn/trunk slepc-dev

If so, could you please be very specific and report the complete setup and the error message?