gߣdZddlZddlZddlmZmZmZmZm Z m Z ddl m Z ddl mZdgZdZdZd Zdd Zd Z dd Zd ZdZ ddZy)a Locally Optimal Block Preconditioned Conjugate Gradient Method (LOBPCG). References ---------- .. [1] A. V. Knyazev (2001), Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method. SIAM Journal on Scientific Computing 23, no. 2, pp. 517-541. :doi:`10.1137/S1064827500366124` .. [2] A. V. Knyazev, I. Lashuk, M. E. Argentati, and E. Ovchinnikov (2007), Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in hypre and PETSc. :arxiv:`0705.2626` .. [3] A. V. Knyazev's C and MATLAB implementations: https://github.com/lobpcg/blopex N)inveigh cho_factor cho_solvecholesky LinAlgError)LinearOperator)issparselobpcgc Rddlm}||jjz }||d}dt j |j jz}t||||dz}||kDr4tjd|d|j d|d|d td y y ) zA Report if `M` is not a Hermitian matrix given its type. r)norm zMatrix z of the type z is not Hermitian: condition: z < z fails. stacklevelN) scipy.linalgr Tconjnpfinfodtypeepsmaxwarningswarn UserWarning)Mnamer mdnmdtols [/var/www/html/venv/lib/python3.12/site-packages/scipy/sparse/linalg/_eigen/lobpcg/lobpcg.py_report_nonhermitianr$s" QSSXXZB r1+C rxx $$ $C c3a# $C Sy v]177)45C51a c~|jdk(r|Stj|}|jddf|_|S)zm If the input array is 2D return it, if it is 1D, append a dimension, making it a column vector. rr)ndimrasarrayshape)arauxs r#_as2dr-/s:  ww!| jjnXXa[!$  r%c4ytrfdSfdS)Nc|SNvms r#z_makeMatMat..@s 1r%c|zSr0r1r2s r#r5z_makeMatMat..Bs Qr%)callable)r4s`r# _makeMatMatr8<sy !r%c`|jdrR|jd|jdk(r3|j|jk(rtj||||S tj||||S#t $r'|rt jdtd||z}Y|SwxYw)zPerform 'np.matmul' in-place if possible. If some sufficient conditions for inplace matmul are met, do so. Otherwise try inplace update and fall back to overwrite if that fails. CARRAYr)outz>Inplace update of x = x @ y failed, x needs to be overwritten.r) flagsr*rrmatmul Exceptionrrr)xyverbosityLevels r#_matmul_inplacerCEs  wwxQWWQZ1771:5!''QWW:L !QA H  IIa " H  1A AA H s#A==,B-,B-cf|jj|z}t||}|||zz}y)zChanges blockVectorV in-place.N)rrr) blockVectorVfactYBY blockVectorBY blockVectorYYBVtmps r#_applyConstraintsrKas4 //   < /C GS !CL3&&Lr%c*|M||}nH ||}|j|jk7r&t d|jd|jd |j j|z} t|d }t|d }t||| }|t||| }|||fS#t$r1}|r%tjd|dtdYd}~yYd}~d}~wwxYw#t$r!|rtjd tdYywxYw)z@in-place B-orthonormalize the given block vector using Cholesky.N(Secondary MatMul call failed with error  r<r)NNN The shape z; of the orthogonalized matrix not preserved and changed to , after multiplying by the secondary matrix. T overwrite_arBzCholesky has failed.) r?rrrr* ValueErrorrrrrrCr)BrE blockVectorBVrBeVBVs r#_b_orthonormalizerYhsK 9(M , !, ""l&8&88  !3!3 45&&3&9&9%:;CD ..   - /C s-#4(& #)  =+s-M]C//= ,!MMC#R!# , " ,>   MM&   s)B+.k(rd}8nnkt j\|!j_|;} t j$||; }|*s?t jZ| j:j=3}?t jZ|1j:j=|3}@t jZ2j:j=|3}At jZ| j:j=4}Bt jZ|1j:j=|4}C|,rOA|Aj:j=zdz }At jZ|2j:j=|4}Dnt j$|0|; }Dt j`| |9|?g|9j:j=|:@g|?j:j=|@j:j=Agg}Et j`|<|>Bg|>j:j=|=Cg|Bj:j=|Cj:j=Dgg}Ftc|E|F| t1EFd&4\}!}"|*rt j`| |9g|9j:j=|:gg}Et j`|<|>g|>j:j=|=gg}Ftc|E|F| t1EFd&4\}!}"tG|!||}#|!|#}!|"dd|#f}"| r |!|)dzddf<|C|*s|"d|}G|"|||0z}H|"||0zd}It jZ|1|H}J|Jt jZ2|Iz }Jt jZ|6|H}K|Kt jZ3|Iz }Kt jZ|5|H}L|Lt jZ4|Iz }LnL|"d|}G|"|d}Ht jZ|1|H}Jt jZ|6|H}Kt jZ|5|H}Lt jZ| GJz} t jZ||GKz}t jZ||GLz}|J|K|L}'}&}%n|*st|"d|}G|"|||0z}H|"||0zd}It jZ|1|H}J|Jt jZ2|Iz }Jt jZ|6|H}K|Kt jZ3|Iz }Kn6|"d|}G|"|d}Ht jZ|1|H}Jt jZ|6|H}Kt jZ| GJz} t jZ||GKz}|J|K}&}%|)|kr ~|||!t jRddfz}n| |!t jRddfz}||z }-t jT|-j=|-zd}t j2t jN|}.| r |!|)dzddf<| r |.|)dzddf<t jTt jN|.|z }/|/|(kr |/}(|)dz}| } t jPt jN|.|kDr,tjdC|)dD|.dE|dF|dG|(dH td|rtdI|!tdJ|.| } |tC| ||| }|j| jk7r&t d | jdK|jd3t jZ| j:j=|} | }|G|| }|j| jk7r&t d | jdK|jd,t jZ| j:j=|} n``, it would likely break internally, so the code calls the standard function `eigh` instead. It is not that ``n`` should be large for the LOBPCG to work, but rather the ratio ``n / k`` should be large. It you call LOBPCG with ``k=1`` and ``n=10``, it works though ``n`` is small. The method is intended for extremely large ``n / k``. The convergence speed depends basically on three factors: 1. Quality of the initial approximations `X` to the seeking eigenvectors. Randomly distributed around the origin vectors work well if no better choice is known. 2. Relative separation of the desired eigenvalues from the rest of the eigenvalues. One can vary ``k`` to improve the separation. 3. Proper preconditioning to shrink the spectral spread. For example, a rod vibration test problem (under tests directory) is ill-conditioned for large ``n``, so convergence will be slow, unless efficient preconditioning is used. For this specific problem, a good simple preconditioner function would be a linear solve for `A`, which is easy to code since `A` is tridiagonal. References ---------- .. [1] A. V. Knyazev (2001), Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method. SIAM Journal on Scientific Computing 23, no. 2, pp. 517-541. :doi:`10.1137/S1064827500366124` .. [2] A. V. Knyazev, I. Lashuk, M. E. Argentati, and E. Ovchinnikov (2007), Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in hypre and PETSc. :arxiv:`0705.2626` .. [3] A. V. Knyazev's C and MATLAB implementations: https://github.com/lobpcg/blopex Examples -------- Our first example is minimalistic - find the largest eigenvalue of a diagonal matrix by solving the non-generalized eigenvalue problem ``A x = lambda x`` without constraints or preconditioning. >>> import numpy as np >>> from scipy.sparse import spdiags >>> from scipy.sparse.linalg import LinearOperator, aslinearoperator >>> from scipy.sparse.linalg import lobpcg The square matrix size is >>> n = 100 and its diagonal entries are 1, ..., 100 defined by >>> vals = np.arange(1, n + 1).astype(np.int16) The first mandatory input parameter in this test is the sparse diagonal matrix `A` of the eigenvalue problem ``A x = lambda x`` to solve. >>> A = spdiags(vals, 0, n, n) >>> A = A.astype(np.int16) >>> A.toarray() array([[ 1, 0, 0, ..., 0, 0, 0], [ 0, 2, 0, ..., 0, 0, 0], [ 0, 0, 3, ..., 0, 0, 0], ..., [ 0, 0, 0, ..., 98, 0, 0], [ 0, 0, 0, ..., 0, 99, 0], [ 0, 0, 0, ..., 0, 0, 100]], shape=(100, 100), dtype=int16) The second mandatory input parameter `X` is a 2D array with the row dimension determining the number of requested eigenvalues. `X` is an initial guess for targeted eigenvectors. `X` must have linearly independent columns. If no initial approximations available, randomly oriented vectors commonly work best, e.g., with components normally distributed around zero or uniformly distributed on the interval [-1 1]. Setting the initial approximations to dtype ``np.float32`` forces all iterative values to dtype ``np.float32`` speeding up the run while still allowing accurate eigenvalue computations. >>> k = 1 >>> rng = np.random.default_rng() >>> X = rng.normal(size=(n, k)) >>> X = X.astype(np.float32) >>> eigenvalues, _ = lobpcg(A, X, maxiter=60) >>> eigenvalues array([100.], dtype=float32) `lobpcg` needs only access the matrix product with `A` rather then the matrix itself. Since the matrix `A` is diagonal in this example, one can write a function of the matrix product ``A @ X`` using the diagonal values ``vals`` only, e.g., by element-wise multiplication with broadcasting in the lambda-function >>> A_lambda = lambda X: vals[:, np.newaxis] * X or the regular function >>> def A_matmat(X): ... return vals[:, np.newaxis] * X and use the handle to one of these callables as an input >>> eigenvalues, _ = lobpcg(A_lambda, X, maxiter=60) >>> eigenvalues array([100.], dtype=float32) >>> eigenvalues, _ = lobpcg(A_matmat, X, maxiter=60) >>> eigenvalues array([100.], dtype=float32) The traditional callable `LinearOperator` is no longer necessary but still supported as the input to `lobpcg`. Specifying ``matmat=A_matmat`` explicitly improves performance. >>> A_lo = LinearOperator((n, n), matvec=A_matmat, matmat=A_matmat, dtype=np.int16) >>> eigenvalues, _ = lobpcg(A_lo, X, maxiter=80) >>> eigenvalues array([100.], dtype=float32) The least efficient callable option is `aslinearoperator`: >>> eigenvalues, _ = lobpcg(aslinearoperator(A), X, maxiter=80) >>> eigenvalues array([100.], dtype=float32) We now switch to computing the three smallest eigenvalues specifying >>> k = 3 >>> X = np.random.default_rng().normal(size=(n, k)) and ``largest=False`` parameter >>> eigenvalues, _ = lobpcg(A, X, largest=False, maxiter=90) >>> print(eigenvalues) [1. 2. 3.] The next example illustrates computing 3 smallest eigenvalues of the same matrix `A` given by the function handle ``A_matmat`` but with constraints and preconditioning. Constraints - an optional input parameter is a 2D array comprising of column vectors that the eigenvectors must be orthogonal to >>> Y = np.eye(n, 3) The preconditioner acts as the inverse of `A` in this example, but in the reduced precision ``np.float32`` even though the initial `X` and thus all iterates and the output are in full ``np.float64``. >>> inv_vals = 1./vals >>> inv_vals = inv_vals.astype(np.float32) >>> M = lambda X: inv_vals[:, np.newaxis] * X Let us now solve the eigenvalue problem for the matrix `A` first without preconditioning requesting 80 iterations >>> eigenvalues, _ = lobpcg(A_matmat, X, Y=Y, largest=False, maxiter=80) >>> eigenvalues array([4., 5., 6.]) >>> eigenvalues.dtype dtype('float64') With preconditioning we need only 20 iterations from the same `X` >>> eigenvalues, _ = lobpcg(A_matmat, X, Y=Y, M=M, largest=False, maxiter=20) >>> eigenvalues array([4., 5., 6.]) Note that the vectors passed in `Y` are the eigenvectors of the 3 smallest eigenvalues. The results returned above are orthogonal to those. The primary matrix `A` may be indefinite, e.g., after shifting ``vals`` by 50 from 1, ..., 100 to -49, ..., 50, we still can compute the 3 smallest or largest eigenvalues. >>> vals = vals - 50 >>> X = rng.normal(size=(n, k)) >>> eigenvalues, _ = lobpcg(A_matmat, X, largest=False, maxiter=99) >>> eigenvalues array([-49., -48., -47.]) >>> eigenvalues, _ = lobpcg(A_matmat, X, largest=True, maxiter=99) >>> eigenvalues array([50., 49., 48.]) Nrr'z2Expected rank-2 array for argument Y, instead got z&, so ignore it and use no constraints.rrz-The mandatory initial matrix X cannot be Nonez$expected rank-2 array for argument XzData type for argument X is z0, which is not inexact, so casted to np.float32.)rr<zSolving standard generalizedz eigenvalue problem withr;z preconditioning zmatrix size %d zblock size %d zNo constraints z%d constraints z%d constraint zThe problem size z minus the constraints size z) is too small relative to the block size zd. Using a dense eigensolver instead of LOBPCG iterations.No output of the history of the iterations.z3The dense eigensolver does not support constraints.rOz> of the primary matrix defined by a callable object is wrong. z&Primary MatMul call failed with error rNz@ of the secondary matrix defined by a callable object is wrong. rMF)subset_by_index check_finiter[z$Dense eigensolver failed with error gz0 of the constraint not preserved and changed to rPTrQzLinearly dependent constraintsrSz)Linearly dependent initial approximationsz< of the initial approximations not preserved and changed to z* after multiplying by the primary matrix. )rlz7 of the restarted iterate not preserved and changed to z iteration zcurrent block size: zeigenvalue(s): zresidual norm(s): zFailed at iteration z with accuracies z' not reaching the requested tolerance .float32zeigh failed at iteration z with error z causing a restart. z with error zExited at iteration z with accuracies z& not reaching the requested tolerance z. Use iteration z instead with accuracy z. zFinal iterative eigenvalue(s): z"Final iterative residual norm(s): z< of the postprocessing iterate not preserved and changed to z(eigh has failed in lobpcg postprocessingz'Exited postprocessing with accuracies z$Final postprocessing eigenvalue(s): zFinal residual norm(s): )4lenr*rrrrTr issubdtyperinexactr)rnzerosprintminNotImplementedError isinstancer eyeintr7r toarrayr?rsqrtrrr8rrrrrKrYrarConesboolabsrnewaxissumwherer-dotdiagastypeblockrevsplitsqueeze)NAXrUrYr"maxiterr_rBretLambdaHistoryretResidualNormsHistoryrestartControl blockVectorXbestblockVectorXrHresidualTolerancebestIterationNumbersizeYnsizeX lambdaHistoryresidualNormsHistoryr,eigvalsrWvalsvecsrGgramYBY blockVectorBX_ blockVectorAXgramXAXr]eigBlockVectorr` activeMask blockVectorP blockVectorAP blockVectorBPsmallestResidualNormiterationNumberrestart forcedRestartexplicitGramFlag blockVectorR residualNorms residualNormcurrentBlockSizeactiveBlockVectorRactiveBlockVectorPactiveBlockVectorAPactiveBlockVectorBPactiveBlockVectorBRactiveBlockVectorARinvRmyepsgramXARgramRAR gramDtypegramXBXgramRBRgramXBRgramXAPgramRAPgramPAPgramXBPgramRBPgramPBPrcrdeigBlockVectorXeigBlockVectorReigBlockVectorPppappbppisN r#r r sxL#L! E |!! "a ' MMD|))*+,78    L &&q)EHII <  !#?@@!!HAu ==++RZZ 8 *<+=+=*>?= > A zz,bjjA 'A+u!5'3'9'9; !xx1e(<.:.@.@ B 9 : C = C )) 9 5LC %% !A%% "U**   ' 'Cqy+e33*U22 c  E a%i  s">ugF77: ; A  E1   #%E  5y!a%(G%!)nG !^,bffQc*+!bffQc*+77q!f$$$QWWI.CD!IIKJJqM = a0"&&#./Aa["&&#./Aww1a&((( 2GHa[ A 1 A  a.5+02JD$DbDzAttG}:  !'8C'?GGBHH\-?-?$@$D$DEIAAAAAA =lOM""l&8&88  !3!3 45&&3&9&9%:;CD)M..%%'-7 F d;G , M&7 <&8"L-DEElOMl000++,-+11239 :  nn!!#m3G"7?G^ 7E7 +BbkG% adZZq"u 56N"n%L$~%M }' >) %.J LMM66"((<+=+=">"B"BCOGM G #1 ='"**a-"88CQ!77C$s* ff\&&(<7;s , "7D !!3 4vvbff]34u< . .#/ "1 +  A~-0DD D MlOM""l&8&88  !3!3 45&&3&9&9%:;AB } !,  &&,*<*<<$$\%7%7$89**7*=*=)>?GHXXm&77u E"_ %>>+  J/0 1 ()9(:; < $WI. / ' 7 8 q "<: #>? Q !&|AzM'B!C "' am(D"E }&+M!Z-,H&I# =!"#5!6   # 0%+* , =!3%%'*<<>"  "4$$&);;="  !.B582/  % MM&&77H /"./q2    23 Q }')+>#1AD="$7'+=7EG.1+"At!-&5'#1'#(  $ $ 1EGGBHH%7%=%=>BBCE     &/?$  $  9(M"4 &8#&&,,.0CD&&+--2246IJ'-- !11Q6Gff\^^002MBG!11Q6Gff\^^002MBGff/11668:MNGff\^^0024GHGggg&--i8GffU)4Gff-Y?Ghh'78 JGff\^^0024GHGff/11668:MNGff/11668:MNGff\^^0024GHGff/11668:MNG"WYY^^%55:&&!3!5!5!:!:!>3CW2MNOEHHw0799>>3CW2MNOE )% G *.u/41A> >!yy)=)+2F)G)J L7KL! 1 LL "L-9MM ML-7 7 "L*>> >L( (S 9#R  ( ?c   7#R  B F=>A E FH !MM3O3DE&&'S(=?#  *  //@ cA  Z LCD!KL^? Ms7B9AV93B9AW-+AW9+ AX8AX6?AY4M AZ.T1A[ VA[V9 AWWAWWAWW AW6W"AW1W1AW6W9 AXXAXXAXX AX3X" AX.X.AX3X6 AY1X?'AY,Y,AY1Y4 AZ+Y=#AZ&Z&AZ+Z. A[Z7 A[[A[)r)Nr) NNNNNTrFFrg)__doc__rnumpyrrrrrrrrscipy.sparse.linalgr scipy.sparser __all__r$r-r8rCrKrYrarer r1r%r#rs&11.! * "  8'6:%&. b-     !l)r%