SIAM J. ScI. STAT. COMPUT.Vol. 7, No. 1, January 1986

1986 Society for Industrial and Applied Mathematics015

A HAMILTONIAN QR ALGORITHM*

RALPH BYERS’f

Abstract. This paper presents a variant QR algorithm for calculating a Hamiltonian-Schur decomposi-

tion [10]. It is defined for Hamiltonian matrices that arise from single input control systems. Numerical

stability and Hamiltonian structure are preserved by using unitary symplectic similarity transformations.

Following a finite step reduction to a Hessenberg-like condensed form, a sequence of similarity transforma-

tions yields a permuted triangular matrix. As the iteration converges, it deflates into problems of lower

dimension. Convergence is accelerated by varying a scalar shift. When the Hamiltonian matrix is real,

complex arithmetic can be avoided by using an implicit double shift technique. The Hamiltonian-Schur

decomposition yields the same invariant subspace information as a Schur decomposition but requires

significantly less work and storage for problems of size greater than about 20.

Key words. QR algorithm, Hamiltonian matrices, symplectic matrices, algebraic Riccati equation,

optimal control

Introduction. A matrix H (2n2n is Hamiltonian [1], [7], [10] if JH H*J 0where

(1) J=-InThe superscript asterisk denotes the conjugate transpose of a matrix, In denotes then-by-n identity matrix and 0n denotes the n-by-n matrix of zeros. Throughout thispaper, H will denote a Hamiltonian matrix. A bold face H will denote the set ofHamiltonian matrices. Partitioned into n-by-n blocks H is of the form

where G G* and F F*. We will write H HAM (A, G, F) for the matrix of (2).Hamiltonian matrices arise in mechanics and control theory. For example, consider

the optimal control problem

(3)

subject to the control system

(4)

minimize y*y + u*u dt

dx/dt Ax+ Bu,

y=Cx

where ACnn BCnp CCqn and x(0)=xoC are given. Set G=C*C andF BB*. It is well known [5] that under mild conditions (3) is minimized by

u -B*Xx

where X Cnn satisfies the algebraic Riccati equation

(5) G+ A*X + XA XFX On.

With this choice of u, x obeysdxd--7 a FX)x.

* Received by the editors October 11, 1983, and in revised form October 19, 1984. This work was partof the author’s doctoral dissertation while a graduate student supported by Cornell University. It merited

co-award of the Householder Prize V (1983) for the best thesis in Numerical Algebra.f Department of Mathematical Sciences, Northern Illinois University, DeKalb, Illinois 60115.

212

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

A HAMILTONIAN QR ALGORITHM 213

Denote the eigenvalues of a matrix M Cnn by h (M) and the open left half complexplane by C-. In general there are many solutions X of (5). The desired one is stabilizingin the sense that h (A-FX)c C-.

Solutions of (5) are related to invariant subspaces of HAM (A, G, F) [11]. If Y, Zand W C n satisfy

then X YZ-1 is a solution. The space spanned by the columns of Y*, Z*]* is aninvariant subspace of H HAM (A, G, F). Any basis of the same invariant subspaceyields the same value of X. We require h(-W)= A(A-FX) to be contained in C-.If the eigenvalues of HAM (A, G, F) have nonzero real part, then the desired invariantsubspace exists and is unique. Under mild conditions Z is nonsingular and X YZ-1

is unique [4].Laub [6] shows how to obtain an orthogonal basis by using the Francis QR

algorithm [3 ]. Although effective, Laub’s method does not take advantage of the specialstructure of Hamiltonian matrices. H H is treated as a general 2n-by-2n matrix.Ignoring the Hamiltonian structure of H is inefficient in both work and storage.Rounding errors can produce some surprising results. Eigenvalues of real Hamiltonianmatrices appear in plus/minus pairs, but eigenvalues calculated by the QR algorithmin the presence of rounding error may not [15]. This paper shows how to use unitarysymplectic matrices to preserve structure, cutting work and storage requirements inhalf. A matrix SC22 is symplectic [1], [7], [10] if S*JS=J. (J is defined in (1).)In the presence ofrounding errors our algorithm produces the eigenvalues of a "nearby"Hamiltonian matrix.

Unfortunately, our algorithm is limited to Hamiltonian matrices in whichrank (F)= 1. Such matrices arise when (4) is a single input control system, i.e., whenB consists of a single column.

Section 1 reviews the Schur decomposition, the Hamiltonian-Schur decomposi-tion, and the symplectic Schur decomposition. The economical use of Householdersymplectic and Jacobi symplectic matrices is discussed. Section 2 uses the Francis QRiteration to motivate the Hamiltonian QR iteration. Each Hamiltonian QR step isequivalent to a Francis QR step applied to an associated symplectic matrix. Section3 describes a finite step reduction of a Hamiltonian matrix to a Hessenberg-likecondensed form. The condensed form is preserved by the rest of the algorithm. Aheuristic argument suggests that one step ofthe Hamiltonian QR algorithm is equivalentto two steps of the Francis QR algorithm. Section 4 develops the Hamiltonian QRstep. Hamiltonian structure is preserved throughout, so no 2n-by-2n array is required.A numerically stable algorithm is presented. Section 5 discusses the details of deflation,choice of shift, and ordering of eigenvalues. Section 6 sketches how to modify theHamiltonian QR algorithm to avoid the use of complex arithmetic. Like the doubleFrancis QR step, the trick is to combine two successive Hamiltonian QR steps intoone. Section 7 presents the results of some numerical experiments solving the algebraicRiccati equation. Especially for problems of high dimension, the Hamiltonian QRiteration represents a significant savings over the Francis QR iteration.

Capital letters denote matrices, boldface lower case denote vectors, nonboldfacelower case letters denote scalars. The ith row and jth column of the matrix A arerepresented by A,. and A.j respectively. The norm IIAII may denote any reasonablywell balanced consistent matrix norm such as the Frobenius norm,

IIA[[ (trace (A’A))(/2).

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

214 RALPH BYERS

1. Numerical tools. A matrix S E C2nx2n is symplectic [1], [7], [10] if S*JS J. (Jis defined in (1).) A bold face S will denote the set of symplectic matrices. Productsof symplectic matrices are symplectic. Symplectic matrices are nonsingular.

If H E H and S S, then SHS-le H. Following the suggestion of [10], we usesymplectic similarity transformations to preserve Hamiltonian structure.

If Q is both symplectic and unitary, then it is of the form

Q=-Q2

where Q1, Q2 cnn [10]. The set of unitary symplectic matrices will be denoted S-U.Note that computer programs need store only the first n rows of Q.

If V Cnn is unitary, then diag V, V) E S U. A Householder symplectic matrix10] is the matrix P P(k, m, x) S- U defined by

P(k,m,x)=0,

where

fi =/5(k, m, x) In 2uu*

and u C is obtained from x C" by setting u v/llv[[2 where

0 ifl<_-j<min(k,m) or max (k, m) <j <-- n,

(6) vj xj if min (k, m) <j < max (k, m) or j= m,+ (x llx l)(Ix l + + IXml2) (1/2) if j k.

If Xk 0, then the quotient Xk/[Xk[ may be set equal to 1. By convention, if v-0, then/3 In. We admit the possibility k > m. The Householder reflection P is a Hermitianunitary matrix such that if y- Px, then

x if l<-j<min(k,m) ormax(k,m)<j<=n,

y 0 if min (k, m) <j < max (k, m) or j m,--(Xk/[Xkl)([Xk[2-- Ar IXm]2)(1/2) if j k.

Algorithms using Householder reflections depend upon their ability to zero some

components of a vector while leaving others unchanged. Specifically, suppose v C n,w- P(k, m,x)v and y= I3(k, m,x)x. If l <=j<min (k, m) or max (k, m)<j<=n, then

w v and yj x. Furthermore, if j m or min (k, m) <j < max (k, m), then yj O.Define a flop as the computational effort to execute the FORTRAN statement

A(I, J)= A(I, J)- S*A(K, J).

We will speak of a real flop when A and S are of REAL type and a complex flopwhen A and S are of COMPLEX type.

Our algorithms derive much of their efficiency from the following well-knowneconomies of work and storage. Let u be as in (6). Set

/U--H

and

s 2U2k/(u’u).

Note tk- 1. fi(k, m, x) can be stored on a computer as s and the m- k nontrivialcomponents of . When/3 is stored in this way, a vector v E C can be replaced by/3v

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

A HAMILTONIAN QR ALGORITHM 215

without explicitly forming [13, p. 234], [17, pp. 157-159] by setting

(7) v:= v- (s(fi*v)).

It is not necessary to perform multiplications by /’k 1 nor by those components of fithat are known to be zero. So (7) requires only 2lm kl+l flops.

The Hermitian matrix M may be overwritten by PMP without the use of an n-by-narray [17, p. 292] by

v:= s(Mfi),

(8) w:= v- (s/2)(*v),

M := M-fiw*-wfi*.

Notice that only the upper triangle ofM need be stored. If we do not bother to multiplyby those components of fi that are known to be zero or one, then (8) requiresapproximately 2( n + 1)lm kl + n + 2 flops.

We will use a superscript "7" to designate Householder reflections that are storedand used as described above.

A Jacobi symplectic matrix 10] J(k, c, s) is a unitary symplectic matrix of the form

(9) J(k, c, s)=-S C

where Icl= / Isl=- 1, s R and

C =diag (1, 1, 1,. ., 1, c, 1,. ., 1),(10) k-I

S=diag (0, 0, 0,. ., 0, s, 0,. ., 0).k-1

Given v, we C such that kWk, there is a Jacobi symplectic matrix Q-J(k, c, s)such that if

then y 0 and for j k, x =v and y w.Let H’= HAM (A’, G’, F’) J(k, c, s) HAM (A, G, F)J*(k, c, s). The matrices A’,

G’ and F’ can be obtained directly from A, G and F without using an extra squarearray simply by forming the obvious linear combinations of the kth rows and columnsof A, G and F. No more than a few n flops are required.

Care must be taken to avoid overflows, underflows and excessive rounding errorswhen constructing Householder and Jacobi symplectics. See [13] or [17] for details.

The following theorem describes a decomposition which displays the invariantsubspace structure of a Hamiltonian matrix. Our algorithm calculates this decompo-sition.

THEOREM 1 (Hamiltonian-Schur decomposition (HSD) [10]). If the eigenvaluesofH HAM A, G, F) have nonzero real part, then there exists Q S-U such that

(11) Q*HQ=[ T*I TEl0, -Twhere T1 is lower triangular and T2 T*2. Q can be chosen so that A (-T1)c C-.

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

216 RALPH BYERS

We will refer to a matrix T H with the zero structure of the right-hand side of(11) as Hamiltonian triangular.

A useful corollary isCOROLLARY 2 (Symplectic-Schur decomposition (SSD)). IfS C2n2n is symplec-

tic and has no eigenvalue of magnitude 1, then there is a matrix Q S-U such that

(12) O’so R [R* R2 ]LO. R-where R is lower triangular and R2R is Hermitian.

Proof. Let H- (S + I2n)(S-I2)-. It is easy to verify from the definition that His Hamiltonian. The eigenvalues of H are of the form (A + 1)/(A 1) where A A(S).By hypothesis IAI # 1, so each eigenvalue of H has nonzero real part. Also note that1 A (H). Apply Theorem 1 to H to get Q S-U and Hamiltonian triangular TQ*HQ. Equation (12) holds with this value of Q and R-(T+In)(T-I2)-.It follows directly from the symplectic structure of R [8, Eq. 14], that R2R isHermitian.

A matrix R S with the zero structure of (12) will be called symplectic triangular.The hypotheses of Theorem 1 and Corollary 2 are not the weakest possible [2],

[10], but are sufficient for the purposes of this paper. Not all Hamiltonian matriceshave an HSD, nor do all symplectic matrices have an SSD. The matrix J of (1) is bothHamiltonian and symplectic, but obviously cannot be factored as (11) or (12).

As the proof of the corollary suggests, there is a duality between the HSD andthe SSD. If k is not an eigenvalue of H- HAM (A, G, F) and Re (k)# 0, then

S(k) (H + k-I,)(H- kI2,)-’

is symplectic and

H (kS(k) + k-I2,,)(S(k) I2,)-’.

By analogy with the QR algorithm [3], we refer to the scalar k as a shift of origin.Observe that Q*HQ-T is an HSD if and only if Q*S(k)Q-R is an SSD. Theproblem of calculating the HSD of H is equivalent to calculating the SSD of S(k).

The HSD and SSD are special cases of the well-known (complex) Schur decompo-sition (CSD).

THEOREM 3 (Schur decomposition (CSD)). If MCn, there exists a unitarymatrix Q C" such that Q*MQ T is upper triangular.

Reversing the last n rows and columns of the matrices in Theorem 1 and Corollary2 gives Schur decompositions. The force of the theorem and corollary is that Q canbe chosen to be symplectic as well as unitary.

2. Outline of the Hamiltonian QR iteration. The QR algorithm [3] is a reliablemethod of calculating the CSD of a general square matrix M. It is based upon the QRiteration"

(13)

M() := M,

FOR j := 1,2,3,.

Factor QJRu := Mu

Mo+I := RUQo.unitary,triangular

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

A HAMILTONIAN QR ALGORITHM 217

At the end of each iteration M(j)- u(J)*MU(j) where U(j) is the unitary matrixu(j)= Q(O)Q(1)... Q(j-I). Under mild conditions [16] the iterates M(k) converge totriangular form in the sense that if <j, then

lim rn 0.koo

As stated, (13) is not a practical algorithm. The QR factorizations are too expensive.Convergence is slow. Practical algorithms are modifications of (13).

In order to calculate a HSD, one is led to specialize the QR iteration to symplecticmatrices. Every symplectic matrix can be factored into the product of a unitarysymplectic matrix and a symplectic triangular matrix [2]. The algorithms in 4 constructthe unitary symplectic factor. Hamiltonian matrices do not admit such a factorization.In principle, but not in practice, the HSD of a matrix H H can be obtained throughthe SSD of S (H + k-I2,,)(H-kI2,,) -1 by the symplectic QR iteration"

S() := S(k):= (H + ,I2n)(H- kI2,,)-’,FOR j := 1,2,3,.

(14)Factor QJR= S Q symplectic unitary,

s(J+l);_ R(j)Q(j)" R symplectic triangular

At the end of each iteration S(j)= u(J)*S(k)U(j) where U is the unitary symplecticmatrix U()= Q(O)Q()... Q(j-). Since (14) is a special case of (13), usually S(j) willconverge to symplectic triangular form. In that case H()= U()*HU() converges toHamiltonian triangular form. Note that the S()’s are symplectic and the H(J)’s areHamiltonian.

Iteration (14) does not provide a practical algorithm for calculating a HSD.Rounding errors involved in forming S(k) can be disastrous. In 5 we explain that itis desirable to choose k to be close to an eigenvalue of H. Since H-kI2,, is almostsingular, it is difficult to form S(k) accurately. Even when H-kI2, is not almostsingular, there is a possibility of cancellation error. Furthermore, the symplectic QRfactorization, costing O(n3) flops [2], is prohibitively expensive. The iteration convergestoo slowly.

The Hamiltonian QR algorithm is a modification of (14) that avoids the aboveproblems. One Hamiltonian QR step corresponds to the similarity transformation

H(+ := Q(*H(Q(

where Q(J is as in (14) and H(= H is the original Hamiltonian matrix. The Hamil-tonian QR algorithm performs symplectic similarity transformations directly to Hwithout forming S(k). The cost of each similarity transformation is reduced to O(n2)flops by reducing H to a Hessenberg-like condensed form. As the iteration converges,the problem deflates to sub-HSD’s and sub-CSD’s of smaller dimension. Convergenceis accelerated by varying k from one iteration to the next.

In broad outline the Hamiltonian QR algorithm is

H :- V*HV where V S-U is chosen to give H a Hessenberg-like zero structureDo While H is not nearly Hamiltonian triangular

Choose k C to be an approximate eigenvalue of HH:=Q*HQ where QS-U is chosen to make Q*S(k) symplectictriangular.

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

218 RLPH BYERS

3. Hamiltonian-Hessenberg form. A matrix M is said to be upper (lower) Hessen-berg if m0 0 for i>j- 1 (i <j + 1). The use of Hessenberg in place of full matricessaves much work in the QR algorithm [3].

The Hamiltonian QR iteration requires a condensed form similar to Hessenbergform. This section describes such a condensed form: Hamiltonian-Hessenberg form.An algorithm is presented that transforms a Hamiltonian matrix to Hamiltonian-Hessenberg form. It is restricted to Hamiltonian matrices H HAM (A, G, F) withrank (F)= 1. Such Hamiltonian matrices arise naturally from single input controlsystems. It is shown that the Hamiltonian-Hessenberg form is invariant through aHamiltonian QR step. The proof suggests that a Hamiltonian QR step is equivalentto two Francis QR steps.

A Hamiltonian matrix H has Hamiltonian-Hessenberg form, if it has the zerostructure of a 2n-by-2n upper Hessenberg matrix with the order of the last n rows andcolumns reversed. In other words, H HAM (A, G, F) is Hamiltonian-Hessenbergwhen A is lower Hessenberg and F is zero except possibly for

Let H HAM (A, G, F) be a Hamiltonian matrix with rank (F)= 1. We nowdescribe an algorithm for reducing H to Hamiltonian-Hessenberg form with a unitarysymplectic similarity transformation. An n 5 example is used for illustration. SinceF is rank 1 and Hermitian, there is a vector f C such that

F= +ff*.

If/3() =/3(5 1, f), then

H := t3(1)H/t3(1)

"X X X X X X X X X X-

X X X X X X X X X X

X X X X X X X X X X

X X X X X X X X X X

X X X X X X X X X X

0 0 0 0 0 x x x x x

0 0 0 0 0 x x x x x0 0 0 0 0 x x x x x0 0 0 0 0 x x x x x

_0 0 0 0 x x x x x x

Set/3(2)=/3(4 1, A,s). Replace H with ]t3(2)H/3(2). H now has the zero structure

H :=/()H/%()=

-X X X X X X X x X x-x X X X X X X x X X

x X X X X X X x X X

X X x X X X X X X X

0 0 0 x x x x x x x

0 0 0 0 0 x x x x 0

0 0 0 0 0 x x x x 0

0 0 0 0 0 x x x x 0

0 0 0 0 0 x x x x x.0 0 0 0 x x x x x x.

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

219 A HAMILTONIAN QR ALGORITHM 219

Set/3(3)=/3(3 1, A,4). Update H to give

H :=

X X X X X X X X X x-X X X X X X X X X X

x X X X X X X X x X

0 0 x x x x x x x x0 0 0 x x x x x x x

0 0 0 0 0 x x x 0 0

0 0 0 0 0 x x x 0 0

0 0 0 0 0 x x x x 0

0 0 0 0 0 x x x x x_0 0 0 0 x x x x x x.

Finally, set (4).._/3(2 1, A,2). Replacing H by (4)H/(4) puts H into Hamiltonian-Hessenberg form.

The unitary symplectic similarity transformation relating the original Hamiltonianmatrix to the final Hamiltonian-Hessenberg matrix is

Q (1)i(2)(3)(4),

Notice that Q is of the form diag (V, V).The following algorithm summarizes the reduction to Hamiltonian-Hessenberg

form.

ALGORITHM 1. Reduction to Hamiltonian-Hessenberg form.INPUT: A, G e C""’, f C"OUTPUT: V C"", V-1 V* chosen so that HAM (A’, G’, f’f’*)

diag V*, V*) HAM (A, G, ff*) diag V, V) is Hamiltonian-Hessen-berg. A is overwritten by A’; G is overwritten by G’; f is overwrittenby f’.

/3(1) :=/3(n, 1, f)A:=/(1)A/(1); G:=/(1)G/(’) f:=FOR j=2,3,4, , n-1

flu):= (n-j+ 1, 1, A,j+)A := fi(J)A(j)" G :=/3U)GI3U)

V:=I,FORj n-1, n-2, n-3,. , 1V := pU V.

The elementary reflectors /3u can be stored in the unused, i.e., the zeroed, locationsof A andf and one auxiliary n-vector. Because of the zero structure of V, multiplicationby 15u affects only the leading (n-j + 1)-by-(n-j + 1) principal submatrix so onlythat submatrix need be calculated. Similarly, the similarity transformation /3UA/3Uaffects only the first j + 1 columns of A, so only those columns need to be calculated.With this economy Algorithm 1 requires approximately 10n3/3 flops.

To serve as an effective condensed form, the Hamiltonian-Hessenberg form mustbe invariant through a Hamiltonian QR iteration as described in 2. The correspon-dence between the Francis QR iteration and the symplectic QR iteration is enough toprove this, but the following proof, similar to 17, p. 529], is more illuminating.

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

220 RALPH BYERS

THEOREM 4 (Invariance of Hamiltonian-Hessenberg form). Suppose HH isHamiltonian-Hessenberg, k V: h (H) and S(k) := (H + k-I2,)(H- kI2,)-1. If Q S-Uand R Q*S(k) is symplectic triangular, then H’- Q*HQ is Hamiltonian-Hessenberg.

Proof. Let K be 12, with the order of the last n rows reversed. We will show thatKH’K is Hessenberg.

Define S(= KSK, Q(O= KQK, R(= KRK, and H(= KHK. H( is upperHessenberg. R( is upper triangular. Define upper Hessenberg matrices H(, H(2;unitary matrices Q(I, Q(2; and triangular matrices R(, R(2 by

(15)

Q(1)R(1) H() +H( R(1Q(- k-/2,,R(2Q(2 H(1- kI2,,

H(2)= Q(2)R(2)+ kI2,.

The first two equations form a single shifted Francis QR step [17, Chap. 8], [13,Chap. 7]. The third equation calls for an RQ (triangular-unitary) factorization insteadof the usual QR factorization. The second two equations form what might be calledan RQ step. Q(1) is Hessenberg because it is the product of the Hessenberg matrix,H()+ kI2. and the triangular matrix, R(1)-’. Similarly H(1), Q(2) and H(2) are alsoHessenberg. Some manipulation of (15) gives

Q()Q(2),][ R(2)-, R()] (H(O) +/I)(H()- kI) -1

So [Q()Q(2),] forms the unitary part and [R(2)-’R(1)] forms the triangular part of theQR factorization of (H()+ I2,)(H()-kI2,)-1. It follows that apart from a unitarydiagonal factor

Q- KQ(1)Q(2)*K,H’ KH(2)K.

Thus, KH’K is Hessenberg. E]

Equation (15) shows that one step of the Hamiltonian QR iteration is equivalentto a QR step followed by an RQ step. This suggests that one Hamiltonian QR stepis equivalent to two Francis QR steps.

4. Hamiltoaiaa QR step. Given a Hamiltonian-Hessenberg matrix H, a unitarysymplectic matrix U and a scalar k C, one step of the Hamiltonian QR iterationconsists of replacing H by Q*HQ and U by UQ where Q S-U is chosen so that

(16) R Q*S(k) Q*(H + k-/2,)(H- kI2,)-’

is symplectic triangular. This section presents a practical procedure for performing aHamiltonian QR step. Neither S(k) nor R is explicitly constructed. The procedure isnumerically stable, requires no work arrays and uses O(n2) flops.

As the next lemma shows, Q can be factored into the product of three simplersymplectic-unitary factors. The Hamiltonian QR step consists of three substeps, onefor each factor of Q. The first step resembles a single shifted Francis QR step appliedto A. It uses symplectic reflections. The second step uses a symplectic rotation toextend the QR step to include all of H HAM (A, G, F). The final step resemblesanother single shifted Francis QR step. It uses symplectic reflections to return H toHamiltonian-Hessenberg form.

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

A HAMILTONIAN QR ALGORITHM 221

The formula for Q is given in the following notation. H HAM (A, G, F) isHamiltonian-Hessenberg and k A(H). For simplicity we will assume rank (F)= 1,i.e., F # 0,. V C"" is a unitary matrix such that

(17) T= v*(a* +is upper triangular with real diagonal components. Observe that V is upper Hessenberg.Define , g R by

(18)

where

(19)

Note that t2 + 2 1 and

[f.. t..] _g

Similarly define c, s so that c2+ s= 1 and

-c L.(k+k)

Define diagonal matrices C, S, , R"" as in (10) by

J(c,s,n)=C S

J(, , n)C

-S C -Let W C"" be a unitary matrix chosen so that

T(21) U= W*[-S C] V*F

is upper triangular.LEMMA 5. The matrix

satisfies (16).The proof appears in the appendix.

(k+ .)I- V’T* -C S][A* *

(24) [A; _]:=[_S C F _][ C S ]-S C

The definition of V (17) shows this to be little more than a (single) shifted QR step.We use [13, p. 378] modified to calculate a single QR step instead of a double one. V

^(1) ^(2) ^(3)is expressed as the product V V V I""-1). Note, T is not explicitly constructed.The lower Hessenberg structure of A is preserved. The matrix F on the left-hand sideof (23) is zero except for the trailing 2-by-2 principal submatrix.

The second part of the Hamiltonian QR step consists of the similarity transfor-mation

(23) A := V*AV, G := V* GV, F := V*FV.

The Hamiltonian QR step has three parts. The first part performs the similaritytransformations

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

222 RLPH YRS

The value of F on the left-hand side of (24) must have rank 1 so that HAM (A, G, F)can be returned to Hamiltonian-Hessenberg form in the third part of the algorithm.There are two solutions for the scalars c and s that leave rank (F) 1. The extraneousone is c + 1, s 0. The other is given by

2c[Re (fn-l,nln-l,n) + Re (f,-1,,-la,,)] s(fn-l,n-lgnn + la,-,,I) 0.

The notation Re (z) indicates the real part of z e C. Note that (24) affects only the lastrows and columns of A, G and F.

The third part ofthe Hamiltonian QR step returns HAM (A, G, F) to Hamiltonian-Hessenberg form with the similarity transformations

A:= W*AW, G:= W*GW, F:= W*FW.

We use Algorithm 1 modified to take advantage of the zero structure of A and F. Ais already lower Hessenberg and F is zero except for the trailing 2-by-2 principalsubmatrix. Reflections of the form/3(j, j + 1, As_l,.) can be used in place of P(n-j +1, 1, A.d+l).

In the next algorithm the matrix U=[U1 U2]C"2" represents the unitarysymplectic matrix

U-- U1-U2 U

ALGORITHM 2. Hamiltonian QR stepINPUT: A, G, FC"" such that HAM (A, G, F) is Hamiltonian-Hessenberg;

U C"2" a scalar shift k COUTPUT: A, G, F, U are overwritten by A’, G’, F’, U’ where HAM (A’, G’, F’)

Q* HAM (A, G, F)Q and U’= UQ for Q defined by (22)Part 1

9:=/3(1, 2, (A+ --/n)lg)A:= ’A’; G:= VGV;For j=2, 3, 4, 5, , n- 1

:=/3(j, j + 1, AS_l,,)A := ’A’; G:= G’;

F:= VFV

U := U diag V, V)

U := U diag V, V)

x := 2IRe (fn_l,nn_l,n) -- Re (fn--l,n--1,y:=f,,_l,,,_lg,,,,+la,,-1,,,I2Choose c, s It so that

Part 2

For j= 1,2,3,..., n-1

[a.j] [ c s][a.s]Lg,od -s c kgnjl

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

A HAMILTONIAN QR ALGORITHM 223

--ann $ C

IF If.-,,.-,I > If..I THEN/r :=/3(n, n- 1, F,.,_I)ELSE r :=/3(n, n 1, F,,)

A:=/C/A//; G:=]C/G/; F:=F; U:= Udiag(,)FOR j= n-l, n-2, n-3,... ,2

:= (j,j-- 1, A,,j+I)A:=A; G:=G; U:=gdiag(,)

Part 3

and

HQ QTII <- c, b ’-’ H

O’Q- .II- Q*Q- I.II <- c2bl-t.

The constants Cl and c2 may depend upon n and the choice of norm but not upon Hor Q. Thus the eigenvalues of T are those of a Hamiltonian matrix that differs fromH by a "small" relative error. The first n columns of Q span an invariant subspaceof a matrix that differs from H by a "small" relative error.

5. Computational details. Three details remain to be discussed: deflation, choiceof shift and ordering of eigenvalues.

As the Hamiltonian QR iteration converges, the codiagonal components of Atend to zero. Once a codiagonal element becomes negligible compared to IlH[I, it maybe set to zero. So H assumes the form

(25) H

A*I A2"1 G,,

0 --AllF22 -A21 -A::J

Let A* UT* U* be a CSD of A*, and let

[A2"2 G221( V ] E_V1FEE -AEEJ V2 V2

The reflections should be used economically as described in 1. Also note that A islower Hessenberg throughout the algorithm, so no component above the codiagonalneeds to be explicitly calculated. Only the trailing 2-by-2 principal submatrix of Fmust be calculated and stored. The other components of F are zero. With theseeconomies Algorithm 2 requires approximately 12n2 complex flops.

From 1.5 n to 2n iterations of Algorithm 2 usually transform H to within roundingerror ofbeing Hamiltonian triangular. See the discussion of deflation in the next section.

Algorithms 1 and 2 are stable in the presence of rounding errors. Standard erroranalysis of unitary similarity transformations [2], [17, Chap. 3] can be used to showthat using digit base b arithmetic, rounding errors cause the original Hamiltonianmatrix H HAM (A, G, if*), the calculated similarity transformation Q, and the finalHamiltonian triangular matrix T HAM (T1, T2, 0n) to satisfy

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

224 RALPH BYERS

be a HSD of HAM (A2z G22, F22). Define Q e S-U by

0 V1 0

0 0 U0 -v 0 vd

A calculation shows that Q*HQ is in Hamiltonian triangular form. Thus (25) deflatesinto two smaller problems: a CSD of AI*I and a HSD of HAM (A22, G22, F29).

When there are two consecutive small codiagonal elements, a partial deflation canoccur. The situation is illustrated by the following diagram.

X X X X X X X X X X"

X X X X X X X X X X

0 d x x x x x x x x0 0 @ x x x x x x x0 0 0 x x x x x x x

0 0 0 0 0 x x 0 0 00 0 0 0 0 x x-d 0 0

0 0 0 0 0 x x x -e 0

0 0 0 0 0 x x x x x0 0 0 0 x x x x x x

The consecutive small components are d and e. Let W be the leading 3-by-3 submatrixof A. Let P be a 3-by-3 unitary matrix such that P*( W- kI) is lower triangular. DefineQmS-U by

P 0 0 0

0 I2 0 0

0 0 P 0

0 0 0 I2

Replacing H by Q*HQ performs a Francis QR step on W, but a42 becomes nonzero.The magnitude of the fill-in is Idei/([d[=+ [a33- k[Z) (/2). When this quantity is negligiblecompared to [[H[[, it may be ignored.

It is important for computer programs to deflate when possible. If shifts are chosenfrom the leading 2-by-2 principal submatrix of All as suggested below, they will notin general be good choices for HAM (A22 G22, F). Convergence will be slow.

Consider the choice of the shift k C. So far it has been convenient to assumethat kh(H). Now suppose kh(H). The fact that (H-kI2,) is singular does notcause Algorithms 1 and 2 to fail. There is no attempt to divide by zero or to invert asingular matrix. Returning to the notation of Theorem 4, the QR-RQ sequence of (15)remains defined when k e h (H). Both (H() + ,Izn) and (H(1) kI2,,) are singular [ 1 ],[7], 10]. Hence some diagonal element of R(1) and of R(2) must be zero. If the problemhas been deflated as described above, then H() is unreduced, i.e., all codiagonalelements are nonzero. It follows that r(1).. 0 and r11"(2) 0. This gives H’ the zero structure

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

A HAMILTONIAN QR ALGORITHM 225

"x X

0 x0 x0 0

0 0

0 0

0 0

0 0

x x x x x xX X X X X X

X X X X X X

X X X X X X

0 0 x 0 0 0

0 0 x x x 0

0 0 x x x x0 x x x x x

Furthermore, k=h=-h’.+l,.+l. No difficulty arises when kh(H). In fact it isdesirable to have k h (H), because the problem deflates immediately.

This suggests that the shift k C be chosen to be hl, since as the iteration converges,hl converges to an eigenvalue. A better strategy is to choose k to be an eigenvalue ofthe leading 2-by-2 principal submatrix of H.

Now consider the problem of ordering eigenvalues. Let H HAM (A, G, F) haveHamiltonian-Schur form

If Q2 is nonsingular, then X =-QQ satisfies the algebraic Riccati equation (5). Theeigenvalues of (A-FX) are the eigenvalues of-TI*. For most purposes, X must bechosen so that (A-FX) is stable, i.e., A(A-FX)c C-. When shifts are chosen asdescribed above, the eigenvalues may appear in any order along the diagonal of T. Itis necessary to use a unitary symplectic similarity transformation to order the eigen-values so that h (- T) C-

Suppose h A(-T) has positive real part. We must interchange it with theeigenvalue - of T*. The method of [14] can be used to find a unitary matrix V e C""such that - is the (n, n) entry of V*T*IV. Replace T by diag (V*, V*)T diag (V, V).Now we must interchange the (n, n) entry of -T1 with that of T*. Choose c, s e sothat

Let K J(n, c, s) (equation (9)). The interchange is effected by replacing T by KTK*.

6. Double Hamiltonian QR step with implicit shifts. Complex arithmetic requirestwice the storage and at least four times the work of real arithmetic. Furthermore,rounding errors will cause quantities which should appear in complex conjugate pairsto fail to do so. It is preferable to use real arithmetic when possible. When the originaldata A, G and F are real, complex arithmetic can be avoided by combining twosuccessive Hamiltonian QR steps into one. The Francis implicit QR algorithm employsa similar trick [3].

The reasoning and numerical techniques used for the double implicit step aresimilar to those used for the simpler algorithm of 4 and 5. This section outlines thedifferences. A full account appears in [2].

Observe first that when Algorithm 1 is applied to a real Hamiltonian matrix, allintermediate quantities and the final Hamiltonian-Hessenberg matrix are real.Algorithm 1 does not need to be modified to avoid complex arithmetic.

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

226 RALPH BYERS

Apply Algorithm 2 with shift k to the real Hamiltonian-Hessenberg matrix H-HAM (A, G, F) to get a not necessarily real Hamiltonian-Hessenberg matrix H’-Q’*HQ’. Apply Algorith.m 2 with shift/ to H’ to get a Hamiltonian-Hessenberg matrixH"= Q"*H’Q". Define S(k) by

g(k) [(H + kI2,)(H + k-/2,)][(H- kI2,)(H-/I2,)]-’.Set Q Q’Q". It is easy to show that

(26) R Q*S(k)

is symplectic triangular. Note that S(k) is real, so by analogy with the general QR(orthogonal-triangular) factorization Q and R may be chosen to be real [17]. HenceH" is real.

To obtain H" directly from H we need to calculate the similarity transformationQ*HQ where Q 2n2n is orthogonal and satisfies (26). Using the method of Lemma5, it can be shown that

Q diag V, V)K diag W, W).

The matrices V and WR"" are orthogonal. K is the product of a Householdersymplectic and three Jacobi symplectic matrices.

The matrix V is chosen so that

L= vT[(A+ I,,)(A+ kI.)]

is upper triangular. The first part of the double implicit step is essentially a doubleimplicit Francis QR step applied to A.

The matrix K plays the role of the rotation in Lemma 5, but its construction ismore complicated. Define

H/ diag V*, V*)(H + kI2n)(H- kI2.) diag (I., V),

H_ (H kI2. (H kI2. diag (I., V),

and

N KH+H-_1.

The relationship between H/, H_, N, (k) and Q is

Q*(k) =diag W*, W*)KH+H- =diag W*, W*)N.So K must be chosen to make (KH+H-) block upper triangular with n-by-nblocks. The trailing 2-by-2 principal submatrices of the n-by-n blocks of H/ and H_determine K.

The final orthogonal matrix W is chosen to return H to Hamiltonian-Hessenbergform. Again we use Algorithm 1 modified to take advantage of the zero structure ofA, G and F. This time reflections of the form P(j,j-2, Aj_,,) are used.

7. Numerical examples and conclusions. The double Hamiltonian QR algorithmwas implemented in FORTRAN using DOUBLE PRECISION arithmetic on Cornell’sIRM370/168. We also wrote a similar program to calculate a real Schur decomposition[3]. We used both algorithms to solve several algebraic Riccati equations as describedin the introduction.

In order to minimize the effect of programming style on our observations, we usedour own implementation of the QR algorithm in preference to one of the fine programsfrom EISPACK [12]. By using the BLAS [9] and several subroutines written in the

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

A HAMILTONIAN QR ALGORITHM 227

spirit of the BLAS, we arranged for many of the innermost loops of both algorithmsto be executed by the same subroutines. Our QR algorithm ran somewhat slower thanEISPACK’s, but the difference could be attributed almost entirely to extra subroutinecalls. When subroutines were expanded in line, the execution times of our QR algorithmand EISPACK’s differed by only a few percent.

Test problems were constructed by performing random orthogonal similaritytransformations of the following 5-by-5 patterns.

-1 1 1 1 1

-2 1 1 11A= 0-3 1 F=

0000-4150 0 0-

0 0 0 0 00 0 0 0 0

G- 0 0 0 0 0 X=0 0 0 0 0

60 0 0 0

1 1 1 1 11 1 1 1

1 1 1 1

1 1 1 1

I_1 1 1 1 1

0 0 0 0 0

O0 0 0 0

0 0 0

0 0 0 010 0 0

Similar problems were run with dimensions n 5, 10, 20 and 30.Both algorithms produced solutions of comparable accuracy. Table 1 compares

execution times. For small problems the two methods used approximately the sameamount of time. For larger problems, the advantage of the Hamiltonian-Schurdecomposition is unmistakable.

TABLEExecution time of Francis QR algorithm versus Hamiltonian QR algorithm.

millisecondsdimension 5 10 20 30

Francis QR algorithm 47 226 1474 4639Hamiltonian QR algorithm 49 181 969 2681

The Francis QR algorithm requires two 2n-by-2n arrays for a total of 8n2 storage.The Hamiltonian QR algorithm requires about 7n2/2 storage.

Both the Francis QR algorithm and the Hamiltonian QR algorithm produce thesame invariant subspace information. For problems of size greater than about 20, theHamiltonian QR algorithm requires significantly less work and storage. It is restrictedto problems that arise from single input control systems. However, the restriction isnecessary only at the first reduction to condensed form, so some modification eitherin the condensed form or in the algorithm by which it is obtained may lead to a moregenerally applicable algorithm.

(27)

Appendix A. Proof of Lemma 5. In the notation of 3, the matrix

satisfies (16).

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

228 RALPH BYERS

Proof Define L C2n2n in n-by-n blocks by

(28, L=[L1 L2]=[a*-kI,. G ][I. ][ q ]L L4 f -A- kin On -S C

The choice of V in (17) and and in (18) make L3=0. and L4 lower triangular.Define M C2"2 by

M=O. F -A+k. O. -g

Paition M into n-by-n blocks and use (17) to get

(29) M= MM M V* (+g-V*r* - d

Recall that is zero except for s, C is diagonal, T is upper triangular, V is upperHessenberg, and F is zero except for f. Hence M is upper triangular and M islower Hessenberg. M is zero except for the last two components of the nth column.An n- 5 example of the zero structure is

X X X X X

0 x x x x0 0 x x x0 0 0 x x0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 x

,0 0 0 0

X X X X

x X X X

X X X xx X X xX X X x

x x 0 0x x x 0x X X xx X X xX X X X

X

xxX

X

0

0

0X

X

Solving for m.., m2n_l, and m2n.n in (29) gives

m.. t..- v.v,.,(31) m2,,-,,,,-

m2n, v.. (f..+ t..)-(k +The choice of and in (18) makes m2._l,.=0 and m2.,. =-(k+/). The (9,5)component of (30) is zero.

Using (18) to simplify (31) further gives

m.. t..-f..V**.GV..)/ r,

m2... k + k)f../ r

where r is given by (19). Define N by

N N4 o. w* -s c M M4"The definition (20) of c and s shows that N3 =0., and, from definition (21), N4 islower triangular.

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

A HAMILTONIAN QR ALGORITHM 229

With Q given by (27), equations (28) through (32) combine to show

NL-1= Q*S(k)= Q*(H+ k-/2,)(H-kI2,,)-’.Recall N and L-1 are block upper triangular with n-by-n blocks. Hence Q*S(k) isblock upper triangular. The (2, 2) block of Q*S(k) is lower triangular since the (2, 2)blocks of N and L-1 are. Thus Q*S(k) is syrnplectic triangular.

Acknowledgments. I am grateful to Charles Van Loan for his guidance and forhelpful comments on earlier drafts of this paper.

REFERENCES

[1] N. BURGOYNE AND R. CUSHMAN, Normalformsfor real linear Hamiltonian systems, The 1976 AmesResearch Center (NASA) Conference of Geometric Control Theory, C. Martin and R. Hermann,eds., Math. Sci. Press, Brookline, MA, 1976, pp. 483-529.

[2] R. BYERS, Hamiltonian and symplectic algorithmsfor the algebraic Riccati equation, Doctoral dissertation,Cornell Univ., Ithaca, NY, 1983.

[3] J. FRANCIS, The QR transformation--Part II, Comput. J., 5 (1962), pp. 332-345.[4] V. KUCERA, A contribution to matrix quadratic equations, IEEE Trans. Automat. Control, AC-17 (1972),

pp. 344-347.[5] H. KWAKERNAAK AND R. SIVAN, Linear Optimal Control Systems, Wiley-Interscience, New York,

1972.[6] A. LAUB, A $chur methodfor solving algebraic Riccati equations, IEEE Trans. Automat. Control, AC-24

(1979), pp. 913-925.[7] A. LAUB AND K. MEYER, Canonical forms for symplectic and Hamiltonian matrices, Celestial Mech.,

9 (1974), pp. 213-238.[8] A. LAUB, Schur techniques in invariant imbedding methodsfor solving two point boundary value problems,

21st IEEE Conference Decision and Control, Orlando, Vol. 1, Institute of Electrical and ElectronicsEngineers, Inc., New York, 1982, pp. 56-61.

[9] C. LAWSON, R. HANSON AND D. KINCAID, Basic linear algebra subprograms for FORTRAN usage,ACM Trans. Math. Software, 3 (1979), pp. 308-323.

10] C. PAIGE AND C. VAN LOAN, A Schur decomposition for Hamiltonian matrices, Linear Algebra Appl.,41 (1981), pp. 11-32.

[11] J. POTTER, Matrix quadratic solutions, SIAM J. Appl. Math., 14 (1966), pp. 496-501.12] B. SMITH, J. BOYLE, B. GARBOW, Y. IKEBE, V. KEMA AND C. MOLER, EISPACK Guide, Springer-

Verlag, New York, 1974.[13] G. STEWART, Introduction to Matrix Computations, Academic Press, New York, 1973.14] G. STEWART, HQR3 and EXCHNG: FORTRANsubroutinesfor calculating and ordering the eigenvalues

of a real upper Hessenberg matrix, ACM Trans. Math. Software, 2 (1975), pp. 275-280.[15] C. VAN LOAN, A symplectic method for approximating all the eigenvalues of a Hamiltonian matrix,

Numerical Analysis Report No. 71, Dept. Mathematics, Univ. Manchester, Manchester, England,1982.

[16] D. WATKINS, Understanding the QR algorithm, SIAM Rev., 24 (1982), pp. 427-440.[17] J. WILKINSON, The Algebraic Eigenvalue Problem, Clarendon Press, Oxford, 1965.

Dow

nloa

ded

09/2

4/13

to 3

5.8.

11.2

. Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php