Seminar LSD 2005 Bw

8/6/2019 Seminar LSD 2005 Bw

1/24

LSD - September 2005 J.C. Gmez 1

Subspace Identification of

Hammerstein and Wiener Models

Subspace Identification ofSubspace Identification of

Hammerstein and Wiener ModelsHammerstein and Wiener Models

Speaker

Juan C. Gmez

Laboratory for System Dynamics and Signal ProcessingFCEIA, Universidad Nacional de Rosario

ARGENTINA

[email protected]

Research SeminarResearch Seminar


OutlineOutlineOutline

Introduction: Motivation, New results

A (veryvery) brief review on Subspace State-Space System IDentifi-

cation Methods Block-oriented Nonlinear Models

Subspace Identification of Hammerstein Models

Subspace Identification of Wiener Models

Simulation Examples

Conclusions


2/24


Most physical processes have a nonlinear behaviour, except in a

limited range where they can be considered linear. The performance of controllers designed from a linear approxi-

mation is strongly influenced by a change in the operating point

of the system.

Nonlinear models are able to describe more accurately the global

behaviour of the system, independently of the operating point.

Many dynamical systems can be represented by the inter-

connection of static nonlinearities and LTI systems. These models

are called blockblock--orientedoriented nonlinear models.

IntroductionIntroductionIntroduction

Motivation for Nonlinear (Subspace) Identification Motivation for NonlinearMotivation for Nonlinear ((SubspaceSubspace)) IdentificationIdentification


Subspace Methods have been very successful for the identification of

LTI models in many practical applications.

Although there is a well developed theory for Subspace Identification

methods for LTI systems, this is not the case for nonlinear systems.

Some recent contributions in this area are: (Verhaegen & Westwick,

1996) in Subspace Identification of Hammersterin and Wiener models,and (Chen & Maciejowski, 2000) and (Favoreel et al., 1999) in

Subspace Identification of bilinear systems.


3/24


New subspace algorithms for the simultaneous identification of the

linear and nonlinear parts ofmultivariable Hammersteinmultivariable Hammerstein and WienerWiener

models are presented.

The proposed algorithms consist basically of two steps:

StepStep 1:1: a standard (linear) subspace algorithm applied to an

equivalent linear system whose inputs (outputs) are filtered (by

the basis functions describing the static nonlinearities)

versions of the original inputs (outputs).

StepStep 2:2: a 2-norm minimization problem which is solved via

an SVD.

Provided the conditions for the consistency of the linear subspacealgorithm used in Step 1 are satisfied, consistencyconsistency of the estimates can

be guaranteed.

The new results (Gomez & Baeyens, 2005) The new resultsThe new results (Gomez & Baeyens, 2005)


1. Gmez, J.C. and Baeyens, E.. Subspace Identification of

Multivariable Hammerstein and Wiener Models, European

Journal of Control, Vol. 11, No. 2, 2005.

2. Gmez, J.C., Jutan, A. and Baeyens, E.. Wiener Model

Identification and Predictive Control of a pH NeutralizationProcess.IEE Proceedings on Control Theory and Applications,

Vol. 151, No. 3, pp. 329-338, May 2004.

References


4/24


Subspace State-Space System IDentificationSSubspaceubspace SStatetate--SSpacepace SSystemystem IDIDentificationentification

They combine tools ofSystem TheorySystem Theory, Numerical Linear AlgebraNumerical Linear Algebra

and GeometryGeometry (projections).

They have their origin in Realization TheoryRealization Theory as developed in the

60/70s (Ho & Kalman, 1966).

They provide reliable state-space models ofmultivariablemultivariable LTI

systems directlydirectly from input-output data.

They dont require iterative optimization procedures no problems

with local minima, convergence and initialization.

4SID Methods4SID Methods

PropertiesProperties


They dont require a particular (canonical) state-space realization

numerical conditioning improves.

They require a modest computational load in comparison to tradi-

tional identification methods like PEM.

The algorithms can be (they have been) efficiently implemented in

software like MatlabMatlab.

Main computational tools are QR and SVD.

All subspace methods compute at some stage the subspacesubspace spanned

by the columns of the extended observability matrix.

The various algorithms (e.g., N4SID, MOESP, CVA) differ in the

way the extended observability matrix is estimated and also in the way

it is used to compute the system matrices.


5/24


The system modelThe system model

kkkk

kkkk

eDuCxy

KeBuAxx

++=

++=+1 StateState--space model inspace model ininnovation forminnovation form

The identification problemThe identification problem

To estimate the system matrices (A, B, C, D) and K, and the model

order n, from an (N+-1)-point data set of input and output

measurements

{ } 11

,+

=

N

kkkyu


RealizationRealization--based 4SID Methodsbased 4SID Methods

=

=0l

ll kk uhy

>

==

0,

0,1 l

lll BCA

Dh

For a LTI system, a minimalminimal state-space realization (A, B, C, D)

completely defines the input-output properties of the system through

where the impulse response coefficients are related to the system

matrices by

convolution sumconvolution sum

lh


6/24


=

++

+

11

132

21

jiii

j

j

ij

hhh

hhh

hhh

H

L

MOMM

L

L

Impulse ResponseImpulse Response

HankelHankel MatrixMatrix

jiijH C=

An estimate of the extended observability matrix can be computed by a

full rankfull rankfactorization of the impulse response Hankel matrix. Thisfactorization is provided by the SVD of matrix Hij.

ExtendedExtended

ObservabilityObservability

MatrixMatrix

((i>n)i>n)

ExtendedExtended

ControlabilityControlability

MatrixMatrix

(j>n)(j>n)


[ ]4342143421

ji

TT

T

T

ij VUVUV

VUUH

C

12

1

1

21

11111

2

1

2

1

210

0

=

=

In the absence of noise, Hij will be a rankn matrix, and 1 will contain

the n non-zero singular values model order is computed.model order is computed. In thepresence of noise, Hij will have full rank and a rank reduction stage will

be required for the model order determination.

rank reductionrank reduction

Problems:Problems: it is necessary to measure or to estimate (for example, via

correlation analysis) the impulse response of the system not goodnot good


7/24


Direct 4SID MethodsDirect 4SID Methods

NUXY ++= H

=

+

+

11

132

21

N

N

N

yyy

yyyyyy

L

MMMM

LL

Y

Output blockOutput blockHankelHankel matrixmatrix

(In a similar way are defined the

Input block Hankel matrix U

and

the Noise block Hankel matrix N.)

=

1

CA

CA

C

MExtendedExtended ( > n)

ObservabilityObservability MatrixMatrix

[ ]Nxxx ,,, 21 L=X

State Sequence MatrixState Sequence Matrix

(1)fundamental equationfundamental equation


=

DBCABCABCA

DCBCAB

DCB

D

H

L

MOMMM

L

L

L

432

0

00

000

Lower triangular blockLower triangular block

ToeplitzToeplitz matrix of impulsematrix of impulse

responsesresponses (unknown).

The main idea of Direct 4SID methodsThe main idea of Direct 4SID methodsIn the absence of noise (N

= 0), eq. (1) becomes

and the part of the output which does not emanate from the state can

be removed by multiplying (from the right) both sides of eq. (2) by the

orthogonal projection onto the null space of U, i.e.by

UXY H+= (2)


8/24


( ) ==

UUUUUU

1TTIT orthogonal projectionorthogonal projection

I=

UU

This yields

=

XUUY

NoteNote that the matrix on the left depends exclusively on the input-output

data. Then, a full rank factorizationfull rank factorization of this matrix will provide an

estimate of the extended observability matrix. Estimates of the

corresponding system matrices can be obtained by resorting to the shiftshift

invariance propertyinvariance property of the extended observability matrix, and by solving

a system of linear equations in the least squares sense.

(3)

such that


The factorization is provided by the SVD of the matrix on the left side

[ ]

=

=

TT

T

T

VUVUV

VUU 1

21

1

21

11111

2

1

2

1

210

0

43421

UY

rank reductionrank reduction

(model order estimation)(model order estimation)

(In the absence of noise 2 = 0)

Weighting MatricesWeighting Matrices

Row and column weighting matrices can be introduced in (4) before

performing the SVD of the matrix in the left hand side. Any choice of

positive-definite weighting matrices Wr and Wc will result in consistent

estimates of the extended observability matrix.

(4)


9/24


[ ]

=

=

TT

T

T

cr VUVUV

VUUWW 1

21

1

21

11111

2

1

2

1

210

0

43421

UY

change of coordinates in statechange of coordinates in state--spacespace

Existing algorithms employ the following choices for matrices Wrand Wc,

MOESP (Verhaegen, 1994):

CVA (Larimore, 1990):

N4SID (Van Overschee and de Moor, 1994):

== TT U

T

Ucr NWIW

11

,

21

21

1,

1

=

= T

Uc

T

Ur TT NW

NW

YY

==

11

, TUcr TN

WIW


Computation of the system matricesComputation of the system matrices

Given an estimate of the extended observability matrix, estimates

of the system matrices can be computed as:

: first row block of

: solving in the least squares sense

C

A

A

= shiftshift--invariance propertyinvariance property

solving a system of linear equations:and DB


10/24


In the presence of noise

NUXY ++= H

+=

UNXUYand

noise term needs to benoise term needs to be

removedremoved

The noise term can be removed by correlating it awaycorrelating it away with a suitable

matrix. This can be interpreted as an oblique projection.oblique projection.

Presence of noisePresence of noise


Block-oriented Nonlinear ModelsBlockBlock--oriented Nonlinear Modelsoriented Nonlinear Models

Fig. 4: Hammerstein-Wiener Model (LNL)

yk

StaticNonlinearityLTI System LTI System

uk

Fig. 2: Wiener Model (LN)

LTI System

yk

uk

StaticNonlinearity

Fig. 3: Hammerstein-Wiener Model (NLN)

StaticNonlinearity LTI System

yk

uk

StaticNonlinearity

LTI SystemStatic

Nonlinearityykuk

Fig. 1: Hammerstein Model (NL)


11/24


Hammerstein Model IdentificationHammerstein Model IdentificationHammerstein Model Identification

Nonlinear subsystemNonlinear subsystem

mk

nk

pnk

mk vxy

,

,, k

( ) ( )=

==r

i

kiikk uguNv

1

( ) ( )rig ppi ,,1,: L=

Problem FormulationProblem FormulationProblem Formulation

k

N(.)DC

BA ykuk

k

vk

Fig. 5: Hammerstein model

++=

++=+

kkkk

kkkk

DvCxy

BvAxx

1

LTILTI subsystemsubsystem

known basis

functions

unknown matrix

parameters

(1)

(2)

(3)

( )rippi ,,1 L=


Identification problem: to estimate the unknown parameter matrices

, and A, B, C, and D characterizing the nonlinear and the linear

parts, respectively, and the model ordern, from an N-point data set of

observed input-output measurements.

Identification problemIdentification problem:: to estimate the unknown parameter matrices

, and A, B, C, and D characterizing the nonlinear and the linear

parts, respectively, and the model ordern, from an N-point data set of

observed input-output measurements.

( )rippi ,,1, L=

{ }Nkkk

yu1

, =

Subspace Identification AlgorithmSubspace Identification AlgorithmSubspace Identification Algorithm

( )

( )

++=

++=

=

=

+

k

r

i

kiikk

k

r

i

kiikk

ugDCxy

ugBAxx

1

1

1

(3) (1), (2)

Identifiabilityproblem

NormalizationNormalization 12=i


12/24


13/24


Let have ranks>p, and let its economy size SVD be

partitioned as

rpmn

BD

+ )(

[ ]

===

=T

Ts

i

T

iii

T

BD V

V

UUvuVU 2

1

2

1

211 0

0

with .( )pprppmn VU ,,,diagand,, 2111)(1 L= +

Then

( ),,argmin,

111

2

2,,

VUD

B

D

B TBD

DB

=

=

and the approximation error is given by

.

2

1

2

2

+=

p

T

BDD

B

(4)

Result 1ResultResult 11

Normalizationin provided by

the SVD


Identification AlgorithmIdentification AlgorithmIdentification Algorithm

The subspace algorithm can be summarized as follows.

StepStep 1:1: Compute estimates of the system matrices , and the

model ordern, using any available (linear) subspace algorithm, such as

N4SID, MOESP, CVA.

StepStep 2:2: Based on the estimates compute an estimate of matrix .

StepStep 3:3: Compute the SVD of and its partition as in (4).

StepStep 4:4: Compute the estimates of the parameter matricesB, D, and as

BDBD

respectively.

( )DCBA ~,,~,

DB~

and~

BD

1

11

V

UD

B

=

=


14/24


Result 2: Consistency AnalysisResultResult 2:2: Consistency AnalysisConsistency Analysis

N

Under some assumptions on persistency of excitationpersistency of excitation of the inputs, which

depend on the particular subspace method used in StepStep 11 of the algorithm,

the estimates are consistentconsistent in the sense that they converge

to the true values when the number of data points .

The consistency of , implies that of B, D, and .

DCBA ~,,~,

DB~

and~


Wiener Model IdentificationWiener Model IdentificationWiener Model Identification

Nonlinear subsystemNonlinear subsystem

m

k

n

k

mnk

pk vxu

,

,, k

( ) ( )=

==r

i

kiikk ygyNv1

1

( ) ( )rig mm

i,,1,: L=

Problem FormulationProblem FormulationProblem Formulation

k

N(.)DCBA

ykuk

k

vk

Fig. 7: Wiener model

++=

++=+

kkkk

kkkk

DuCxv

BuAxx

1

LTILTI subsystemsubsystem

known basisfunctions

unknown matrixparameters

(5)

(6)

(7)

( )rimmi ,,1 L=


15/24


Identification problem: to estimate the unknown parameter matrices

, and A, B, C, and D characterizing the nonlinear and the

linear parts, respectively, and the model ordern, from an N-point data set

of observed input-output measurements.

Identification problemIdentification problem:: to estimate the unknown parameter matrices

, and A, B, C, and D characterizing the nonlinear and the

linear parts, respectively, and the model ordern, from an N-point data set

of observed input-output measurements.

( )rimmi ,,1, L=

{ }Nkkk

yu1

, =

Subspace Identification AlgorithmSubspace Identification AlgorithmSubspace Identification Algorithm

( )

++==

++=

=

+

kkk

r

i

kiik

kkkk

DuCxygY

BuAxx

1

1

(7) (6)

NormalizationNormalization 12=+

[ ] ( ) ( )[ ]TkTrkTkr ygygY ,,,,, 11 LL ==

++=

++=+

kkkk

kkkk

uDxCY

BuAxx

~~~1

DDCC +

+

== ~

,~

Identifiabilityproblem


k

DC

BA~~

Ykuk

k

Fig. 8: Equivalent LTI model

with output Yk

++=

++=+

kkkk

kkkk

uDxCY

BuAxx

~~~1

LinearLinear Subspace AlgorithmsSubspace Algorithms

(N4SID, MOESP,CVA)

EstimatesEstimates nDCBA ordermodel,~

,~

,,


16/24


The problemproblem is how to compute estimates of matrices C, D, and +

from the estimates of the matrices~

and,~

DC

+

and,

,

DC

( ) [ ]

= ++

+

2

2,,

~~argmin,, DCDCDC

DC

The solutionsolution to this optimization problem is provided by the SVD of

the matrix

Similarly to what was done for the Hammerstein model the closest, in the

2-norm sense, estimates are such that

DC

~~


Let have ranks>m, and let its economy size SVD

be partitioned as

)(~~ pnmrDC +

[ ]

===

=T

Ts

i

T

iii

T

V

VUUvuVUDC

2

1

2

1

21

1 0

0~~

with .( )mmpnmmr VU ,,,diagand,, 211

)(

11 L=+

Then

[ ]( ) [ ] ( ),,~~argmin, 1112

2,,

T

DC

VUDCDCDC =

= ++

+

and the approximation error is given by

[ ] .~~2

1

2

2

++ =

mDCDC

(8)

Result 3ResultResult 33

Normalization

in + provided

by the SVD


17/24


Identification AlgorithmIdentification AlgorithmIdentification Algorithm

The subspace algorithm can be summarized as follows.

StepStep 1:1: Compute estimates of the system matrices , and the

model ordern, using any available (linear) subspace algorithm, such as

N4SID, MOESP, CVA.StepStep 2:2: Compute the SVD of and its partition as in (8).

StepStep 3:3: Compute the estimates of the parameter matrices C, D, and + as

respectively.

( )DCBA ~,~,,

DC

~~

[ ]+=

=

1

11

U

VDC T


Simulation ExamplesSimulation ExamplesSimulation Examples

The True SystemThe True System

002.015.09.0

5.17.0)(

23

2

+++

+=

zzz

zzzG

linear subsystem

( ) 432 0263.05113.00149.08589.0 kkkkk uuuuuN += nonlinear subsystem

The input and noiseThe input and noise

input

( )( )

cos4.02.11064.0

8

=

Spectrum of the zero

mean coloured noise

Example 1: Hammerstein Model ID (academic)ExampleExample 1:1: Hammerstein ModelHammerstein Model ID (ID (academicacademic))

( ) ( )

( ) ( ) k

k

kk

kku

+++

++=

0035.0sin1.00025.0sin3.0

0015.0sin5.00005.0sin

( white noise withvariance )

k610


18/24


The Estimated Nonlinear SubsystemThe Estimated Nonlinear Subsystem

( ) 432 0260.05113.00142.08589.0 kkkkk uuuuuN += Estimated nonlinear subsystem

Fig.9: True (blue) and Estimated (green)

nonlinear characteristic.

The EstimatedThe Estimated LinearLinear SubsystemSubsystem

( )0014.01495.09002.0

4984.16997.09986.023

2

+++

+=

zzz

zzzG Estimated linear subsystem


Validation resultsValidation results

Fig. 10: True (green) and Estimated (blue) Output.


19/24


Example 2: Hammerstein Model ID (Binary Distillation Column)ExampleExample 2:2: Hammerstein ModelHammerstein Model ID (ID (Binary Distillation ColumnBinary Distillation Column))

Fig. 11: Schematic representation of thedistillation column

(Weischedel & McAvoy, 1980)

InputInput:: reflux ratio (u)

OutputsOutputs:: overhead flow rate (y1)

overhead methanol concentration (y2)

bottom flow rate (y3)

bottom methanol concentration (y4)


Fig. 12: Left Plot: Estimation (first 1000 points), and validation (remaining1000 points) Input Data. Right Plot: Estimation (first 1000 points) and

Validation (remaining 1000 points) Output Data.


20/24


Fig. 13: True (blue) and Estimated (red) Outputs (validation data)



Third order model with eigenvalues at { }9726.0,9557.0,4916.0


Third order polynomial

Fig. 14: Estimated Nonlinear

Characteristic


21/24


Example 3: Wiener Model ID (pH Neutralization Process)ExampleExample 3:3: Wiener ModelWiener Model ID (ID (pH Neutralization ProcesspH Neutralization Process))

Fig. 15: Schematic representation of the pHNeutralization Process

(Henson & Seborg, 92, 94, 97)

base:base: NaOH acid:acid: HNO3

buffer:buffer:NaHCO3

effluent solution

Manipulated variable:Manipulated variable: base

flow rate (u1)

Disturbances:Disturbances:buffer flow rate

(u2) and acid flow rate (u3)

Output:Output: pH of the effluent

solution (y)

(u3)


Simulation ModelSimulation Modelbased on first principlesfirst principles (introducing two reaction

invariants for each inlet stream)

0),(

)()()( 21

=

++=

yxh

uxpuxgxfx&

[ ] [ ]

( ) ( )

( ) ( )

( ) ( )

21

2

10101

10211010),(

1,

1)(

1,

1)(

,)(

,,

2

14

1

2212

2111

233

133

21

pKyypK

pKyyy

T

ba

T

ba

T

ba

Tba

T

xxyxh

xWV

xWV

xp

xWV

xWV

xg

xWV

uxW

V

uxf

WWxxx

++

+++=

=

=

=

==where


22/24


Fig. 16: Estimation (first 1000 points) and validation

(remaining 600 points) input-output data.

Estimation and Validation dataEstimation and Validation data



Third order model ( )9474.08940.29466.2

006.00122.00062.023

2

+

+=

zzz

zzzG

Third order polynomial

kkkk yyyyN 9989.00358.00319.0)( 231 ++=


Fig. 17: Estimated Nonlinear Characteristic.


23/24


Validation resultsValidation results

Fig. 18: True (blue) and estimated (red) Output (Estimation/Validation data).


ConclusionsConclusionsConclusions

New subspace methods for the simultaneous identification of thelinear and nonlinear parts of multivariable Hammerstein andmultivariable Hammerstein andWiener modelsWiener models have been presented.

The proposed methods make use of a standard (linear) subspacemethod followed by a 2-norm minimization problem which is

solved via an SVD. The proposed methods generalizegeneralize all the families of linear

subspace methods to this class of nonlinear models.

The method provides consistent estimatesconsistent estimates under the sameconditions on persistency of excitation required by the (linear)subspace method used as the first step of the algorithm.

The estimated models are in a format which is suitable for theiruse in standard (linear) Model Predictive Control schemes.


24/24


Subspace Identification of

Hammerstein and Wiener Models

Subspace Identification ofSubspace Identification of

Hammerstein and Wiener ModelsHammerstein and Wiener Models

Speaker

Juan C. Gmez

Laboratory for System Dynamics and Signal ProcessingFCEIA, Universidad Nacional de Rosario

ARGENTINA

[email protected]

Research SeminarResearch Seminar

Seminar LSD 2005 Bw

Documents

Transcript of Seminar LSD 2005 Bw