of 29

### Transcript of Op Tim Ization Uw 06

• 7/30/2019 Op Tim Ization Uw 06

1/29

Introduction to Optimization

Anjela Govan

North Carolina State University

• 7/30/2019 Op Tim Ization Uw 06

2/29

What is Optimization?

Optimization is the mathematical disciplinewhich is concerned with finding the maxima

and minima of functions, possibly subject to

constraints.

• 7/30/2019 Op Tim Ization Uw 06

3/29

Where would we use optimization?

Architecture

Nutrition

Electrical circuits

Economics

Transportation

etc.

• 7/30/2019 Op Tim Ization Uw 06

4/29

What do we optimize?

A real function of n variables

with or without constrains

),,,(21 n

xxxf

• 7/30/2019 Op Tim Ization Uw 06

5/29

Unconstrained optimization

22 2),(min yxyxf

• 7/30/2019 Op Tim Ization Uw 06

6/29

Optimization with constraints

2

2),(min

1,52

2),(min

0

2),(min

22

22

22

or

or

yx

yxyxf

yx

yxyxf

x

yxyxf

• 7/30/2019 Op Tim Ization Uw 06

7/29

Lets Optimize

Suppose we want to find the minimum of the

function

• 7/30/2019 Op Tim Ization Uw 06

8/29

Review max-min forR2

What is special about a local max or a local

min of a function (x)?

at local max or local min (x)=0

(x) > 0 if local min

(x) < 0 if local max

• 7/30/2019 Op Tim Ization Uw 06

9/29

Review max-min forR3

• 7/30/2019 Op Tim Ization Uw 06

10/29

Review max-min forR3

Second Derivative Test

Local min, local max, saddle point

Gradient of vector (d dx d /dy d /dz)

direction of fastest increase of

Global min/max vs. local min/max

• 7/30/2019 Op Tim Ization Uw 06

11/29

Minimize function

11,11

)(5.0),(22

yx

yxyxf

Minimize function

4,4)cos()cos(),(

yxyxyxf

• 7/30/2019 Op Tim Ization Uw 06

12/29

Use function gd(alpha,x0) Does gd.m converge to a local min? Is there a

difference if > 0 vs. < 0?

How many iterations does it take to converge to alocal min? How do starting points x0 affectnumber of iterations?

Use function gd2(x0)

Does gd2.m converge to a local min? How do starting points x0 affect number of

iterations and the location of a local minimum?

• 7/30/2019 Op Tim Ization Uw 06

13/29

How good are the optimization methods?

Starting point

Convergence to global min/max.

Classes of nice optimization problems

Example: f(x,y) = 0.5(x2+y2), > 0

Every local min is global min.

• 7/30/2019 Op Tim Ization Uw 06

14/29

Other optimization methods

Non smooth, non differentiable surfaces

can not compute the gradient of

Others

• 7/30/2019 Op Tim Ization Uw 06

15/29

Convex Hull

A set C is convex ifevery point on the line

segment connecting xand y is in C.

The convex hull for aset of points X is the

minimal convex setcontaining X.

• 7/30/2019 Op Tim Ization Uw 06

16/29

Simplex

A simplex orn-simplex isthe convex hull of a set of

(n+1) . A simplex is an n-dimensional analogue of a

triangle.

Example:

a 1-simplex is a line segment

a 2-simplex is a triangle a 3-simplex is a tetrahedron

a 4-simplex is a pentatope

• 7/30/2019 Op Tim Ization Uw 06

17/29

n = number of variables, n+1 points

form simplex using these points; convex hull

move in direction away from the worst of

these points: reflect, expand, contract, shrink

Example:

2 variables 3 points simplex is triangle

3 variables 4 points simplex is tetrahedron

• 7/30/2019 Op Tim Ization Uw 06

18/29

• 7/30/2019 Op Tim Ization Uw 06

19/29

• 7/30/2019 Op Tim Ization Uw 06

20/29

A tour of Matlab: Snapshots from the minimization

After 0 steps

• 7/30/2019 Op Tim Ization Uw 06

21/29

A tour of Matlab: Snapshots from the minimization

After 1 steps

• 7/30/2019 Op Tim Ization Uw 06

22/29

A tour of Matlab: Snapshots from the minimization

After 2 steps

• 7/30/2019 Op Tim Ization Uw 06

23/29

A tour of Matlab: Snapshots from the minimization

After 3 steps

• 7/30/2019 Op Tim Ization Uw 06

24/29

A tour of Matlab: Snapshots from the minimization

After 7 steps

• 7/30/2019 Op Tim Ization Uw 06

25/29

A tour of Matlab: Snapshots from the minimization

After 12 steps

• 7/30/2019 Op Tim Ization Uw 06

26/29

A tour of Matlab: Snapshots from the minimization

After 30 steps (converged)

• 7/30/2019 Op Tim Ization Uw 06

27/29

fminsearch function

parameters: q =[C,K]

cost function:

Minimize cost function

[q,cost]=

fninsearch(@cost_beam, q0,[],time,y_tilde)

2N

1ii

)y_tilde]),[,(y(tcosti

KC

• 7/30/2019 Op Tim Ization Uw 06

28/29

Our optimization problem

In our problem

Our function:

cost function lives in R3

2 parameters C and K, n=2

Simplex is a triangle

2N

1ii

)y_tilde]),[,(y(tcosti

KC

• 7/30/2019 Op Tim Ization Uw 06

29/29

Done!