Op Tim Ization Uw 06

download Op Tim Ization Uw 06

of 29

Embed Size (px)

Transcript of Op Tim Ization Uw 06

  • 7/30/2019 Op Tim Ization Uw 06

    1/29

    Introduction to Optimization

    Anjela Govan

    North Carolina State University

    SAMSI NDHS Undergraduate workshop 2006

  • 7/30/2019 Op Tim Ization Uw 06

    2/29

    What is Optimization?

    Optimization is the mathematical disciplinewhich is concerned with finding the maxima

    and minima of functions, possibly subject to

    constraints.

  • 7/30/2019 Op Tim Ization Uw 06

    3/29

    Where would we use optimization?

    Architecture

    Nutrition

    Electrical circuits

    Economics

    Transportation

    etc.

  • 7/30/2019 Op Tim Ization Uw 06

    4/29

    What do we optimize?

    A real function of n variables

    with or without constrains

    ),,,(21 n

    xxxf

  • 7/30/2019 Op Tim Ization Uw 06

    5/29

    Unconstrained optimization

    22 2),(min yxyxf

  • 7/30/2019 Op Tim Ization Uw 06

    6/29

    Optimization with constraints

    2

    2),(min

    1,52

    2),(min

    0

    2),(min

    22

    22

    22

    or

    or

    yx

    yxyxf

    yx

    yxyxf

    x

    yxyxf

  • 7/30/2019 Op Tim Ization Uw 06

    7/29

    Lets Optimize

    Suppose we want to find the minimum of the

    function

  • 7/30/2019 Op Tim Ization Uw 06

    8/29

    Review max-min forR2

    What is special about a local max or a local

    min of a function (x)?

    at local max or local min (x)=0

    (x) > 0 if local min

    (x) < 0 if local max

  • 7/30/2019 Op Tim Ization Uw 06

    9/29

    Review max-min forR3

  • 7/30/2019 Op Tim Ization Uw 06

    10/29

    Review max-min forR3

    Second Derivative Test

    Local min, local max, saddle point

    Gradient of vector (d dx d /dy d /dz)

    direction of fastest increase of

    Global min/max vs. local min/max

  • 7/30/2019 Op Tim Ization Uw 06

    11/29

    Gradient Descent Method Examples

    Minimize function

    11,11

    )(5.0),(22

    yx

    yxyxf

    Minimize function

    4,4)cos()cos(),(

    yxyxyxf

  • 7/30/2019 Op Tim Ization Uw 06

    12/29

    Gradient Descent Method Examples

    Use function gd(alpha,x0) Does gd.m converge to a local min? Is there a

    difference if > 0 vs. < 0?

    How many iterations does it take to converge to alocal min? How do starting points x0 affectnumber of iterations?

    Use function gd2(x0)

    Does gd2.m converge to a local min? How do starting points x0 affect number of

    iterations and the location of a local minimum?

  • 7/30/2019 Op Tim Ization Uw 06

    13/29

    How good are the optimization methods?

    Starting point

    Convergence to global min/max.

    Classes of nice optimization problems

    Example: f(x,y) = 0.5(x2+y2), > 0

    Every local min is global min.

  • 7/30/2019 Op Tim Ization Uw 06

    14/29

    Other optimization methods

    Non smooth, non differentiable surfaces

    can not compute the gradient of

    can not use Gradient Method

    Nelder-Mead Method

    Others

  • 7/30/2019 Op Tim Ization Uw 06

    15/29

    Convex Hull

    A set C is convex ifevery point on the line

    segment connecting xand y is in C.

    The convex hull for aset of points X is the

    minimal convex setcontaining X.

  • 7/30/2019 Op Tim Ization Uw 06

    16/29

    Simplex

    A simplex orn-simplex isthe convex hull of a set of

    (n+1) . A simplex is an n-dimensional analogue of a

    triangle.

    Example:

    a 1-simplex is a line segment

    a 2-simplex is a triangle a 3-simplex is a tetrahedron

    a 4-simplex is a pentatope

  • 7/30/2019 Op Tim Ization Uw 06

    17/29

    Nelder-Mead Method

    n = number of variables, n+1 points

    form simplex using these points; convex hull

    move in direction away from the worst of

    these points: reflect, expand, contract, shrink

    Example:

    2 variables 3 points simplex is triangle

    3 variables 4 points simplex is tetrahedron

  • 7/30/2019 Op Tim Ization Uw 06

    18/29

    Nelder-Mead Methodreflect, expand

  • 7/30/2019 Op Tim Ization Uw 06

    19/29

    Nelder-Mead Method-reflect, contract

  • 7/30/2019 Op Tim Ization Uw 06

    20/29

    A tour of Matlab: Snapshots from the minimization

    After 0 steps

  • 7/30/2019 Op Tim Ization Uw 06

    21/29

    A tour of Matlab: Snapshots from the minimization

    After 1 steps

  • 7/30/2019 Op Tim Ization Uw 06

    22/29

    A tour of Matlab: Snapshots from the minimization

    After 2 steps

  • 7/30/2019 Op Tim Ization Uw 06

    23/29

    A tour of Matlab: Snapshots from the minimization

    After 3 steps

  • 7/30/2019 Op Tim Ization Uw 06

    24/29

    A tour of Matlab: Snapshots from the minimization

    After 7 steps

  • 7/30/2019 Op Tim Ization Uw 06

    25/29

    A tour of Matlab: Snapshots from the minimization

    After 12 steps

  • 7/30/2019 Op Tim Ization Uw 06

    26/29

    A tour of Matlab: Snapshots from the minimization

    After 30 steps (converged)

  • 7/30/2019 Op Tim Ization Uw 06

    27/29

    fminsearch function

    parameters: q =[C,K]

    cost function:

    Minimize cost function

    [q,cost]=

    fninsearch(@cost_beam, q0,[],time,y_tilde)

    2N

    1ii

    )y_tilde]),[,(y(tcosti

    KC

  • 7/30/2019 Op Tim Ization Uw 06

    28/29

    Our optimization problem

    In our problem

    Our function:

    cost function lives in R3

    2 parameters C and K, n=2

    Simplex is a triangle

    2N

    1ii

    )y_tilde]),[,(y(tcosti

    KC

  • 7/30/2019 Op Tim Ization Uw 06

    29/29

    Done!