Post on 04-Jun-2018
8/14/2019 IRP OK.pdf
1/104
Inventory Routing
Problems
Martin Savelsbergh
8/14/2019 IRP OK.pdf
2/104
Goals
Introduce Inventory Routing Problems
Introduce Solution Approaches forInventory Routing Problems
Introduce Inventory Routing Game
8/14/2019 IRP OK.pdf
3/104
Inventory Routing
Inventory Management
Vehicle Routing
C ti l I t
8/14/2019 IRP OK.pdf
4/104
Conventional Inventory
Management Customer
monitors inventory levels places orders
Vendor
manufactures/purchases product assembles order
loads vehicles
routes vehicles makes deliveries
You call We haul
P bl ith C ti l
8/14/2019 IRP OK.pdf
5/104
Problems with Conventional
Inventory Management Large variation in demands on
production and transportationfacilities
workload balancing
utilization of resources unnecessary transportation
costs
urgent vs nonurgent orders setting priorities
8/14/2019 IRP OK.pdf
6/104
Vendor Managed Inventory
Customer
trusts the vendor to manage
the inventory Vendor
monitors customers inventory
customers call/fax/e-mail
remote telemetry units
set levels to trigger call-in
controls inventory
replenishment & decides when to deliver
how much to deliver
how to deliverYou rely We supply
8/14/2019 IRP OK.pdf
7/104
Vendor Managed Inventory
VMI transfers inventory management (and
possibly ownership) from the customer to thesupplier
VMI synchronizes the supply chain through the
process of collaborative order fulfillment
8/14/2019 IRP OK.pdf
8/104
Advantages of VMI
Customer
less resources for inventory
management assurance that product will be
available when required
Vendor
more freedom in when & how to
manufacture product and make deliveries
better coordination of inventory levels atdifferent customers
better coordination of deliveries to
decrease transportation cost
8/14/2019 IRP OK.pdf
9/104
Inventory Routing
Decide when to deliver to a customer
Decide how much to deliver to a customer
Decide on the delivery routes
8/14/2019 IRP OK.pdf
10/104
Inventory Routing
Inventory Management
Vehicle Routing
Long-Term Problem
8/14/2019 IRP OK.pdf
11/104
Inventory Costs
Transportation Costs
+
8/14/2019 IRP OK.pdf
12/104
Single Customer (Single Link)
Inventory cost Transportation cost
AAA B Determine shippingpolicies that optimizethe trade-offbetween:
transp.cost
inv. cost
8/14/2019 IRP OK.pdf
13/104
Problem Description
A B
Fleet of vehicles:- transportation capacity- transportation cost (ABA)
1=rc
0 1 2 3 4 5
8/14/2019 IRP OK.pdf
14/104
Problem Description
A B
Volume produced in A per unit time: v Volume consumed in B per unit time: v Inventory cost per unit time: h
8/14/2019 IRP OK.pdf
15/104
Goal
A B
Determine shipping policies
that minimize
inventory cost + transportation cost
8/14/2019 IRP OK.pdf
16/104
The Continuous Variant
Single frequency f Continuous time between shipments t = 1/f Single vehicle
Optimal solution
vt rt 0
min hvt + ct
t
= min(p c
hv ,
r
v )
8/14/2019 IRP OK.pdf
17/104
Minimum Intershipment Times
A practical constraint:Minimum intershipment times, e.g., 1 day
ZIO (Zero Inventory Ordering)
FBPS (Frequency Based Periodic Shipping)
8/14/2019 IRP OK.pdf
18/104
Zero Inventory Policy
Minimum inter-shipment time Single frequency f
Continuous time between shipments t= 1/f
A shipment is performed when the inventory
level of the products is zero
*t *t
= v
v
h
vct ,,1maxmin*
Frequency Based Periodic
8/14/2019 IRP OK.pdf
19/104
Frequency Based Periodic
Shipping Policies
Minimum intershipment time
One or more frequencies Integer time between consecutive shipments
0 1 2 Single frequency
Double frequency
8/14/2019 IRP OK.pdf
20/104
Best Single Frequency Policy
1/k*: best single frequency
Lemma: ( )
= 1,max*1h
cvvkk
+=
vkk
chkz
kk
Best
1
SF min
0 1 2
8/14/2019 IRP OK.pdf
21/104
Best Double Frequency Policy
1/k1*,1/ k2*: best frequencies
Lemma:
...k*k*k
...k*k
= T :vT(d) =PT
j=1pj(vTj(d) +S)
Probability that stockout occurs on dayj
Stockout cost
Probability that no stockout occurs
Delivery cost
vT(d) =(d) +(d)T +f(T, d)
(d)constant, f(T,d)goes to zero exponentially fast as T
(d) = pS+(1p)cPdj=1jpj
Single Customer Problem
8/14/2019 IRP OK.pdf
36/104
Single Customer Problem
An optimal constant replenishment period strategy over alarge T-day planning period will correspond to choosing d*to
minimize (d)
(d) = pS+(1p)cPdj=1jpj
Expected number of daysbetween deliveries
Expected cost of a delivery
Best d-day policy
8/14/2019 IRP OK.pdf
37/104
Best d-day policy
Demand: uniformly in [1,20]Deliver cost: 40Stockout cost: 50
Optimal policy
8/14/2019 IRP OK.pdf
38/104
Optimal policy
8/14/2019 IRP OK.pdf
39/104
Two Customer Problem
8/14/2019 IRP OK.pdf
40/104
Two Customer Problem
Stochastic policy:
Storage capacity: 20
P[demand = 0] = 0.4, P[demand = 10] = 0.6
Shortage penalty: 1000 customer 1; 1005 customer 2
Vehicle capacity: 10 Individual routes: 120; Combined route: 180
Infinite horizon Markov Decision Process Minimize expected total discounted cost
8/14/2019 IRP OK.pdf
41/104
Bounds
8/14/2019 IRP OK.pdf
42/104
Bounds
Customer usage during period: ui
Customer storage capacity: Ci
Vehicle capacity: Q
Minimize total mileage D*
subject to
Total volume delivered to customer i ui
Maximum volume delivered per trip Q
Maximum quantity delivered to customer i min(Ci, Q)
8/14/2019 IRP OK.pdf
43/104
Two Customer Analysis
8/14/2019 IRP OK.pdf
44/104
Two Customer Analysis
1 201t
12
t
120102 ttt +=
QC >1
QC
8/14/2019 IRP OK.pdf
45/104
Two Customer Analysis
Plant
Customer01t
12t
02t
QC 20
1
2
Assume :
Patterns:
(C1,0)
(0,Q)
(C1,Q-C1)
Extra mileage
Improved Bounds
8/14/2019 IRP OK.pdf
46/104
p o ed ou ds
Delivery Patterns
Pj = (dj1,dj2, ..., djn) is feasible if
i Idji Q 0 dji Ci i I .and Pj) = {iI : dji > 0} : Customers visited inPjc(Pj) : The cost of delivery patternPj - optimal TSP value
: Set of all feasible delivery patterns
Pattern Selection LP
D* =minPj c(Pj) xjs.t. Pj dji xj ui, i I
xj
0xj : How many times
should patternPj be
used
Obstacles
8/14/2019 IRP OK.pdf
47/104
Obstacle I : Infinite number of feasibledelivery patterns
Obstacle II : The calculation of the cost of
each delivery pattern involves the solution of a
traveling salesman problem
Obstacle I
8/14/2019 IRP OK.pdf
48/104
Base Pattern
A feasible delivery pattern P is a base pattern
if at most one customer, say k, in (P) receives adelivery quantity less than min(Ck, Q), and, in thatcase, the delivery quantity is Q i (P)\k Ci
Theorem
The base patterns are sufficient to find an optimal
solution to the Pattern Selection LP
Number of columns of Pattern Selection LP is finite
Obstacle II
8/14/2019 IRP OK.pdf
49/104
Focus on upper and lower bounds on D* instead of D* itself.
Lower bound (LBk) :If Ci < Q/k, then assume Ci = Q/k
LB1 LB2 D*If mini I(Ci) Q/k, then LBk = D*
|(P)| k for any base pattern PUpper bound (UBk) :
At most k stops in a tour
For values k=3 and
k=4, the TSPs that
have to be solved
involve at most 4 and
5 stops, respectively,
and thus can be solved
relatively easily by
enumeration
UB1 UB2 D*If Q/mini I(Ci) k, then UBk = D*|(P)| k for any base pattern P
Dominance
8/14/2019 IRP OK.pdf
50/104
Do we need base patternP ?
If z c(P),
then the base patterns with j > 0 collectively dominate P
Base pattern P can be eliminated from the Pattern Selection LP
Simple Dominance
8/14/2019 IRP OK.pdf
51/104
p
Consider base patternP withd4
8/14/2019 IRP OK.pdf
52/104
ApproachSpecialized solver for LPs with a large ratio of
number of columns to number of rows
A Partial LP
Subset of full
set of columns
Check reduced
costs for the
remaining columns
Optimal
solution
Instance n # of patterns # of iterations default(sec) sifting(sec)
1 136 3,029,980 5 85.66 77.09
2 157 4,336,466 6 118.37 121.34
3 169 5,420,907 6 155.29 152.56
4 147 8,838,137 5 360.20 240.83
5 157 8,975,615 6 397.33 254.81
6 194 16,640,122 6 675.98 533.56
Computational Experiments
8/14/2019 IRP OK.pdf
53/104
36 plants, ~2000 customers
2500000
2900000
3300000
3700000
4100000
1 2 3 4k
UB
LB
Inventory Routing
8/14/2019 IRP OK.pdf
54/104
Schedule 1: 420 miles per day
10
100
10
140
3
4
100
100
100
1 2
(1000,3000,0,0),(0,0,2000,1500)
Schedule 2: 380 miles per day(0,3000,2000,0)(2000,3000,0,0), (0,0,2000,3000)
Pattern Selection LP with T=1day
Optimal Objective Value : 380
0.5 : (0,3000,2000,0)
0.5 : (2000,3000,0,0)0.5 : (0,0,2000,3000)
Pattern Selection LP found schedule 2
and it shows no better schedule exists!Q = 5000
Solution Approaches
8/14/2019 IRP OK.pdf
55/104
Deterministic
Based on average product usage Stochastic
Based on probability distribution of product
usage
Deterministic SolutionApproach
8/14/2019 IRP OK.pdf
56/104
Approach
Two Phase Approach Phase I: Determine which customers should
receive a delivery on each day of the planning
period and how much
Phase II: Create the precise delivery routes for
each day
Rolling horizon approach
Deterministic SolutionApproach
8/14/2019 IRP OK.pdf
57/104
Approach
Two Phase ApproachPhase I: Integer program
Phase II: Insertion heuristic
Integer Program
8/14/2019 IRP OK.pdf
58/104
Lower bound on the total volume that has to be delivered
to customer i by the end of day t:
Upper bound on the total volume that can be delivered
to customer i by the end of day t:
Delivery constraint:
Integer Program
8/14/2019 IRP OK.pdf
59/104
Resource constraints:
Vehicle capacity
Number of vehicles
Integer Program
8/14/2019 IRP OK.pdf
60/104
Vehicle capacity
Number of vehicles
Storage capacity
Integer Program
8/14/2019 IRP OK.pdf
61/104
Improve the efficiency by
Route elimination
Aggregation
Insertion Heuristic
8/14/2019 IRP OK.pdf
62/104
Input for next kdays:
List of customers
List of recommendeddelivery amounts
Output for next k days for each vehicle:
Start time
Sequence of deliveries
Arrival time at each customerActual delivery amount at each customer
Key Issue
8/14/2019 IRP OK.pdf
63/104
How to handle variable delivery quantities?
We may be able to increase delivery amounts We may be able to decrease delivery
amounts
We may be able to postpone deliveries to
another day
Insertion Heuristic
8/14/2019 IRP OK.pdf
64/104
Minimum delivery volume:
Amount suggested bythe integer program
Earliest time a delivery can be made:
Insertion Heuristic
8/14/2019 IRP OK.pdf
65/104
Latest time a delivery can be made:
Maximum delivery volume:
Insertion Heuristic
8/14/2019 IRP OK.pdf
66/104
For each route:
Earliest time a route can start Latest time a route can start
Earliest time a route can end
Latest time a route can end
Sum of minimum deliveries
Sum of maximum deliveries
Insertion Heuristic
8/14/2019 IRP OK.pdf
67/104
Feasibility check:
Compute minimum delivery volume. Will theminimum delivery volume fit given the other
deliveries?
Compute earliest and latest delivery can takeplace. Is late greater than early?
Compute maximum delivery volume. Is
minimum less than maximum?
Delivery Volume Optimization
8/14/2019 IRP OK.pdf
68/104
Observe:
The amount that can be delivered at a customer
depends on the time at which the delivery starts The time it takes to make the delivery depends on the
size of the delivery
There is a limit on the elapsed time of a route
Result:
It is nontrivial to determine, given a route, i.e., a
sequence of customer visits, what the maximumamount of product is that can be delivered on this
route !!
Delivery Volume Optimization
8/14/2019 IRP OK.pdf
69/104
Tank capacity or Truck capacity
Earliest delivery time Latest delivery time
Usage rate Pump rate
Delivery Volume Optimization
8/14/2019 IRP OK.pdf
70/104
There is a polynomial time algorithm that solves this
problem. The algorithm constructs a series of piecewiselinear graphs (one for each customer on the route)
representing the maximum amount of product that can
be delivered on the remainder of the route as a functionof the start time of the delivery at the customer.
Delivery Volume Optimization
8/14/2019 IRP OK.pdf
71/104
Customer 1
Customer 2
Delivering a little less atCustomer 1 allows a much
larger delivery at Customer 2
Pump time +Travel time
8/14/2019 IRP OK.pdf
72/104
Stochastic IRP
8/14/2019 IRP OK.pdf
73/104
Determine
inventory
levels
Assign
customersto vehicles
Deliver at
customer &drive back
Load to
capacity & drive
to customer
Markov Decision ProcessModel
8/14/2019 IRP OK.pdf
74/104
State, x
inventory levels at different customers Action, a
Which customers to replenish
How much to deliver at each customer
How to combine customers into vehicle routes
Objective
Solving Problems Exactly
8/14/2019 IRP OK.pdf
75/104
Algorithm: Policy Iteration
For each problem
Customer capacity 10 units
Customer demand 1, , 10 w.p. 0.1 each
Vehicle capacity 5 units
Direct delivery only
MDP Model: Issues
8/14/2019 IRP OK.pdf
76/104
Optimality Equation
Computing optimal value function
Computing expected value
Computing optimal action
V(x) = maxaA(x)
E[g(x, a) + V(Xt+1
)|Xt=x, A
t=a]
Approximation Methods
8/14/2019 IRP OK.pdf
77/104
Idea
Approximate V with V
Motivation
Parameterized approximation function
Examples for Basis Functions
8/14/2019 IRP OK.pdf
78/104
Polynomial function inventory level at customers
second order effects
4450
4650
4850
5050
5250
0 1 2 3 4 5 6 7 8 9 10
Inventory at customer 3 (X3)
Value
opt, X2 = 0
opt, X2 = 5
opt, X2 = 10
app, X2 = 0
app, X2 = 5
app, X2 = 10
Approximating the ValueFunction
8/14/2019 IRP OK.pdf
79/104
Although IRP is not separable, the major
costs (including transportation) are
associated with small groups of
customers (vehicle routes)
We do not know in advance whichgroups will be in each vehicle route
We can identify subsets of customers
that can possibly be in the same vehicle
route
MDP for subset of customers
8/14/2019 IRP OK.pdf
80/104
State, (xi, ki)
inventory at customers vehicleswhich could be allocated
Action, ai
deliveries to customers in the subset
Transition probability
MDPs for small subsets of customers can
be solved optimally in advance
Approximating the ValueFunction
8/14/2019 IRP OK.pdf
81/104
In advance, optimally solve problems for subsets ofcustomers
On each day, partition the customers and vehicles into
subsets by solving a cardinality constrained partitioning
problem
1-customer subsets: nonlinear knapsack problem
2-customer subsets: maximum weight perfect matching
problem
Non-linear Knapsack Problem
8/14/2019 IRP OK.pdf
82/104
Vehicles
Customers
0,0
2,0
1,1
2,1
1,0
0,1 0,3
2,3
1,3
0,2
1,2
2,2
0
V1(x1,0)
V1(x1,0)
V1(x1,0)
V1 (
x1 ,2
)
V2(x2,1
)
V2(x2,0)
V2(x2,0)
V3
(x3
,0)
V3 (
x3 ,2
)
0
0
V3 (x
3 ,1)
V3(x3,0)0 0
0
0
0
0
0
0
0
Computing ParametersMethod I
8/14/2019 IRP OK.pdf
83/104
Objective Function
Looks like weightedleast squares
regression problem
Cannot be
computed for
large problems
Computing ParametersStochastic Approximation Algorithm
8/14/2019 IRP OK.pdf
84/104
Simulate system under policy Sample path x0, x1, , xt, ...
Update coefficients
Step size
Temporal difference
Eligibility vector
t =g(xt,(xt)) + V(xt+1, rt) V(xt, rt)
Pt=0
t = ,P
t=0(t)2 <
zt+1 =zt + OrV(xt, rt)
Convergence typically very slow
Computing ParametersMethod II
8/14/2019 IRP OK.pdf
85/104
Value function for policy
Looks like weighted least squares
regression problem
Cannot be
computed for
large
problems
Computing ParametersKalman Filter Algorithm
8/14/2019 IRP OK.pdf
86/104
Simulate system under policy
Sample path x0, x1, , xt, ...
Update matrices Mt (similar to XX) & Yt
(similar to XY)
rt is the solution of Mtrt = Yt
Convergence significantly faster
Computing ParametersStoch App vs. Kalman F + Stoch App
8/14/2019 IRP OK.pdf
87/104
0
2
4
6
8
0:00 0:09 0:18 0:28 0:37 0:46 0:56 1:05 1:14 1:24 1:33
Time (hours)
Para
meters
Customer 1
Customer 2
Customer 1
Customer 2StochasticA
pp
StochasticApp
KalmanF
Estimating Expected Value
8/14/2019 IRP OK.pdf
88/104
Multi-dimensional Integral
d = #dimensions = #customers
very hard to compute Deterministic Methods
MSE = O(n2 - 2c/d)
Randomized Methods
MSE = O(1/n)
Deterministic methods are better when 2 - 2c/d < -1
Randomized methods are
better for large d
Choosing the Best Action
8/14/2019 IRP OK.pdf
89/104
Based on sample averages of actions
Question
How large should the sample be so that we are reasonablysure of choosing the best action?
Nelson and Matejcik (1995)
Sample size to ensure chosen alternative has value withintolerance of best value with specified probability
Variance reduction methods
Common random numbers
Orthogonal arrays
Variance Reduction MethodsNumber of Observations for Choosing Best Action
8/14/2019 IRP OK.pdf
90/104
0
400
800
1200
1600
2000
1 101 201 301 401 501 601 701 801 901Simulation Steps
NumberofOb
servations
OA
Random
Approximate Policy Iteration
8/14/2019 IRP OK.pdf
91/104
1. Initialization. Simulate initial policy 0 and obtain parameters r0
2. Use parameters rt1 for policy t
and obtain actions using
3. Simulate policy t and obtain parameters rt
4. t t + 1; go to Step 2
Performance ComparisonSmall Instances
8/14/2019 IRP OK.pdf
92/104
9.4
9.6
9.8
10
10.2
10.4
10.6
10.8
0 1 2 3 4 5 6 7 8 9 10
Inventory at customer 3 (X3)
Value*10-3
Optimal value
Policy 1
Policy 2
Policy 3
CBW policy
Performance ComparisonLarge Instances
8/14/2019 IRP OK.pdf
93/104
Price-Direct Replenishment
8/14/2019 IRP OK.pdf
94/104
A control policy based on a simple
economic mechanism for dispatching
The dispatcher receives a transfer price
diV
ifrom management for replenishing d
i
units of product at customer i.
The dispatcher is responsible for paying
the distribution costs cI, when replenishinga set of customers I.
Price-Direct Replenishment
8/14/2019 IRP OK.pdf
95/104
Net value for dispatcher
Incremental value for dispatcher
PiIVidi cI
diVi (cI{i}cI)
Price-Directed Replenishment
8/14/2019 IRP OK.pdf
96/104
Managements problem: Set Viso that the
dispatcher is motivated to minimize the
long-run time average replenishment costs
Price-Direct Replenishment
8/14/2019 IRP OK.pdf
97/104
Management problem (single customer):
Primal Dual
min cZ
dZ =u
0 d min{C, Q}
0 Z
max uVdV c 0 d min{C, Q}
replenishment
frequencyusage
Price-Direct Replenishment
8/14/2019 IRP OK.pdf
98/104
Management problem (single customer):
Dual
max uVdV c 0 d min{C, Q}
If Vis interpreted as the transferprice received by the dispatcher
for replenishing one unit, thenthis dual program maximizes therate at which transfer revenueaccumulates, subject to theconstraint that the total transfer
payment cannot exceed the coston any replenishment
Direct Replenishment
8/14/2019 IRP OK.pdf
99/104
Price directed operating policy maximizing
the net value of a replenishment
max0dmin{C,Q}{Vd c}
References
P J ill t J B d L H M D (2002) D li t
8/14/2019 IRP OK.pdf
100/104
P. Jaillet, J. Bard, L. Huang, M. Dror (2002). Delivery costapproximations for inventory routing problems in a rollinghorizon framework. TS 36, 292-300.
A. Campbell, M. Savelsbergh (2004). A decompositionapproach for the inventory routing problem. TS 38, 488-502.
A. Campbell, M. Savelsbergh (2004). Delivery volumeoptimization. TS 38, 210-223.
A. Kleywegt, V. Nori, M. Savelsbergh (2002). The stochasticinventory routing problem with direct deliveries. TS 36, 94-118.
A. Kleywegt, V. Nori, M. Savelsbergh (2004). Dynamic
programming approximations for a stochastic inventoryrouting problem. TS 38, 42-70.
J.-H. Song, M. Savelsbergh (2006). Performancemeasurement for inventory routing. TS. To appear.
References
L B t i M G S (2002) C ti d di t
8/14/2019 IRP OK.pdf
101/104
L. Bertazzi, M.G. Speranza (2002). Continuous and discrete
shipping strategies for the single link problem. TS 36, 314-
325.
L. Bertazzi, G. Palletta, M.G. Speranza (2002). Deterministic
order-up-to level policies in an inventory routing problem. TS
36, 119-132.
L. Bertazzi, G. Palletta, M.G. Speranza (2005). Minimizing thetotal cost in an integrated vendor-managed inventory system.
JH 11, 393-419.
References
D Adelman (2003) Price directed replenishment of s bsets
8/14/2019 IRP OK.pdf
102/104
D. Adelman (2003). Price-directed replenishment of subsets:methodology and its application to inventory routing. MSOM5, 348-371.
V. Gaur, M. Fisher (2004). A periodic inventory routingproblem at a supermarket chain. OR 52, 813-822.
A. Campbell, J. Hardin (2005). Vehicle minimization forperiodic deliveries. EJOR 165, 668-684.
W. Bell, L. Dalberto, M. Fisher, A. Greenfield, R, Jaikumar, P.Kedia, R. Mack, P. Prutzman (1983). Improving thedistribution of industrial gases with an online computerized
routing and scheduling optimizer. Interfaces 13, 4-23.
Inventory Routing Game
htt //k i t h d 8081/IRG
http://kronos.isye.gatech.edu:8081/IRGamehttp://kronos.isye.gatech.edu:8081/IRGame8/14/2019 IRP OK.pdf
103/104
http://kronos.isye.gatech.edu:8081/IRGame
Login: player1, , player20
Password: player1, , player20
Play Instance 3
Winner gets prize on Friday
http://kronos.isye.gatech.edu:8081/IRGamehttp://kronos.isye.gatech.edu:8081/IRGame8/14/2019 IRP OK.pdf
104/104
Questions?