```Homogenization and other multi-scale problems
in Bellman-Isaacs equations
Martino Bardi
Dipartimento di Matemetica Pura ed Applicata
New Trends in Analysis and Control of Nonlinear PDEs
Roma, June 13–15, 2011
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
1 / 29
Plan
1
2
Homogenization of non-coercive Hamilton–Jacobi equations
(joint work with G. Terrone)
I
Some previous results
I
Examples of non-homogenization
I
Homogenization on subspaces
I
Convex-concave eikonal equations and differential games
Optimal control with stochastic volatility
(joint work with A. Cesaroni and L. Manca)
I
Motivation: financial models
I
Singular perturbations of H-J-B equations
I
Merton portfolio optimization with stochastic volatility
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
2 / 29
Plan
1
2
Homogenization of non-coercive Hamilton–Jacobi equations
(joint work with G. Terrone)
I
Some previous results
I
Examples of non-homogenization
I
Homogenization on subspaces
I
Convex-concave eikonal equations and differential games
Optimal control with stochastic volatility
(joint work with A. Cesaroni and L. Manca)
I
Motivation: financial models
I
Singular perturbations of H-J-B equations
I
Merton portfolio optimization with stochastic volatility
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
2 / 29
1. Homogenization of Hamilton–Jacobi Equations
Problem: let ε → 0+ in
utε + H x, xε , Du ε = 0
in (0, T ) × RN
u ε (0, x) = h x, xε .
ZN − periodic case:
H(x, ξ + k , p) = H(x, ξ, p),
h(x, ξ + k ) = h(x, ξ)
∀k ∈ ZN .
Goal: find continuous effective Hamiltonian and initial data H, h s.t.
uε → u
locally uniformly, and u solves
ut + H (x, Du) = 0
in (0, T ) × RN
u(0, x) = h (x) .
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
3 / 29
Classical result for coercive H
Coercivity in p is
lim H(x, ξ, p) = +∞
|p|→∞
uniformly in x, ξ.
P.-L. Lions - Papanicolaou - Varadhan ’86, L.C. Evans ’92:
The effective Hamiltonian H = H(x, p) is the unique constant such
that the cell problem, with x, p frozen parameters,
H(x, ξ, Dξ χ + p) = H,
in RN ,
has a ZN −periodic (viscosity) solution χ(ξ), called the corrector;
if h = h(x) is independent of xε , u ε → u locally uniformly.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
4 / 29
Classical result for coercive H
Coercivity in p is
lim H(x, ξ, p) = +∞
|p|→∞
uniformly in x, ξ.
P.-L. Lions - Papanicolaou - Varadhan ’86, L.C. Evans ’92:
The effective Hamiltonian H = H(x, p) is the unique constant such
that the cell problem, with x, p frozen parameters,
H(x, ξ, Dξ χ + p) = H,
in RN ,
has a ZN −periodic (viscosity) solution χ(ξ), called the corrector;
if h = h(x) is independent of xε , u ε → u locally uniformly.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
4 / 29
Classical result for coercive H
Coercivity in p is
lim H(x, ξ, p) = +∞
|p|→∞
uniformly in x, ξ.
P.-L. Lions - Papanicolaou - Varadhan ’86, L.C. Evans ’92:
The effective Hamiltonian H = H(x, p) is the unique constant such
that the cell problem, with x, p frozen parameters,
H(x, ξ, Dξ χ + p) = H,
in RN ,
has a ZN −periodic (viscosity) solution χ(ξ), called the corrector;
if h = h(x) is independent of xε , u ε → u locally uniformly.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
4 / 29
The effective initial data
If the initial data h = h(x, xε ) must find the initial condition h(x).
O. Alvarez - M. B. (ARMA ’03):
Assume there exists the recession function of H
H(x, ξ, λp)
.
λ→+∞
λ
H 0 (x, ξ, p) := lim
Consider, for frozen x,
If
then
wt + H 0 (x, ξ, Dξ w) = 0,
w(0, ξ) = h(x, ξ).
limt→+∞ w(t, ξ) = constant =: h(x),
limt→0 limε→0 u ε = h(x).
In the coercive case
h (x) = minξ h(x, ξ).
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
5 / 29
The effective initial data
If the initial data h = h(x, xε ) must find the initial condition h(x).
O. Alvarez - M. B. (ARMA ’03):
Assume there exists the recession function of H
H(x, ξ, λp)
.
λ→+∞
λ
H 0 (x, ξ, p) := lim
Consider, for frozen x,
If
then
wt + H 0 (x, ξ, Dξ w) = 0,
w(0, ξ) = h(x, ξ).
limt→+∞ w(t, ξ) = constant =: h(x),
limt→0 limε→0 u ε = h(x).
In the coercive case
h (x) = minξ h(x, ξ).
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
5 / 29
The effective initial data
If the initial data h = h(x, xε ) must find the initial condition h(x).
O. Alvarez - M. B. (ARMA ’03):
Assume there exists the recession function of H
H(x, ξ, λp)
.
λ→+∞
λ
H 0 (x, ξ, p) := lim
Consider, for frozen x,
If
then
wt + H 0 (x, ξ, Dξ w) = 0,
w(0, ξ) = h(x, ξ).
limt→+∞ w(t, ξ) = constant =: h(x),
limt→0 limε→0 u ε = h(x).
In the coercive case
h (x) = minξ h(x, ξ).
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
5 / 29
Bellman-Isaacs equations
In control theory the Hamiltonian is, for A compact,
H(x, ξ, p) = max {−f (x, ξ, a) · p − l (x, ξ, a)} ,
a∈A
ξ=
x
ε
and in 0-sum differential games it is, for B compact,
H(x, ξ, p) = min max {−f (x, ξ, a, b) · p − l (x, ξ, a, b)} .
b∈B a∈A
The associated control system in an oscillating periodic medium is
x
ẋ = f x, , a, b
ε
and cost - payoff
Z t x(t)
x(s)
, a(s), b(s) ds + h x(t),
J :=
l x(s),
ε
ε
0
that player a wants to minimize and player b to maximize.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
6 / 29
Bellman-Isaacs equations
In control theory the Hamiltonian is, for A compact,
H(x, ξ, p) = max {−f (x, ξ, a) · p − l (x, ξ, a)} ,
a∈A
ξ=
x
ε
and in 0-sum differential games it is, for B compact,
H(x, ξ, p) = min max {−f (x, ξ, a, b) · p − l (x, ξ, a, b)} .
b∈B a∈A
The associated control system in an oscillating periodic medium is
x
ẋ = f x, , a, b
ε
and cost - payoff
Z t x(t)
x(s)
, a(s), b(s) ds + h x(t),
J :=
l x(s),
ε
ε
0
that player a wants to minimize and player b to maximize.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
6 / 29
Alternative conditions to coercivity
coercivity of H is equivalent to Small Time Local Controllability in
a time proportional to the distance (by one of the players for any
control of the other),
it is a STRONG assumption, can be replaced by
Controllability in a uniformly Bounded Time (by one of the players
for any control of the other) (M.B. - Alvarez Mem. AMS 2010)
Example: hypoelliptic eikonal equation
utε +
k
X
i=1
|fi
x x
· Du ε | = l x,
ε
ε
in (0, T ) × RN
with f1 , ..., fk ∈ C ∞ satisfying Hörmander bracket-generating
condition.
Non-resonance conditions (Arisawa - Lions ’99, M.B. - Alvarez ’10
for control problems, Cardaliaguet ’10 for games).
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
7 / 29
Alternative conditions to coercivity
coercivity of H is equivalent to Small Time Local Controllability in
a time proportional to the distance (by one of the players for any
control of the other),
it is a STRONG assumption, can be replaced by
Controllability in a uniformly Bounded Time (by one of the players
for any control of the other) (M.B. - Alvarez Mem. AMS 2010)
Example: hypoelliptic eikonal equation
utε +
k
X
i=1
|fi
x x
· Du ε | = l x,
ε
ε
in (0, T ) × RN
with f1 , ..., fk ∈ C ∞ satisfying Hörmander bracket-generating
condition.
Non-resonance conditions (Arisawa - Lions ’99, M.B. - Alvarez ’10
for control problems, Cardaliaguet ’10 for games).
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
7 / 29
Alternative conditions to coercivity
coercivity of H is equivalent to Small Time Local Controllability in
a time proportional to the distance (by one of the players for any
control of the other),
it is a STRONG assumption, can be replaced by
Controllability in a uniformly Bounded Time (by one of the players
for any control of the other) (M.B. - Alvarez Mem. AMS 2010)
Example: hypoelliptic eikonal equation
utε +
k
X
i=1
|fi
x x
· Du ε | = l x,
ε
ε
in (0, T ) × RN
with f1 , ..., fk ∈ C ∞ satisfying Hörmander bracket-generating
condition.
Non-resonance conditions (Arisawa - Lions ’99, M.B. - Alvarez ’10
for control problems, Cardaliaguet ’10 for games).
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
7 / 29
Examples of non-homogenization
Example 1
utε + |uxε + uyε |β = cos 2π x−y
ε
u ε (0, x, y ) = 0
is solved by
x, y ∈ R (β ≥ 1)
u ε (t, x, y ) = t cos 2π x−y
.
ε
Example 2
utε + |uxε + uyε |β = 0
x, y ∈ R
u ε (0, x, y ) = cos 2π x−y
ε
is solved by
u ε (t, x, y ) = cos 2π x−y
.
ε
In both cases
u ε has no limit as ε → 0 if x 6= y .
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
8 / 29
Example 3
utε + |uxε | − |uyε | = cos 2π x−y
ε
u ε (0, x, y ) = 0
is solved by
x, y ∈ R
u ε (t, x, y ) = t cos 2π x−y
.
ε
Example 4
utε + |uxε | − |uyε | = 0 x, y ∈ R
u ε (0, x, y ) = 2π x−y
ε
is solved by
Again
u ε (t, x, y ) = cos 2π x−y
.
ε
u ε has no limit as ε → 0 if x 6= y .
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
9 / 29
H vanishing in a direction
The previous examples are special cases of
Proposition
Assume ∃ p0 ∈ ZN \ {0} such that
z ,
utε + H(Dz u ε ) = l
ε
H(λp0 ) = 0 ∀ λ ∈ R in
z u ε (0, z) = h
ε
If h = 0 ∃ smooth ZN −periodic l such that limε→0 u ε @
if l = 0 ∃ smooth ZN −periodic h such that limε→0 u ε @
The 1st statement is SHARP if H is 1-homogeneous and convex:
∀ p0 ∈ ZN \ {0} H(p0 ) 6= 0 is nonresonance condition
⇒ homogenization!
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
10 / 29
H vanishing in a direction
The previous examples are special cases of
Proposition
Assume ∃ p0 ∈ ZN \ {0} such that
z ,
utε + H(Dz u ε ) = l
ε
H(λp0 ) = 0 ∀ λ ∈ R in
z u ε (0, z) = h
ε
If h = 0 ∃ smooth ZN −periodic l such that limε→0 u ε @
if l = 0 ∃ smooth ZN −periodic h such that limε→0 u ε @
The 1st statement is SHARP if H is 1-homogeneous and convex:
∀ p0 ∈ ZN \ {0} H(p0 ) 6= 0 is nonresonance condition
⇒ homogenization!
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
10 / 29
Homogenization on subspaces
If H satisfies the assumption of the negative Proposition, must restrict
the data l, h to oscillate only in directions where H does not vanish.
Let V M–dimensional subspace of RN , zV := ΠV (z) projection
z
z z V
V
V
utε + H
, Dz u ε = l z,
, u ε (0, z) = h z,
ε
ε
ε
N
Assume coercivity in V : ∀ p̄ ∈ R
lim
|p|→+∞, p∈V
H(θ, p + p̄) = +∞,
uniformly in θ ∈ V .
Theorem
∃ H̄(z, p) continuous such that, ∀h(z, θ) periodic in θ, u ε converges
uniformly on compacta of (0, +∞) × RN to u, unique solution of
ut + H̄(z, Du) = 0,
u(0, z) = min h(z, θ)
Multi-scale Bellman-Isaacs
θ∈V
Roma, June 15th, 2011
11 / 29
Homogenization on subspaces
If H satisfies the assumption of the negative Proposition, must restrict
the data l, h to oscillate only in directions where H does not vanish.
Let V M–dimensional subspace of RN , zV := ΠV (z) projection
z
z z V
V
V
utε + H
, Dz u ε = l z,
, u ε (0, z) = h z,
ε
ε
ε
N
Assume coercivity in V : ∀ p̄ ∈ R
lim
|p|→+∞, p∈V
H(θ, p + p̄) = +∞,
uniformly in θ ∈ V .
Theorem
∃ H̄(z, p) continuous such that, ∀h(z, θ) periodic in θ, u ε converges
uniformly on compacta of (0, +∞) × RN to u, unique solution of
ut + H̄(z, Du) = 0,
u(0, z) = min h(z, θ)
Multi-scale Bellman-Isaacs
θ∈V
Roma, June 15th, 2011
11 / 29
Related results: Barles 2007, Viterbo 2008 (by symplectic methods).
Example 1
utε + |uxε + γuyε | = l x, y , x−y
ε
u ε (0, x, y ) = h x, y , x−y
ε
x, y ∈ RN/2
there is homogenization ∀ l, h periodic in θ =
x−y
ε
⇐⇒
γ 6= 1.
γ = 1 by the Examples 1 and 2 of non-homogenization;
γ 6= 1 fits in the Thm. with V = {(q, −q) : q ∈ RN/2 }.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
12 / 29
Related results: Barles 2007, Viterbo 2008 (by symplectic methods).
Example 1
utε + |uxε + γuyε | = l x, y , x−y
ε
u ε (0, x, y ) = h x, y , x−y
ε
x, y ∈ RN/2
there is homogenization ∀ l, h periodic in θ =
x−y
ε
⇐⇒
γ 6= 1.
γ = 1 by the Examples 1 and 2 of non-homogenization;
γ 6= 1 fits in the Thm. with V = {(q, −q) : q ∈ RN/2 }.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
12 / 29
Related results: Barles 2007, Viterbo 2008 (by symplectic methods).
Example 1
utε + |uxε + γuyε | = l x, y , x−y
ε
u ε (0, x, y ) = h x, y , x−y
ε
x, y ∈ RN/2
there is homogenization ∀ l, h periodic in θ =
x−y
ε
⇐⇒
γ 6= 1.
γ = 1 by the Examples 1 and 2 of non-homogenization;
γ 6= 1 fits in the Thm. with V = {(q, −q) : q ∈ RN/2 }.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
12 / 29
Convex-concave eikonal equation
|Dx u ε | − g2 x, y , x−y
|Dy u ε | = l(x, y , x−y
utε + g1 x, y , x−y
ε
ε
ε )
u ε (0, x, y ) = h(x, y , x−y
ε )
there is homogenization if
(g1 − g2 )(x, y , θ) ≥ γo > 0 ∀ x, y , θ,
or, symmetrically, g2 − g1 ≥ γo .
Proof: by the Thm. with V = {(q, −q) : q ∈ RN/2 }.
Motivation: Pursuit-Evasion type differential games, with pursuer
controlling the x variables, evader the y variables, and cost depending
on the distance between them.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
13 / 29
The result is sharp in the next
Example 2
utε + |uxε | − γ|uyε | = l x, y , x−y
ε
there is homogenization ∀ l, h periodic in θ =
x, y ∈ RN/2
x−y
ε
⇐⇒
γ 6= 1.
γ = 1 by the Examples 3 and 4 of non-homogenization;
γ 6= 1 fits in the preceding slide.
Other results
by control-game methods we can treat problems with different
assumptions: instead of strongly interacting fast variables,
decoupling of xε and yε , e.g., h or l has a saddle point in this
entries;
many problems on homogenization of convex-concave H-J
equations remain open!
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
14 / 29
The result is sharp in the next
Example 2
utε + |uxε | − γ|uyε | = l x, y , x−y
ε
there is homogenization ∀ l, h periodic in θ =
x, y ∈ RN/2
x−y
ε
⇐⇒
γ 6= 1.
γ = 1 by the Examples 3 and 4 of non-homogenization;
γ 6= 1 fits in the preceding slide.
Other results
by control-game methods we can treat problems with different
assumptions: instead of strongly interacting fast variables,
decoupling of xε and yε , e.g., h or l has a saddle point in this
entries;
many problems on homogenization of convex-concave H-J
equations remain open!
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
14 / 29
The result is sharp in the next
Example 2
utε + |uxε | − γ|uyε | = l x, y , x−y
ε
there is homogenization ∀ l, h periodic in θ =
x, y ∈ RN/2
x−y
ε
⇐⇒
γ 6= 1.
γ = 1 by the Examples 3 and 4 of non-homogenization;
γ 6= 1 fits in the preceding slide.
Other results
by control-game methods we can treat problems with different
assumptions: instead of strongly interacting fast variables,
decoupling of xε and yε , e.g., h or l has a saddle point in this
entries;
many problems on homogenization of convex-concave H-J
equations remain open!
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
14 / 29
2. Optimal control with stochastic volatility
In Financial Mathematics the evolution of a stock S is described by
d log Ss = γ ds + σ dWs
and the classical Black-Scholes formula for the option pricing problem
is derived assuming the parameters are constants.
However the volatility σ is not really a constant, it rather looks like an
ergodic mean-reverting stochastic process, so it is often modeled as
σ(ys ) with ys an Ornstein-Uhlenbeck diffusion process.
It is argued in the book
Fouque, Papanicolaou, Sircar: Derivatives in financial markets with
stochastic volatility, 2000,
that the process ys also evolves on a faster time scale than the stock
prices. Next picture shows the typical bursty behavior.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
15 / 29
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
16 / 29
The model with fast stochastic volatility σ proposed in [FPS] is
d log Ss = γ ds + σ(ys ) dWs
dys = 1ε (m − ys ) +
√ν d W̃s
ε
For the option pricing problem the limit is given by the Black-Scholes
formula of the model with (constant) mean historical volatility
d log Ss = γ ds + σ dWs ,
2
2
e−(y −m) /2ν
σ =
σ (y ) √
dy ,
2πν 2
R
2
Z
2
the mean being w.r.t. the (Gaussian) invariant measure of the
Ornstein-Uhlenbeck process driving ys .
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
17 / 29
Merton portfolio optimization problem
Invest βs in the stock Ss , 1 − βs in a bond with interest rate r .
Then the wealth xs evolves as
d xs = (r + (γ − r )βs )xs ds + xs βs σ dWs
and want to maximize the expected utility at time t,
for some h increasing and concave.
E[h(xt )],
If h(x) = ax δ /δ with a > 0, 0 < δ < 1, a HARA function, and the
parameters are constants the problem has an explicit solution.
The version with fast stochastic volatility is
d xs = (r + (γ − r )βs )xs ds + xs βs σ(ys ) dWs
dys = 1ε (m − ys ) +
√ν d W̃s
ε
QUESTIONS:
1. Is the limit as ε → 0 a Merton problem with constant volatility σ?
2. If so, is σ an average of σ(·) ?
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
18 / 29
Merton portfolio optimization problem
Invest βs in the stock Ss , 1 − βs in a bond with interest rate r .
Then the wealth xs evolves as
d xs = (r + (γ − r )βs )xs ds + xs βs σ dWs
and want to maximize the expected utility at time t,
for some h increasing and concave.
E[h(xt )],
If h(x) = ax δ /δ with a > 0, 0 < δ < 1, a HARA function, and the
parameters are constants the problem has an explicit solution.
The version with fast stochastic volatility is
d xs = (r + (γ − r )βs )xs ds + xs βs σ(ys ) dWs
dys = 1ε (m − ys ) +
√ν d W̃s
ε
QUESTIONS:
1. Is the limit as ε → 0 a Merton problem with constant volatility σ?
2. If so, is σ an average of σ(·) ?
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
18 / 29
By formal asymptotic expansions [FPS] argue that the answers are
1. Yes, the limit is a Merton problem with constant volatility σ
2. σ is a harmonic average of σ
Z
−1/2
1
,
σ :=
dµ(y )
σ 2 (y )
NOT the linear average as in Black-Scholes!
Practical consequence:
if I have N empirical data σ1 , ..., σN of the volatility, in the Black-Scholes
formula for option pricing I estimate the volatility by the arithmetic mean
σa2 =
N
1X 2
σi ,
N
i=1
whereas in the Merton problem I use the harmonic mean
!−1
N
1X 1
2
σh =
≤ σa2 .
2
N
σ
i=1 i
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
19 / 29
By formal asymptotic expansions [FPS] argue that the answers are
1. Yes, the limit is a Merton problem with constant volatility σ
2. σ is a harmonic average of σ
Z
−1/2
1
,
σ :=
dµ(y )
σ 2 (y )
NOT the linear average as in Black-Scholes!
Practical consequence:
if I have N empirical data σ1 , ..., σN of the volatility, in the Black-Scholes
formula for option pricing I estimate the volatility by the arithmetic mean
σa2 =
N
1X 2
σi ,
N
i=1
whereas in the Merton problem I use the harmonic mean
!−1
N
1X 1
2
σh =
≤ σa2 .
2
N
σ
i=1 i
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
19 / 29
By formal asymptotic expansions [FPS] argue that the answers are
1. Yes, the limit is a Merton problem with constant volatility σ
2. σ is a harmonic average of σ
Z
−1/2
1
,
σ :=
dµ(y )
σ 2 (y )
NOT the linear average as in Black-Scholes!
Practical consequence:
if I have N empirical data σ1 , ..., σN of the volatility, in the Black-Scholes
formula for option pricing I estimate the volatility by the arithmetic mean
σa2 =
N
1X 2
σi ,
N
i=1
whereas in the Merton problem I use the harmonic mean
!−1
N
1X 1
2
σh =
≤ σa2 .
2
N
σ
i=1 i
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
19 / 29
Conclusion: the correct average seems to depend on the problem!
QUESTIONS:
Why?
Can the formula for the Merton problem be rigorously justified and
what is the convergence as ε → 0?
Is there a unified "formula" for the two problems and for other
similar problems?
Note: the Black-Scholes model has no control, the associated PDE is
a heat-type linear equation;
Merton is a stochastic control problem whose Hamilton-Jacobi-Bellman
equation is fully nonlinear and degenerate parabolic.
I’ll present a result on two-scale Hamilton-Jacobi-Bellman equations
that will answer these (and other) questions.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
20 / 29
The H-J approach to Singular Perturbations
Control system
dxs = f (xs , ys , αs ) ds + σ(xs , ys , αs ) dWs ,
dys = 1ε g(xs , ys ) ds +
x0 = x,
√1 ν(xs , ys ) dWs ,
ε
xs ∈ Rn , αs ∈ A,
ys ∈ Rm ,
y0 = y
Cost functional
J ε (t, x, y , α) :=
Rt
u ε (t, x, y ) :=
inf E [J ε (t, x, y , α)]
0
l(xs , ys , αs ) ds + h(xt , yt )
Value function
α∈A(t)
A(t) admisible controls on [0, t].
GOAL: let ε → 0 and find a simplified model.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
21 / 29
The H-J approach to Singular Perturbations
Control system
dxs = f (xs , ys , αs ) ds + σ(xs , ys , αs ) dWs ,
dys = 1ε g(xs , ys ) ds +
x0 = x,
√1 ν(xs , ys ) dWs ,
ε
xs ∈ Rn , αs ∈ A,
ys ∈ Rm ,
y0 = y
Cost functional
J ε (t, x, y , α) :=
Rt
u ε (t, x, y ) :=
inf E [J ε (t, x, y , α)]
0
l(xs , ys , αs ) ds + h(xt , yt )
Value function
α∈A(t)
A(t) admisible controls on [0, t].
GOAL: let ε → 0 and find a simplified model.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
21 / 29
Assumptions on data:
h continuous, supy h(x, y ) ≤ K (1 + |x|2 )
f , σ, g, τ, l Lipschitz in (x, y ) (unif. in α) with linear growth
Then u ε solves the HJB equation
∂u ε
1
ε
2 ε 1
2 ε
+ H x, y , Dx u , Dxx u , √ Dxy u − Lu ε = 0 in R+ × Rn × Rm ,
∂t
ε
ε
n
o
H (x, y , p, M, Z ) := max −tr(σσ T M) − f · p − l − tr(σνZ T )
a∈A
2
L := tr(νν T Dyy
) + g · Dy .
Theorem (Da Lio - Ley). The value function u ε is the unique viscosity
solution with growth u ε (t, x, y ) ≤ C(1 + |x|2 + |y |2 ) of the Cauchy
problem with initial condition
u ε (0, x, y ) = h(x, y ).
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
22 / 29
Next assume on the linear operator L
Ellipticity:
Lyapunov:
∃ Λ(y ) > 0 s.t. ∀ x
ν(x, y )ν T (x, y ) ≥ Λ(y )I
∃w ∈ C(Rm ), k > 0, R0 > 0 s.t.

 −Lw ≥ k for |y | > R0 , ∀x,

w(y ) → +∞
as |y | → +∞.
Then,
the fast subsystem rescaled by τ = s/ε and with x ∈ Rn frozen
(FS)
dyτ = g(x, yτ ) dτ + ν(x, yτ ) dWτ ,
is uniquely ergodic, i.e., has a unique invariant measure µx .
Liouville property:
any bounded subsolution of −Lv = 0 is constant.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
23 / 29
Next assume on the linear operator L
Ellipticity:
Lyapunov:
∃ Λ(y ) > 0 s.t. ∀ x
ν(x, y )ν T (x, y ) ≥ Λ(y )I
∃w ∈ C(Rm ), k > 0, R0 > 0 s.t.

 −Lw ≥ k for |y | > R0 , ∀x,

w(y ) → +∞
as |y | → +∞.
Then,
the fast subsystem rescaled by τ = s/ε and with x ∈ Rn frozen
(FS)
dyτ = g(x, yτ ) dτ + ν(x, yτ ) dWτ ,
is uniquely ergodic, i.e., has a unique invariant measure µx .
Liouville property:
any bounded subsolution of −Lv = 0 is constant.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
23 / 29
Next assume on the linear operator L
Ellipticity:
Lyapunov:
∃ Λ(y ) > 0 s.t. ∀ x
ν(x, y )ν T (x, y ) ≥ Λ(y )I
∃w ∈ C(Rm ), k > 0, R0 > 0 s.t.

 −Lw ≥ k for |y | > R0 , ∀x,

w(y ) → +∞
as |y | → +∞.
Then,
the fast subsystem rescaled by τ = s/ε and with x ∈ Rn frozen
(FS)
dyτ = g(x, yτ ) dτ + ν(x, yτ ) dWτ ,
is uniquely ergodic, i.e., has a unique invariant measure µx .
Liouville property:
any bounded subsolution of −Lv = 0 is constant.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
23 / 29
A typical sufficient condition: ellipticity + recurrence condition
i
h
lim sup sup g(x, y ) · y + tr(νν T (x, y )) < 0
|y |→+∞
x
Proof by choosing as Lyapunov function w(y ) = |y |2 .
Example: Ornstein-Uhlenbeck process
dyτ = (m(x) − yτ )dτ + ν(x)dWτ ,
where m, ν are bounded.
GOAL: let ε → 0 in the Cauchy problem for the HJB equation.
Similarity with homogenization problems: the "fast" variable y plays the
role of xε .
Main difference with periodic homogenization: the state variables ys
are unbounded and uncontrolled.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
24 / 29
A typical sufficient condition: ellipticity + recurrence condition
i
h
lim sup sup g(x, y ) · y + tr(νν T (x, y )) < 0
|y |→+∞
x
Proof by choosing as Lyapunov function w(y ) = |y |2 .
Example: Ornstein-Uhlenbeck process
dyτ = (m(x) − yτ )dτ + ν(x)dWτ ,
where m, ν are bounded.
GOAL: let ε → 0 in the Cauchy problem for the HJB equation.
Similarity with homogenization problems: the "fast" variable y plays the
role of xε .
Main difference with periodic homogenization: the state variables ys
are unbounded and uncontrolled.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
24 / 29
Convergence Theorem
Under the previous assumptions, the (weak) limit u(t, x) as ε → 0 of
the value function u ε (t, x, y ) solves

R
∂u
2
n

 ∂t + H x, y , Dx u, Dxx u, 0 dµx (y ) = 0 in R+ × R
(CP)

R

u(0, x) = h(x, y ) dµx (y )
If, moreover, either g, ν do not depend on x or
g(x, ·), ν(x, ·) ∈ C 1 and g(·, y ), ν(·, y ) ∈ Cb1
Dx g, Dy g, Dx ν, Dy ν are Hölder in y uniformly w.r.t. x,
then u is the unique viscosity solution (with quadratic growth) of (CP)
and u ε (t, x, y ) → u(t, x) locally uniformly on (0, +∞) × Rn .
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
25 / 29
Examples
Denote hφi :=
R
φ(y )dµx (y ). Got simple formulas for effective H and h
H(x, p, M) = hH(x, ·, p, M, 0)i,
h(x) = hh(x, ·)i.
For split systems, i.e.,
σ = σ(x, y ),
f = f0 (x, y ) + f1 (x, a),
l = l0 (x, y ) + l1 (x, a),
the linear averaging of the data is the correct limit, i.e.,
Z t
ε
lim u (t, x, y ) = u(t, x) := inf E
hli(xs , αs ) ds + hhi(xt ) ,
α.
ε→0
0
dxs = hf i(xs , αs ) ds + hσσ T i1/2 (xs ) dWs
Proof: H(x, p, M) = −trace(Mhσσ T i)/2 + maxA {−hf i · p − hli} .
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
26 / 29
Examples
Denote hφi :=
R
φ(y )dµx (y ). Got simple formulas for effective H and h
H(x, p, M) = hH(x, ·, p, M, 0)i,
h(x) = hh(x, ·)i.
For split systems, i.e.,
σ = σ(x, y ),
f = f0 (x, y ) + f1 (x, a),
l = l0 (x, y ) + l1 (x, a),
the linear averaging of the data is the correct limit, i.e.,
Z t
ε
lim u (t, x, y ) = u(t, x) := inf E
hli(xs , αs ) ds + hhi(xt ) ,
α.
ε→0
0
dxs = hf i(xs , αs ) ds + hσσ T i1/2 (xs ) dWs
Proof: H(x, p, M) = −trace(Mhσσ T i)/2 + maxA {−hf i · p − hli} .
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
26 / 29
Example: we recover the Black-Scholes formula with stochastic
volatility.
In general, for system or cost NOT split,
H(x, p, M) = h max{...}i > max h{...}i
A
A
and the limit control problem is not obvious.
We try to write H as a Bellman Hamiltonian in some other way in order
to find an explicit effective control problem approximating the singularly
perturbed one as ε → 0. We did it for
Merton problem with stochastic volatility (see next slides),
Ramsey model of optimal economic growth with (fast) random
parameters,
Vidale - Wolfe advertising model with random parameters,
advertising game in a duopoly with Lanchester dynamics and
random parameters.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
27 / 29
Merton portfolio optimization with stochastic volatility
Given a control βs the wealth xs evolves as
dxs = (r + (γ − r )βs )xs ds + xs βs σ(ys ) dWs
dys = 1ε (m − ys ) ds +
and the value functions is
√ν d W̃s
ε
x0 = x
y0 = y
V ε (t, x, y ) := supβ. E[h(xt )].
Let ρ = correlation of Ws and W̃s . The HJB equation is
∂V ε
b2 x 2 σ2 ε
bxρσν ε
− rxVxε − max (γ − r )bxVxε +
Vxx + √ Vxy
∂t
2
ε
b
ε
(m − y )Vyε + ν 2 Vyy
=
ε
Assume the utility h has h0 > 0 and h00 < 0. Then expect a value
ε < 0.
function strictly increasing and concave in x, i.e., Vxε > 0, Vxx
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
28 / 29
The HJB equation becomes
ε
√ V ε ]2
[(γ − r )Vxε + xρσν
(m − y )Vyε + ν 2 Vyy
xy
∂V ε
ε
− rxVxε +
=
ε
∂t
ε
σ 2 (y )2Vxx
By the Theorem, V ε (t, x, y ) → V (t, x) as ε → 0 and V solves
(γ − r )2 Vx2
∂V
− rxVx +
∂t
2Vxx
Z
1
σ 2 (y )
dµ(y ) = 0
in R+ × R+
This is the HJB equation of a Merton problem with the harmonic
average of σ as constant volatility
Z
σ :=
−1/2
1
dµ(y )
.
σ 2 (y )
Therefore this is the limit control problem.
The limit of the optimal control βsε,∗ as ε → 0 can also be studied.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
29 / 29
The HJB equation becomes
ε
√ V ε ]2
[(γ − r )Vxε + xρσν
(m − y )Vyε + ν 2 Vyy
xy
∂V ε
ε
− rxVxε +
=
ε
∂t
ε
σ 2 (y )2Vxx
By the Theorem, V ε (t, x, y ) → V (t, x) as ε → 0 and V solves
(γ − r )2 Vx2
∂V
− rxVx +
∂t
2Vxx
Z
1
σ 2 (y )
dµ(y ) = 0
in R+ × R+
This is the HJB equation of a Merton problem with the harmonic
average of σ as constant volatility
Z
σ :=
−1/2
1
dµ(y )
.
σ 2 (y )
Therefore this is the limit control problem.
The limit of the optimal control βsε,∗ as ε → 0 can also be studied.
Multi-scale Bellman-Isaacs
Roma, June 15th, 2011
29 / 29
```

# Homogenization and other multi-scale problems in Bellman #### Prof. A. Zamperini #### In collaboration with Algo Mas financial support by Politecnico di #### Modello slides #### 2014-2015-coro-val-san-martino-senza #### San MaRTInO 