Differentiability of the Feynman-Kac semigroup and a control

A TTI A CCADEMIA N AZIONALE L INCEI C LASSE S CIENZE F ISICHE M ATEMATICHE N ATURALI
R ENDICONTI L INCEI
M ATEMATICA E A PPLICAZIONI
Giuseppe Da Prato, Jerzy Zabczyk
Differentiability of the Feynman-Kac semigroup
and a control application
Atti della Accademia Nazionale dei Lincei. Classe di Scienze Fisiche,
Matematiche e Naturali. Rendiconti Lincei. Matematica e Applicazioni,
Serie 9, Vol. 8 (1997), n.3, p. 183–188.
Accademia Nazionale dei Lincei
<http://www.bdim.eu/item?id=RLIN_1997_9_8_3_183_0>
L’utilizzo e la stampa di questo documento digitale è consentito liberamente per motivi
di ricerca e studio. Non è consentito l’utilizzo dello stesso per motivi commerciali. Tutte
le copie di questo documento devono riportare questo avvertimento.
Articolo digitalizzato nel quadro del programma
bdim (Biblioteca Digitale Italiana di Matematica)
SIMAI & UMI
http://www.bdim.eu/
Atti della Accademia Nazionale dei Lincei. Classe di Scienze Fisiche, Matematiche e
Naturali. Rendiconti Lincei. Matematica e Applicazioni, Accademia Nazionale dei
Lincei, 1997.
Rend. Mat. Ace. Lincei
s. 9, v. 8:183-188 (1997)
Analisi matematica. — Differentiability of the Feynman-Kac semigroup and a control
application. Nota di GIUSEPPE DA PRATO e JERZY ZABCZYK, presentata (*) dal Corrisp.
G. Da Prato.
ABSTRACT. — The Hamilton-Jacobi-Bellman equation corresponding to a large class of distributed control problems is reduced to a linear parabolic equation having a regular solution. A formula for the first
derivative is obtained.
KEY WORDS: Stochastic control problem; Feynman-Kac formula; Hamilton-Jacobi equations.
RIASSUNTO. — Differenziabilità del semigruppo ài Feynman-Kac e applicazioni. L'equazione di HamiltonJacobi-Bellman corrispondente a un'ampia classe di problemi di controllo distribuiti viene ridotta a una
equazione parabolica lineare avente una soluzione regolare. Viene inoltre ottenuta una formula per la derivata prima della soluzione.
1. INTRODUCTION
The Note is concerned with a distributed control system in which the control action
is perturbed by noise. To write down the system assume that H is a separable Hilbert
space and A: D(A) cH-^H,
F:H-^H,
B: H->L(H,H)
are mappings such
that:
Al) Operator A is the infinitesimal generator of a strongly continuous semigroup
S(t), t ^ 0,
A2) Mappings F and B are Lipschitz and
supdlB^II + llB- 1 ^)!!) < + o o .
XBH
Let, in addition, W(t), t ^ 0, be a cylindrical Wiener process on H, defined on a
probability space (Q, &y P) with a filtration {&}t. The control system under consideration is described by the equations:
(1.1)
dX = (AX + F(X))dt + B(X)(zdt + adW(t)),
X(0)=x,
where z{i), t ^ 0, is a control process and a is a constant. Note that if a = 0, the equation (1.1) is deterministic and the action of the controller is not affected by random
perturbations. The effect of the noise is present if a ^ 0.
Assume that the cost functional is of the form
/,(*,*) = E l j[g(X(s,x))+
\z(s)\2]ds +
y(X(tyx))V
where X(- ,x) denotes the solution to (1.1) and g and ip are given functions. The corre(*) Nella seduta del 19 giugno 1997.
184
G. DA PRATO - J. ZABCZYK
sponding Hamilton-Jacobi-Bellman equation for the value function
V(t,x) =
m£Jt(x,z),
is then
f Vt{t,x) = ( a 2 / 2 ) T r [ B U ) B M x ) V « ( ^ ) ] + {Ax + F(x), Vx{t,x))
d-2)
\
2
~
\B*(x)Vx(t,x)\ /4+g(x)y
{ V(0,-) = y .
It is not difficult to show that if equation (1.2) has a classical solution, then it can be
identified with the value function and the control
z(s)= -o2B*
(1.3)
{X(s,x))Vx(t
~ s,X(s,x)),
*e[0,f[,
is an optimal one. However the classical solution does not exist in general. Under some
condition one can prove existence of a viscosity solution, see [9-11,13], which is however only continuous and the formula (1.3) loses its meaning. Another way of solving
the problem was proposed for systems with diffusion B independent of x. This approach was introduced in [1,2] and it was developped on in [6-8,3]. One writes (1.2)
in the so-called mild form:
t
(1.4)
V(tr)-Pty>
+ jPt.s({F,Vx(s,-))-
\B*Vx(s,-)\2/4+g)ds,
0
where Pt, t ^ 0 is the transition semigroup corresponding to (1.1) with F = 0, a = 0.
Under assumptions implying regularizing properties of Pt, t ^ 0 one shows existence of
a solution to (1.4) for which Vx does exist. There are essential difficulties to extend this
method to the case of B dependent of x. The corresponding semigroup Pn t ^ 0 has
regularizing properties under very strong assumptions. In the present paper we show
that the so-called logarithmic transform can be used to obtain solution of class
C1.
Note that setting
V(t,x) = - 2 a 2 l n ^ U , x ) ,
(1.5)
one arrives, after straightforward calculations, to a linear equation on u:
f ut(t,x) = ( a 2 / 2 ) T r [ E ( x ) E * («)««(*,*)] +
(1.6)
I
[ V(0, •)=e-^{2°2)
+(Ax + F{x)yux{t,x))
-
g(x)u{t,x)/{2o2),
= (p,
which is of the Feynman-Kac type.
In the next section we will derive a formula for ux giving a meaning to the feedback
law (1.5). Some of the assumptions of our theorem could be removed as we intend to
show in a future paper. We conjecture that if the functions g, F and B are Gateaux differentiate with bounded and weakly continuous derivatives, and cp is bounded continuous, then equation (1.6) has a solution and the optimal control is of the form
(1.5).
DIFFERENTIABILITY OF THE FEYNMAN-KAC SEMIGROUP ...
185
2. THE FEYNMAN-KAC SEMIGROUP
Let g be a bounded and continuous function from H into R. We shall denote by Pf,
t ^ 0 the Feynman-Kac semigroup
-
(2.1)
P?cp(x)=E[(p(X(t>x))e
\g{X(s,x))ds
o
],
f^o,
for cp on the space By (H) of bounded Borei functions. The function
u(t,x) =Pf(p(x),
t^O,
xeH,
is a candidate for a solution to equation (1.2) in which for simplicity, we set a = 1 and
replace g/2 by g.
We show that under conditions Al), A2) the semigroup P/ has regularizing properties similarly as in the finite dimensional case. The formula (2.2) below is new. It is well
known in the special case g = 0, see [5,12].
THEOREM 2.1. Assume that conditions Al), A2 ) hold. Assume moreover that F, B,gare
twice differentiahle functions with hounded and continuous derivatives up to the second order. If cp e Cy(H) then Pf cp is differentiahle in any direction h s H and
(2.2)
D;Pf<pM = E (p(X(t,x))e
-\g(X(a,x))dal
o
J
lt-1j(B-1(X(s,x)Xxb(s,x),dW(s))
- | ( 1 - s/t){Xhx (s, x), Dxg(X(s, x)))ds
o
Im
where Dx denotes the derivative in the direction h.
Let {e„} be a complete orthonormal system on H. For each n E N let
X„('yX) be the solution of the problem
PROOF.
(2.3)
dXH = (AXn + F(Xn))dt + B(XH)QHdW(t)
Xn(0)=x,
1
where A„ = nA(n - A)' is the Yosida approximation of A and Qn is the orthogonal
projection of H onto linj^j, ...,e„}. Then the function
- jg(Xn(s,x))ds
un{t,x) = E[cp(Xn(t,x))e
o
],
is a strict solution of parabolic equation
Dtu„ (t, x) = Tr [B(x) QnB* (x)D2un (*, x)]/2 +
+ {Ax + F(x),D2un(t,x))
- g{x)un{t,x)
u„(0,x) = cp{x).
Fix t > 0. Applying Itô's formula to the process
- jg(Xn(a,x))do
e °
u„(t-s,X„(s,x)),
se
[0,t]
= £„un{t,x)
186
G. DA PRATO - J. ZABCZYK
we have
-
^g(Xn(o,x))do
de °
un{t - s,Xn(s,x))
=
-
= ( -Dtu„(t
- s,Xn(s,x))
+ £nun{t -syXn(s,x)))e
^g(X„(o,x))do
°
ds +
-
+ (Dxun(t-s,Xn(s,x)),B(Xn(s,x))QJW(s))e
\g(Xn(o,x))do
»
Therefore
e o
cp{Xn{t,x)) =u„{t,x) +
'
-
+ \e
\g{Xn(o,x))do
«
(DA(/-5,X,(J,X)),B(X,(J,X))û,^)).
0
Letting /z tend to infinity we obtain that
-\g{X(o,x))do
e °
cp{X(t,x)) = u(t,x) +
'
~ \g(X(o,x))do
+ \e »
(Dxu^-^^^))^(X(s,x))QndW(s)).
o
Multiplying this identity by
t
^(B-1(X(s>x))X*(s,x),dW(s)),
0
and taking expectation, we arrive at
K:=E\e
•^g(X(o,x))do
<p(X(t, x)) j(B - 1 (X(s, x)) Xhx {s, x), dW(s)) =
°
-
\g(X(a,x))da
{Dxu{t-s,X{s,x)),Xhx{s,x))).
= E(e •
On the other hand
-
K = Dhxie »
\g(X(a,x))da
u(t-s,X(s,x))]
=
- \g(X(a,x))da
/
\(Dxg{X{o,x)),Xhx{o,x))dou(t-s,X{s,x))
= -e '
+
0
+
{Dxu(t-s,X{s,x)),Xhx{s,x)).
DIFFERENTIABILITY O F T H E FEYNMAN-KAC S E M I G R O U P
187
...
Therefore
J
K=E
JD*(e
- fg(X(o,x))da
o
\
u{t-s,X{syx)))ds\
/
-
\g(X(o,x))do
((Dxg(X(o,x),Xxb(o,x))do)u(t-s,X(s,x))
\e °
+E
+
Applying to both the expressions the Markov property we obtain that
t
K = Dhx\Pf{Pf^cp){x)ds
+
0
(
- \g{X{o,x))da
cp(X{tyx))e
o
I
* *
\
J j(Dxg(X(o,x)),X*(o,x))dods\
=
\g(X(a,x))da J
\
<p(X(t, x)) e«
J (t - o)(Dxg(X(o, *)), Xhx (a, x))da ,
and the result follows for cp e Cl (lì).
Let now cp be an arbitrary function from Q (H). Then there exists a sequence
{cpn} eCl(H) uniformly bounded and pointwise convergent to cp. Let us fix t > 0,
x E H and h e H, and consider functions
vn{o)=Pfcpn{x
+ oh),
O^o^l.
Then
vl(o)=DÏPf(pH(x
+ oh),
0 ^ o ^ 1.
From the definition of vn and the formula for DxPfcpn valid for {cpn} s Cl (H) one
easily gets that functions vn, v'n are convergent, in a bounded way, to Pfcp{x + oh) and to
- \g(X{o,x + oh))dol
E cp{X{t,x + ah))e °
\rl{B-l{X(s,x
+ oh)Xxh(s,x + oh),dW(s))
- J ( l -s/t)(Xxh(syx
+ oh), Dxg{X{syx + oh)))ds
for o e]0, 1]. Taking o = 0 one gets that the formula for the directional derivative is
true in general.
•
REFERENCES
[1] P . CANNARSA - G. D A PRATO, Some results on nonlinear optimal control problems and
equations infinite dimensions. J. Funct. Anal., 90, 1990, 27-47.
Hamilton-]acobi
[2] P . CANNARSA - G. D A PRATO, Direct solution of a second order Hamilton-]acobi equation in Hilbert spaces.
In: G. D A PRATO - L. TUBARO (eds.), Stochastic partial differential equations and applications. Pitman Research Notes in Mathematics Series n. 268, 1992, 72-85.
188
G. DA PRATO - J. ZABCZYK
[3] G. D A PRATO - A. DEBUSSCHE, Control of the stochastic Burgers model of turbulence. Scuola Normale
Superiore preprint n. 4, Pisa 1996.
[4] G. D A PRATO - J. ZABCZYK, Ergodicity for infinite dimensions. Enciclopedia of Mathematics and its Applications, Cambridge University Press, 1996.
[5] K. D . ELWORTHY, Stochastic flows on Riemannian manifolds. In: M. A. P I N SKY - V. VIHSTUTZ (eds.), Diffusion Processes and Related Problems in Analysis. Birkhàuser, 1992, vol. II, 33-72.
[6] F. G o z z i , Regularity of solutions of a second order Hamilton-]acobi equation and application to a control
problem.
Commun, in partial differential equations, 20 (5&6), 1995, 775-826.
[7] F. G o z z i , Global regular solutions of second order Hamilton-]acobi
equations in Hilbert spaces with locally
Lipschitz nonlinearities. Journal of Mathematical Analysis and Applications, 198, 1996, 399-443.
[8] F. G o z z i - E. ROUY, Regular solutions of second order stationary Hamilton-]acobi
equations. J. Differential
Equations, to appear.
[9] P . L. L I O N S , Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in
infinite
dimensions.
Part I: The
case of bounded
stochastic
evolution.
Acta Math.,
161, 1988,
243-278.
[10] P . L. L I O N S , Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in
infinite dimensions. Part II: Optimal control fo Zakais equation. In: G. D A PRATO - L. TUBARO (eds.),
Stochastic Partial Differential
Equations
and Applications.
Lecture Notes in Mathematics N o . 1390,
Springer-Verlag, 1989, 147-170.
[11] P. L. L I O N S , Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in
infinite dimensions. Part III: Uniqueness of viscosity solutions for general second order equations. J. Funct.
A n a l , 86, 1989, 1-18.
[12] S. PESZAT - J. ZABCZYK, Strong Feller property and irreducibility for diffusions on Hilbert spaces. Annals of
Probability, 1996.
[13] A. SWIECH, Viscosity solutions of fully nonlinear partial differential equations with «unbounded» terms in infinite dimensions.
Ph. D . Thesis, University of California at Santa Barbara, 1993.
G. D a Prato:
Scuola Normale Superiore
Piazza dei Cavalieri, 7 - 56126 PISA
J. Zabczyk:
Instytut Matematyczny
Polskiej Akademii N a u k
ul. Sniadeckich 8
skr. pocztowa, 137 - 00-950 VARSAVIA (Polonia)