MATHEMATICS

M. G. SHUR

CONTINUOUS ADDITIVE FUNCTIONALS OF MARKOV PROCESSES AND EXCESSIVE FUNCTIONS

(Presented by Academician P. S. Aleksandrov on 5 XI 1960)

E. B. Dynkin posed the problem of describing the structure of additive functionals of Markov processes. V. A. Volkonskii succeeded in making considerable progress toward solving this problem. In particular, in Theorems 1.1 and 1.2 of the work \((^1)\), V. A. Volkonskii established a one-to-one correspondence between a broad class of homogeneous additive functionals and a certain subclass of the class of excessive functions associated with a homogeneous Markov process*. These theorems of V. A. Volkonskii are developed in the present work**.

Consider a homogeneous Markov process*** \(X=(x_t,\zeta,M_t,P_x,\theta_t)\), given in a measurable space \((E,B)\). A function \(\varphi_t(\omega)\) \((0\le t<\zeta(\omega))\) is called \((^1)\) a homogeneous additive functional of the process \(X\), if: 1) the function \(\varphi_t\) is \(\overline{N}_t\)-measurable; 2) \(\varphi_s(\omega)+\theta_s\varphi_t(\omega)=\varphi_{s+t}(\omega)\) \(P_x\)-almost surely on the set \(s+t<\zeta(\omega)\) for all \(s,t\ge0\), \(x\in E\); 3) \(0\le\varphi_t(\omega)\le\infty\) \((t\ge0)\). For convenience we shall agree to regard the functional \(\varphi_t(\omega)\) as defined for all \(t\ge0\) (and not only for \(t<\zeta(\omega)\)) and set, for \(\infty\ge t\ge\zeta(\omega)\), \(\varphi_t(\omega)=\varphi_{\zeta(\omega)-0}(\omega)\). The functional \(\varphi_t(\omega)\) is continuous if the function \(\varphi_t(\omega)\) is continuous in \(t\) for all \(\omega\in\Omega\).

We shall say that the function \(f(x)\) is the generalized potential of the functional \(\varphi_t\), if \(f(x)=M_x\varphi_\infty\). It is not difficult to verify \((^1)\) that if \(f(x)\) is the generalized potential of a continuous homogeneous additive functional \(\varphi_t(\omega)\), then for all \(t\ge0\)

\[ f(x)\ge T_t f(x)^{****},\qquad f(x)=\lim_{t\to0}T_t f(x). \tag{1} \]

Functions satisfying the relations (1) are, by definition, excessive functions.

We shall say that a function \(f(x)\) satisfies condition (A) if for every nondecreasing sequence \(\tau_n\) of random—

* The notion of an excessive function associated with a Markov process was first introduced by Hunt \((^5)\); see also \((^6)\).

** As E. B. Dynkin has informed the author, analogous results were obtained in France by P. Meyer (P. A. Meyer) (unpublished).

*** We use the terminology and notation of the book \((^2)\). In particular, henceforth \(N_t\) denotes the \(\sigma\)-algebra generated by the events \(\{x_s(\omega)\in\Gamma,\ \zeta>s\}\), where \(s\le t,\ \Gamma\in B\). By \(N_{t+0}\) is denoted the \(\sigma\)-algebra of events \(A\) such that for every \(u>t\), \(\{A,\zeta>u\}\in N_u\). We put \(A\in\overline{N}_{t+0}\) (respectively \(\overline{N}_t\)) if, for any measure \(\mu\) on the \(\sigma\)-algebra \(B\), there exist sets \(A_1,A_2\in N_{t+0}\) \((N_t)\) such that \(A_1\subseteq A\subseteq A_2\) and \(\int P_x(A_2\setminus A_1)\mu(dx)=0\).

**** The operators \(T_t\) are given by the formula \(T_tg(x)=\int g(y)\,P(t,x,dy)\) on the class of functions \(g(x)\) for which this formula makes sense.

random variables independent of the future, and for any \(x\in E\)

\[ \lim_{n\to\infty} M_x f(x_{\tau_n})=M_x f(x_\tau), \]

where \(\tau=\lim_{n\to\infty}\tau_n\).

The main result of this paper is Theorem 1.

Theorem 1. Let \(X\) be a standard \(*\) process. In order that a bounded \(B\)-measurable excessive function \(f(x)\) be the generalized potential of some continuous additive homogeneous functional \(\varphi_t(\omega)\), it is necessary and sufficient that \(f(x)\) satisfy condition (A).

Proof of necessity. Let \(f(x)=M_x\varphi_\infty\), where \(\varphi_t\) is a continuous additive functional. Consider a random variable \(\xi\), independent of the future, and denote by \(\bar\chi\) the characteristic function of the event \(\{\xi<\zeta\}\). Approximating \(\xi\) from above by random variables independent of the future and having at most a countable set of values, it is not difficult to prove that

\[ M_x f(x_\xi)=M_x\{\bar\chi(\varphi_\infty-\varphi_\xi)\}. \]

Now let \(\tau_n\) be any nonincreasing sequence of random variables independent of the future, and let \(\tau=\lim_{n\to\infty}\tau_n\). Denote the characteristic functions of the events \(\{\tau_n<\zeta\}\), \(\bigcap_n\{\tau_n<\zeta\}\), \(\{\tau<\zeta\}\) by \(\chi_n\), \(\chi\), \(\chi'\). Obviously, as \(n\to\infty\), \(\chi_n\) tends to \(\chi\), while \(\varphi_{\tau_n}\) tends to \(\varphi_\tau\), and hence

\[ \lim_{n\to\infty} M_x f(x_{\tau_n}) = M_x\{\chi(\varphi_\infty-\varphi_\tau)\}. \]

On the other hand,

\[ M_x f(x_\tau)=M_x\{\chi'(\varphi_\infty-\varphi_\tau)\}. \]

Here \(\chi'\leqslant \chi\) and \(\{\chi'<\chi\}\subseteq\{\tau=\zeta\}\), while on the set \(\{\tau=\zeta\}\) one has \(\varphi_\infty-\varphi_\tau=0\), which proves condition (A).

The proof of sufficiency is considerably more difficult. First we establish several auxiliary results. We shall assume that the function \(f(x)\) is bounded, excessive, and satisfies condition (A). It is easily checked that for any \(t\geqslant0\) the function \(T_t f(x)\) also has these properties.

Denote by \(\tau_U\) the moment of first hitting the set \(U\in B\), more precisely \(\tau_U=\zeta\) if for all \(t>0\), \(x_t\barin U\), and \(\tau_U=\inf\{t:t>0,\ x_t\in U\}\) otherwise. Let

\[ B_h=\{x:\ |f(x)-T_h f(x)|>\varepsilon\}, \]

where \(\varepsilon>0\) is fixed, and \(\tau_h=\tau_{B_h}\).

Lemma. For any \(x\in E\),

\[ P_x\left\{\lim_{h\to0}\tau_h=\zeta\right\}=1. \]

Proof. Let \(h_n\downarrow0\) as \(n\to\infty\). Obviously, \(B_h\subseteq B_l\) if \(h\leqslant l\), and \(\tau_n\geqslant\tau_l\). Put \(\Omega_n=\{\tau_{h_n}<\zeta\}\), \(x_n=x(\tau_{h_n})\), and let \(k>n\). Then

\[ f(x_k)-T_{h_n}f(x_k)\geqslant f(x_k)-T_{h_k}f(x_k). \tag{2} \]

But

\[ f(x_k)-T_{h_k}f(x_k)\geqslant\varepsilon \]

(almost surely \(P_x\) on \(\Omega_k\)) \(**\), since for any excessive function \(g(x)\) the function \(g(x_t)\) is right-continuous in \(t\) (almost surely \(P_x\)) \((^5)\). Since \(f(x)\) and \(T_t f(x)\) satisfy condition (A) for all \(t\geqslant0\), it follows from (2), by what has been said, that

\[ M_x f(x_\tau)-M_x\{T_{h_n}f(x_\tau)\}\geqslant \varepsilon \lim_{k\to\infty} P_x\{\tau_k<\zeta\}. \]

As \(n\to\infty\), the left-hand side of this inequality tends to zero, and therefore

\[ \lim_{k\to\infty} P_x\{\tau_k<\zeta\}=0. \]

\(*\) A strictly Markov homogeneous process, continuous from the right, is called standard \((^2)\) if: a) the \(\sigma\)-algebra \(B\) is the system of Borel sets of a Hausdorff locally bicompact space \((E,C)\) with a countable base; b) if \(\tau_n\) is a nonincreasing sequence of random variables independent of the future and \(\tau=\lim_{n\to\infty}\tau_n\), then for any \(x\in E\), \(x_{\tau_n}\to x_\tau\) \(P_x\)-almost surely on the set \(\widehat{\Omega}=\{\tau<\zeta\}\).

\(**\) That is, \(P_x\)-almost surely on the set \(\Omega_k\).

Remark. Let

\[ B'_h=\left\{x:\left|f(x)-h^{-1}\int_0^h T_s f(x)\,ds\right|>\varepsilon\right\} \quad\text{and}\quad \tau'_h=\tau_{B'_h}. \]

It is clear that \(\tau'_h\ge \tau_h\), and hence

\[ P_x\left\{\lim_{h\to 0}\tau'_h=\zeta\right\}=1. \]

Now put \(f_h(x)=h^{-1}[f(x)-T_h f(x)]\). Let \(\zeta(\omega)\equiv\infty\) and \(s,t\ge 0\), \((s\le t)\). Then

\[ M_x\left[\int_s^t f_h(x_u)\,du\right]^2\le 2C^2,\qquad C=\sup_{x\in E} f(x). \tag{3} \]

Indeed, the left-hand side of this inequality is equal to

\[ 2M_x\int_s^t f_h(x_u)\left[\int_u^t f_h(x_v)\,dv\right]du = 2M_x\int_s^t f_h(x_u)M_{x_u}\int_0^{t-u} f_h(x_v)\,dv\,du, \]

whence (3) follows.

Theorem 2. For any \(s\ge 0\) and \(t\ge 0\) \((s\le t)\) there exists

\[ \varphi_t^s(\omega)=\lim_{h\downarrow 0}\int_{\min(s,\zeta)}^{\min(t,\zeta)} f_h(x_u)\,du \]

in the sense of convergence in mean square with respect to any measure \(P_x\).

Proof. Suppose that \(\zeta(\omega)\equiv\infty\). Denote by \(g(x)\) the difference of the functions \(f_h(x)\) and \(f_l(x)\), and by \(g_1(x)\) their sum. Let \(n\) be a natural number and \(s_k=s+k(t-s)/n,\ n\ge k\ge 1\). Define on the interval \([s,t]\) a function \(w(u)\) by the equalities \(w(u)=s_k\) if \(s_{k-1}<u\le s_k\). We have:

\[ M_x\left\{\int_s^t [f_h(x_u)-f_l(x_u)]\,du\right\}^2 = 2M_x\int_s^t g(x_u)\left[\int_u^t g(x_v)\,dv\right]du. \]

Estimate the integral

\[ I_1=M_x\int_s^t g(x_u)\left[\int_{w(u)}^t g(x_v)\,dv\right]du. \]

Using the Markov property, we readily obtain that

\[ I_1=\sum_{k=1}^n M_x\int_{s_{k-1}}^{s_k} g(x_u)\,du\, M_{x_{s_k}}\int_0^{s_n-k} g(x_v)\,dv. \]

Here the integral

\[ F_k(x)=M_x\int_0^{s_n-k} g(x_v)\,dv \]

is equal to the expression

\[ \left[ h^{-1}\int_0^h T_u f\,du - l^{-1}\int_0^l T_u f\,du \right] + \left[ l^{-1}\int_0^l T_{s_n-k+u}f\,du - h^{-1}\int_0^h T_{s_n-k+u}f\,du \right], \]

and therefore its modulus is no greater than \(2C\).

Consider the event \(A_v^u\) \((u\le v)\), consisting in the fact that during the time from \(u\) to \(v\) the trajectory of the process visits at least once the set where the modulus of at least one of the functions \(F_k(x)\) exceeds \(\varepsilon\). Denote the characteristic function of the event \(A_v^u\) by \(\chi_{u,v}\). By virtue of the remark to the lemma, for sufficiently small \(h\) and \(l\),

\[ P_x\{A_v^u\}<\varepsilon. \]

Therefore

\[ I_1\le \varepsilon M_x\int_s^t g_1(x_u)\,du + 2C M_x M_{x_s}\left[\chi_{0,t-s}\int_0^{t-s} g_1(x_u)\,du\right]. \]

The first term of this sum does not exceed \(2C\varepsilon\), and the second, by the Cauchy–Bunyakovsky inequality and (3), does not exceed

\[ 2CM_x\left[P_{xs}\{A_0^{t-s}\}M_{xs}\left\{\int_0^{t-s} g_1(x_u)\,du\right\}^2\right]^{1/2} \leqslant 8C^2 M_x[P_{xs}\{A_0^{t-s}\}]^{1/2}. \]

From these estimates, using the lemma, it is not hard to conclude that \(I_1\to 0\) as \(h\) and \(l\to 0\). In an analogous way it is established that the integral

\[ I_2=M_x\int_s^t g(x_u)\int_u^{\omega(u)} g(x_v)\,dv\,du \]

tends to zero as \(h\) and \(l\to 0\) and \(n\to\infty\).

Thus Theorem 2 is proved for nonterminating processes.

If \(\xi(\omega)\ne\infty\), then the process \(X\) can be transformed into a nonterminating process by introducing an additional state \(e\) ((5), § 17). In this case the function \(f_1(x)\), coinciding with \(f(x)\) for \(x\in E\) and equal to zero for \(x=e\), satisfies condition (A) for the new process. Using this circumstance, it is not hard to prove Theorem 2 also in the case \(\xi(\omega)\ne\infty\).

To construct a continuous functional \(\bar\varphi_t(\omega)\) such that \(f(x)=M_x\bar\varphi_\infty\), we shall use the ideas of V. A. Volkonskii \((^{1})\). For each \(x\in E\) choose a sequence \(h_k\) (depending on \(x\)) such that for all rational \(t\) there exists the limit

\[ \psi_t(\omega)=\lim_{k\to\infty}\int_0^{\min(t,\xi)} f_{h_k}(x_v)\,dv \quad (\text{almost surely }P_x), \]

and put \(\varphi_t(\omega)=\psi_t(\omega)\) on the set \(\{\omega:x_0(\omega)=x\}\). We shall show that the function \(\varphi_t(\omega)\) is uniformly continuous on the set of rational numbers \(t\) (\(t\geqslant 0\)) not exceeding any fixed integer \(m\). For this it is enough to show that the quantity

\[ J_n=M_x\sum_{k=0}^{2^n m}\left[\varphi_{2^{-n}(k+1)}-\varphi_{2^{-n}k}\right]^2 \]

tends to zero as \(n\to\infty\).

As in the proof of Theorem 2, we shall restrict ourselves to the case \(\xi(\omega)\equiv\infty\). Using Theorem 2 and Fatou’s lemma, we obtain

\[ J_n=M_x\sum_{k=0}^{2^n m}\lim_{h\to 0} \left[\int_{2^{-n}k}^{2^{-n}(k+1)} f_h(x_u)\,du\right]^2 \leqslant \lim_{h\to 0}M_x\sum_{k=0}^{2^n m} \left[\int_{2^{-n}k}^{2^{-n}(k+1)} f_h(x_u)\,du\right]^2 \leqslant \]

\[ \leqslant \lim_{h\to 0}M_x\sum_{k=0}^{2^n m} \int_{2^{-n}k}^{2^{-n}(k+1)} f_h(x_u)\left[M_{x_u}\int_0^{2^{-n}} f_h(x_v)\,dv\right]du. \]

Applying to the estimate of the last expression the method of proof of Theorem 2, we obtain that \(\lim_{n\to\infty}J_n=0\). Now the proof of Theorem 1 is completed without difficulty \((^{1})\).

Moscow State University
named after M. V. Lomonosov

Received
1 XI 1960

REFERENCES

\(^{1}\) V. A. Volkonskii, Tr. Mosc. Math. Soc., 9, 143 (1960).
\(^{2}\) E. B. Dynkin, Foundations of the Theory of Markov Processes, Moscow, 1959.
\(^{3}\) E. B. Dynkin, DAN, 127, No. 1, 17 (1959).
\(^{4}\) E. B. Dynkin, Proc. of the Fourth Berkeley Symposium on Math. Statistics and Probability, 1960.
\(^{5}\) G. A. Hunt, Ill. J. Math., 1, Nos. 1, 3 (1957); 2, No. 2 (1958).