Optimal preventive maintenance for systems having a continuous output and operating in a random environment

Abstract

We consider systems that are operating in a random environment modeled by an external shock process. Performance of a system is characterized by a quality (output) function that is decreasing (due to degradation) in the absence of shocks. Shocks have a double impact, i.e., they affect the failure rate of a system directly and at the same time, and each shock contributes to the additional decrease in the quality function. The unconditional and conditional (on survival) expectations for the corresponding stochastic quality process are obtained. The system is replaced either on failure or on the predetermined replacement time, whichever comes first. The corresponding optimization problem is considered and illustrated by detailed numerical examples.

Appendix

Proof of Proposition 1

Observe that

$$ Q_{{E}} (t) = E[\tilde{Q}(t) \cdot I(t)] = E[E[\tilde{Q}(t) \cdot I(t)|N(t),T_{1} ,T_{2} , \ldots ,T_{N(t)} ;W_{1} ,W_{2} , \ldots ,W_{N(t)} ]], $$

where, given the shock process, $\tilde{Q}(t)$ becomes just constant and

$$ \begin{aligned} & E[\tilde{Q}(t) \cdot I(t)|N(t),T_{1} ,T_{2} , \ldots ,T_{{N(t)}} ;W_{1} ,W_{2},\ldots,W_{N(t)}] \\ & = Q(t)\left[ {\prod\limits_{{i = 1}}^{{N(t)}} {\exp \{ - \psi (T_{i} )\} } } \right] \cdot \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\} \cdot {\text{ }}\exp \left\{ { - \int\limits_{0}^{t} {\sum\limits_{1}^{{N(u)}} {W_{i} h(u - T_{i} )} {\text{d}}u} } \right\} \\ & = Q(t)\left[ {\prod\limits_{{i = 1}}^{{N(t)}} {\exp \{ - \psi (T_{i} )\} } } \right] \cdot \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\} \cdot \exp \left\{ { - \sum\limits_{{i = 1}}^{{N(t)}} {W_{i} H(t - T_{i} )} } \right\}. \\ \end{aligned} $$

The joint distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(t)},N(t); W_{1},W_{2} , \ldots ,W_{N(t)})$ is

$$ \begin{aligned} & f_{(T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t);W_{1} ,W_{2} , \ldots ,W_{N(t)} )}\;(t_{1} ,t_{2} , \ldots ,t_{n} ,n;w_{1} ,w_{2} , \ldots ,w_{n} ) \\ & = \lambda (t_{1} )\exp \left\{ { - \int\limits_{0}^{{t_{1} }} {\lambda (u){\text{d}}u} } \right\}\lambda (t_{2} )\exp \left\{ { - \int\limits_{{t_{1} }}^{{t_{2} }} {\lambda (u){\text{d}}u} } \right\} \ldots \\ & \quad \times \lambda (t_{n} )\exp \left\{ { - \int\limits_{{t_{n - 1} }}^{{t_{n} }} {\lambda (u){\text{d}}u} } \right\}\exp \left\{ { - \int\limits_{{t_{n} }}^{t} {\lambda (u){\text{d}}u} } \right\} \cdot \left( {\prod\limits_{i = 1}^{n} {f_{W} (w_{i} )} } \right) \\ & = \left( {\prod\limits_{i = 1}^{n} {\lambda (t_{i} )f_{W} (w_{i} )} } \right)\exp \left\{ { - \int\limits_{0}^{t} {\lambda (u){\text{d}}u} } \right\},\\ &\quad 0 < t_{1} < t_{2} < \ldots < t_{n} \le t,n = 1,2, \ldots ,\;0 \le w_{i} < \infty ,\;i = 1,2, \ldots . \\ \end{aligned} $$

Thus, $Q_{{E}} (t)$ can be obtained as

$$ \begin{aligned} Q_{{E}} (t) & = \sum\limits_{n = 0}^{\infty } {} \int\limits_{0}^{t} \ldots \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {\int\limits_{0}^{\infty } \ldots \int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {} } E[\tilde{Q}(t) \cdot I(t)|N(t),T_{1} ,T_{2} , \ldots ,T_{N(t)} ;W_{1} ,W_{2} , \ldots ,W_{N(t)} ]} } \\ & \quad \times f_{{T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t);W_{1} ,W_{2} , \ldots ,W_{N(t)} }} (t_{1} ,t_{2} , \ldots ,t_{n} ,n;w_{1} ,w_{2} , \ldots ,w_{n} ){\text{d}}w_{1} {\text{d}}w_{2} \ldots {\text{d}}w_{n} {\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n} \\ & = Q(t)\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\} \\ & \quad \times \sum\limits_{n = 0}^{\infty } {} \int\limits_{0}^{t} \ldots \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {} } \int\limits_{0}^{\infty } \ldots \int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {} } \prod\limits_{i = 1}^{n} {\lambda (t_{i} )} f_{W} (w_{i} )\exp \left\{ { - w_{i} H(t - t_{i} ) - \psi (t_{i} ))} \right\}{\text{d}}w_{1} {\text{d}}w_{2} \ldots {\text{d}}w_{n} {\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n} \\ & = Q(t)\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\} \\ & \quad \times \sum\limits_{n = 0}^{\infty } {} \int\limits_{0}^{t} \ldots \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {} } \prod\limits_{i = 1}^{n} {\lambda (t_{i} )M_{W} ( - H(t - t_{i} ))} \exp \left\{ { - \psi (t_{i} )} \right\}{\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n} \\ & = Q(t)\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\}\sum\limits_{n = 0}^{\infty } {\frac{{\left( {\int\nolimits_{0}^{t} {M_{W} ( - H(t - x))\exp \{ - \psi (x)\} \lambda (x){\text{d}}x} } \right)^{n} }}{n!}} \\ & = Q(t)\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\}\exp \left\{ {\int\limits_{0}^{t} {M_{W} ( - H(t - x))\exp \{ - \psi (x)\} \lambda (x){\text{d}}x} } \right\}, \\ \end{aligned} $$

where, in obtaining the third equality, the following property is used: for any integrable function $\delta (x)$,

$$ \int\limits_{0}^{t} \ldots \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {} } \prod\limits_{i = 1}^{n} {\delta (t_{i} )} {\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n} = \frac{{\left( {\int_{0}^{t} {\delta (x){\text{d}}x} } \right)^{n} }}{n!}. $$

□

Proof of Proposition 2

Observe that $\tilde{Q}(t)$ is a function of $(T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t))$; therefore, to obtain $Q_{{ES}} (t)$, we need the joint conditional distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t)|T_{l} > t)$. The conditional survival function given the shocks and random increment history is

$$ P(T_{l} > t|T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t);W_{1} ,W_{2} , \ldots ,W_{N(t)} ) = \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \sum\limits_{i = 1}^{N(t)} {W_{i} H(t - T_{i} )} } \right\}. $$

Thus, the joint distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t);W_{1} ,W_{2} , \ldots ,W_{N(t)} ,T_{l} > t)$ is given by

$$ \begin{aligned} & \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \sum\limits_{i = 1}^{n} {w_{i} H(t - t_{i} )} } \right\} \cdot \left( {\prod\limits_{i = 1}^{n} {\lambda (t_{i} )} f_{W} (w_{i} )} \right)\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\}, \\ & \quad \quad \quad 0 < t_{1} < t_{2} < \cdots < t_{n} \le t,n = 1,2, \ldots ,\quad 0 \le w_{i} < \infty ,\quad i = 1,2, \ldots . \\ \end{aligned} $$

The survival probability $P(T_{l} > t)$ is obtained as

$$ \begin{aligned} P(T_{\text{l}} > t) & = \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\} \\ & \quad \times \sum\limits_{n = 0}^{\infty } {\int\limits_{0}^{t} \ldots } \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {\int\limits_{0}^{\infty } \ldots } } \int\limits_{0}^{\infty } {\int\limits_{0}^{\infty } {\prod\limits_{i = 1}^{n} {\lambda (t_{i} )} f_{W} (w_{i} ) \times \exp \left\{ { - w_{i} H(t - t_{i} )} \right\}{\text{d}}w_{1} {\text{d}}w_{2} \ldots {\text{d}}w_{n} {\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n} } } \\ & = \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\}\exp \left\{ {\int\limits_{0}^{t} {M_{W} ( - H(t - x))\lambda (x){\text{d}}x} } \right\}. \\ \end{aligned} $$

Then, the joint conditional distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t);W_{1} ,W_{2} , \ldots ,W_{N(t)} |T_{l} > t)$ is given by

$$ \begin{aligned} & \frac{1}{{P(T_{l} > t)}}\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\}\exp \left\{ { - \sum\limits_{i = 1}^{n} {w_{i} H(t - t_{i} )} } \right\} \cdot \left( {\prod\limits_{i = 1}^{n} {\lambda (t_{i} )} f_{W} (w_{i} )} \right), \\ & 0 < t_{1} < t_{2} < \cdots < t_{n} \le t,n = 1,2, \ldots ,\quad 0 \le w_{i} < \infty ,\quad i = 1,2, \ldots \\ \end{aligned} $$

and, thus, the joint conditional distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t)|T_{{l}} > t)$ is

$$ \begin{aligned} & \frac{1}{{P(T_{l} > t)}}\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\} \cdot \left( {\prod\limits_{i = 1}^{n} {M_{W} ( - H(t - t_{i} ))\lambda (t_{i} )} } \right) \\ & 0 < t_{1} < t_{2} < \cdots < t_{n} \le t,n = 1,2, \ldots \\ \end{aligned} $$

Finally, using the joint conditional distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(t)} ,N(t)|T_{l} > t)$,

$$ \begin{aligned} Q_{{ES}} (t) & = E[\tilde{Q}(t)|T_{{l}} > t] = E\left[ {Q(t)\prod\limits_{i = 1}^{N(t)} {\exp \{ - \psi (T_{i} )\} |T_{{l}} > t} } \right] \\ & = \frac{1}{{P(T_{{l}} > t)}}\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (u){\text{d}}u} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (u){\text{d}}u} } \right\} \\ & \quad \times \sum\limits_{n = 0}^{\infty } {\int\limits_{0}^{t} \ldots } \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {\left( {\prod\limits_{i = 1}^{n} {M_{W} ( - H(t - t_{i} ))\lambda (t_{i} )} } \right)} } Q(t)\prod\limits_{i = 1}^{n} {\exp \{ - \psi (t_{i} )\} {\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n} } \\ & = Q(t)\exp \left\{ {\int\limits_{0}^{t} {M_{W} ( - H(t - x))\exp \{ - \psi (x)\} \lambda (x){\text{d}}x} - \int\limits_{0}^{t} {M_{W} ( - H(t - x))\lambda (x){\text{d}}x} } \right\}, \\ \end{aligned} $$

where, as in the previous proof, the property

$$ \int\limits_{0}^{t} \ldots \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {\prod\limits_{i = 1}^{n} {\delta (t_{i} )} {\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n} } } = \frac{{\left( {\int_{0}^{t} {\delta (x){\text{d}}x} } \right)^{n} }}{n!} $$

was used.□

Proof of Proposition 4

In a renewal cycle, the expected cost due to preventive replacement and the replacement upon failure are given by $C_{{r}} \bar{F}(T) + C_{{f}} F(T)$. Thus, now we derive the average gain in a cycle. For this, we have to derive the conditional average quality at time $s$ given $T_{l} > T$(where $s < T$) or $T_{{l}} = t$, $t \le T$ (where $s < t$).

(Case I) $T_{l} > T$.

Define $Q_{\text{ES}} (s|T + ) \equiv E[\tilde{Q}(s)|T_{{l}} > T]$, $s < T$. Observe that $\tilde{Q}(s)$ is a function of $(T_{1} ,T_{2} , \ldots ,T_{N(s)} ,N(s))$, and to obtain $Q_{\text{ES}} (s|T + )$, we need the joint conditional distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(s)} ,N(s)|T_{{l}} > t)$. For convenience, define

$$ \begin{aligned} &H_{s,t} \equiv (T_{1} = t_{1} ,T_{2} = t_{2} , \ldots ,T_{N(s)} = t_{n(s)} ,T_{N(s) + 1} = t_{n(s) + 1} ,\\ &\qquad\quad T_{N(s) + 2} = t_{n(s) + 2} , \ldots ,T_{N(t)} = t_{n(t)} ,N(s) = n(s),N(t) = n(t)). \\ \end{aligned} $$

Observe that the conditional probability of $(T_{{l}} > t|H_{{{\text{s}},t}} )$ is given by

$$ \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\prod\limits_{i = 1}^{n(s)} {\exp \{ - \eta (t - t_{i} )\} \cdot } \prod\limits_{i = n(s) + 1}^{n(t)} {\exp \{ - \eta (t - t_{i} )\} } . $$

and thus the conditional distribution of $(H_{s,t} |T_{l} > t)$ is

$$ \begin{aligned} & \frac{1}{{P(T_{{l}} > t)}}\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\prod\limits_{i = 1}^{n(s)} {\exp \{ - \eta (t - t_{i} )\} \cdot } \prod\limits_{i = n(s) + 1}^{n(t)} {\exp \{ - \eta (t - t_{i} )\} } \\ & \quad \times \left( {\prod\limits_{i = 1}^{n(s)} {\lambda (t_{i} )} } \right)\exp \left\{ { - \int\limits_{0}^{s} {\lambda (x){\text{d}}x} } \right\}\left( {\prod\limits_{i = n(s) + 1}^{n(t)} {\lambda (t_{i} )} } \right)\exp \left\{ { - \int\limits_{s}^{t} {\lambda (x){\text{d}}x} } \right\} \\ & = \frac{{\prod\nolimits_{i = 1}^{n(s)} {} \lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} \cdot \prod\nolimits_{i = n(s) + 1}^{n(t)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } }}{{\exp \left\{ {\int_{0}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}}}. \\ \end{aligned} $$

Thus, the conditional joint distribution of $(T_{1} = t_{1} ,T_{2} = t_{2} , \ldots ,T_{n(s)} = t_{n(s)} ,N(s) = n(s)|T_{{l}} > t)$ is

$$ \begin{aligned} & \frac{{\prod\nolimits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } }}{{\exp \left\{ {\int_{0}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}}}\sum\limits_{n(t) = n(s)}^{\infty } {\int\limits_{s}^{t} \ldots } \int\limits_{s}^{{t_{n(s) + 3} }} {\int\limits_{s}^{{t_{n(s) + 2} }} {\prod\limits_{i = n(s) + 1}^{n(t)} {} \lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} {\text{d}}t_{n(s) + 1} {\text{d}}t_{n(s) + 2} \ldots {\text{d}}t_{n(t)} } } \\ & = \frac{{\prod\nolimits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } }}{{\exp \left\{ {\int_{0}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}}}\sum\nolimits_{n(t) = n(s)}^{\infty } {\frac{{\left( {\int_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right)^{n(t) - n(s)} }}{(n(t) - n(s))!}} \\ & = \frac{{\prod\nolimits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } }}{{\exp \left\{ {\int_{0}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}}}\exp \left\{ {\int_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\} \\ & = \frac{{\prod\nolimits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } }}{{\exp \left\{ {\int_{0}^{s} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}}}. \\ \end{aligned} $$

Thus,

$$ \begin{aligned} Q_{\text{ES}} (s|T + ) & = E[\tilde{Q}(s)|T_{l} > T] = E\left[ {Q(s)\prod\limits_{i = 1}^{N(s)} {} \exp \{ - \psi (T_{i} )\} |T_{l} > T} \right] \\ & = Q(s)\sum\limits_{n(s) = 0}^{\infty } {\int\limits_{0}^{s} \ldots } \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {\frac{{\prod\nolimits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (T - t_{i} ) - \psi (t_{i} )\} } }}{{\exp \left\{ {\int_{0}^{s} {\exp \{ - \eta (T - x)\} \lambda (x){\text{d}}x} } \right\}}}{\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n(s)} } } \\ & = Q(s)\exp \left\{ {\int\limits_{0}^{s} {\exp \{ - \eta (T - x) - \psi (x)\} \lambda (x){\text{d}}x} - \int\limits_{0}^{s} {\exp \{ - \eta (T - x)\} \lambda (x){\text{d}}x} } \right\}. \\ \end{aligned} $$

Given $T_{l} > T$, the conditional total average gain during the cycle is

$$ \kappa \int\limits_{0}^{T} {Q_{\text{ES}} (s|T + ){\text{d}}s} . $$

(Case II) $T_{l} \le T$.

For $t \le T$, define $Q_{\text{ES}} (s|t) \equiv E[\tilde{Q}(s)|T_{{l}} = t]$, $s < t$. In this case, to obtain $Q_{\text{ES}} (s|t)$, we need the joint conditional distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(s)} ,N(s)|T_{{l}} = t)$.

Observe that

$$ P(t < T_{{l}} \le t + \Delta t|H_{{{\text{s}}:t}} ) = P(t < T_{{l}} \le t + \Delta t|T_{{l}} > t,H_{{{\text{s}}:t}} )P(T_{{l}} > t|H_{{{\text{s}}:t}} ), $$

where

$$ P(T_{{l}} > t|H_{{{\text{s}}:t}} ) = \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\prod\limits_{i = 1}^{n(s)} {\exp \{ - \eta (t - t_{i} )\} } \cdot \prod\limits_{i = n(s) + 1}^{n(t)} {\exp \{ - \eta (t - t_{i} )\} } . $$

When $\Delta t \approx 0$,

$$ P(t < T_{{l}} \le t + \Delta t|T_{{l}} > t,H_{s:t} ) = \left( {r_{0} (t) + n(s)\eta + (n(t) - n(s))\eta } \right)\Delta t. $$

Thus,

$$ \begin{aligned} P(t < T_{{l}} \le t + \Delta t|H_{{{\text{s}}:t}} ) & = P(t < T_{{l}} \le t + \Delta t|T_{{l}} > t,H_{{{\text{s}}:t}} )P(T_{{l}} > t|H_{{{\text{s}}:t}} ) \\ & = \left( {r_{0} (t) + n(s)\eta + (n(t) - n(s))\eta } \right) \\ & \quad \times \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\prod\limits_{i = 1}^{n(s)} {\exp \{ - \eta (t - t_{i} )\} } \cdot \prod\limits_{i = n(s) + 1}^{n(t)} {\exp \{ - \eta (t - t_{i} )\} \Delta t} \\ \end{aligned} $$

and, accordingly, the conditional pdf of $(T_{{l}} |H_{{{\text{s}}:t}} )$ is given by

$$ \begin{aligned} & \left( {r_{0} (t) + n(s)\eta + (n(t) - n(s))\eta } \right) \\ & \quad \times \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\prod\limits_{i = 1}^{n(s)} {\exp \{ - \eta (t - t_{i} )\} } \cdot \prod\limits_{i = n(s) + 1}^{n(t)} {\exp \{ - \eta (t - t_{i} )\} } . \\ \end{aligned} $$

Thus, the joint distribution of $(T_{{l}} ,H_{{{\text{s}}:t}} )$ is

$$ \begin{aligned} & \left( {r_{0} (t) + n(s)\eta + (n(t) - n(s))\eta } \right) \\ & \quad \times \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\prod\limits_{i = 1}^{n(s)} {\exp \{ - \eta (t - t_{i} )\} } \cdot \prod\limits_{i = n(s) + 1}^{n(t)} {\exp \{ - \eta (t - t_{i} )\} } \\ & \quad \times \left( {\prod\limits_{i = 1}^{n(s)} {\lambda (t_{i} )} } \right)\exp \left\{ { - \int\limits_{0}^{s} {\lambda (x){\text{d}}x} } \right\}\left( {\prod\limits_{i = n(s) + 1}^{n(t)} {\lambda (t_{i} )} } \right)\exp \left\{ { - \int\limits_{s}^{t} {\lambda (x){\text{d}}x} } \right\}, \\ \end{aligned} $$

From this, the conditional distribution of $(H_{{{\text{s}}:t}} |T_{{l}} )$ can be obtained as

$$ \begin{aligned} & \frac{1}{f(t)}\left( {r_{0} (t) + n(s)\eta + (n(t) - n(s))\eta } \right)\prod\limits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } \cdot \prod\limits_{i = n(s) + 1}^{n(t)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } \\ & \quad \times \exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\}. \\ \end{aligned} $$

The joint conditional distribution of $(T_{1} ,T_{2} , \ldots ,T_{N(s)} ,N(s)|T_{{l}} = t)$ is given by

$$ \begin{aligned} &\frac{1}{f(t)}\sum\limits_{n(t) = n(s)}^{\infty } {\left( {r_{0} (t) + n(s)\eta + (n(t) - n(s))\eta } \right)} \prod\limits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp ( - \eta (t - t_{i} ))} \hfill \\ & \quad \times \frac{{\left( {\int_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right)^{n(t) - n(s)} }}{(n(t) - n(s))!}\exp \left\{ { - \int\limits_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int\limits_{0}^{t} {\lambda (x){\text{d}}x} } \right\} \hfill \\ & = \frac{{\exp \left\{ { - \int_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int_{0}^{t} {\lambda (x){\text{d}}x} } \right\}}}{f(t)} \hfill \\ & \quad \times \left[ {\left( {r_{0} (t) + n(s)\eta } \right)\prod\limits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } \cdot \exp \left\{ {\int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}} \right. \hfill \\ & \quad \left. { + \eta \prod\limits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} )\} } \cdot \int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} \cdot \exp \left\{ {\int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}} \right] \hfill \\ \end{aligned} $$

Thus,

$$ \begin{aligned} Q_{\text{ES}} (s|t) \equiv E[\tilde{Q}(s)|T_{{l}} = t] &= E\left[ {Q(s)\prod\limits_{i = 1}^{N(s)} {\exp \{ - \psi (T_{i} )\} |T_{{l}} = t} } \right] \hfill \\ & = \frac{{Q(s)\exp \left\{ { - \int_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int_{0}^{t} {\lambda (x){\text{d}}x} } \right\}}}{f(t)} \hfill \\ & \quad \times \left[ {\sum\limits_{n(s) = 0}^{\infty } {\left( {r_{0} (t) + n(s)\eta } \right)} \int\limits_{0}^{s} \ldots \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {\prod\limits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} ) - \psi (t_{i} )\} {\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n(s)} } } } } \right. \hfill \\ & \quad \times \exp \left\{ {\int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\} \hfill \\ & \quad + \eta \sum\limits_{n(s) = 0}^{\infty } {\int\limits_{0}^{s} \ldots } \int\limits_{0}^{{t_{3} }} {\int\limits_{0}^{{t_{2} }} {\prod\limits_{i = 1}^{n(s)} {\lambda (t_{i} )\exp \{ - \eta (t - t_{i} ) - \psi (t_{i} )\} {\text{d}}t_{1} {\text{d}}t_{2} \ldots {\text{d}}t_{n(s)} } } } \hfill \\ & \quad \left. { \times \int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} \cdot \exp \left\{ {\int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}} \right] \hfill \\ &= \frac{{Q(s)\exp \left\{ { - \int_{0}^{t} {r_{0} (x){\text{d}}x} } \right\}\exp \left\{ { - \int_{0}^{t} {\lambda (x){\text{d}}x} } \right\}}}{f(t)}\left[ {\left( {r_{0} (t)\exp \left\{ {\int\limits_{0}^{s} {\exp \{ - \eta (t - x) - \psi (x)\} \lambda (x){\text{d}}x} } \right\}} \right.} \right. \hfill \\ & \quad \left. { + \eta \int\limits_{0}^{s} {\exp \{ - \eta (t - x) - \psi (x)\} \lambda (x){\text{d}}x \cdot } \exp \left\{ {\int\limits_{0}^{s} {\exp \{ - \eta (t - x) - \psi (x)\} \lambda (x){\text{d}}x} } \right\}} \right) \hfill \\ & \quad \times \exp \left\{ {\int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\} \hfill \\ & \quad + \eta \exp \left\{ {\int\limits_{0}^{s} {\exp \{ - \eta (t - x) - \psi (x)\} \lambda (x){\text{d}}x} } \right\} \hfill \\ & \quad \left. { \times \int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} \cdot \exp \left\{ {\int\limits_{s}^{t} {\exp \{ - \eta (t - x)\} \lambda (x){\text{d}}x} } \right\}} \right] \hfill \\ \end{aligned} $$

Thus, given $T_{{l}} \le T$, the conditional total average gain during the cycle is obtained as

$$ \kappa \int\limits_{0}^{T} {\int\limits_{0}^{t} {Q_{\text{ES}} (s|t){\text{d}}s\frac{f(t)}{F(T)}{\text{d}}t} } . $$

Combining Case I and Case II, the average gain during a renewal cycle is given by

$$ \kappa \int\limits_{0}^{T} {Q_{\text{ES}} (s|T + ){\text{d}}s\overline{F} (T)} + \kappa \int\limits_{0}^{T} {\int\limits_{0}^{t} {Q_{\text{ES}} (s|t){\text{d}}sf(t){\text{d}}t} } . $$

The proof is completed.□