Hostname: page-component-cb9f654ff-nr592 Total loading time: 0 Render date: 2025-08-18T22:38:21.315Z Has data issue: false hasContentIssue false
  • English
  • Français

Sensitivity analysis for Markov reward structures until entrance times

Part of: Markov processes Operations research and management science Stochastic processes

Published online by Cambridge University Press:  14 July 2016

N. M. van Dijk  and
H. Korezlioglu
N. M. van Dijk*
Affiliation:
University of Amsterdam
H. Korezlioglu*
Affiliation:
Ecole Nationale Supérieure des Télécommunications
*
∗ Postal address: Department of Econometrics, University of Amsterdam, Roeterstraat 11, 1018 WB, Amsterdam, The Netherlands
∗∗ Postal address: Ecole Nationale Supérieure des Télécommunications, 46 rue Barrault, 75634 Paris Cedex 13, France. Email address: korez@inf.enst.fr
Get access Rights & Permissions [Opens in a new window]

Abstract

This work presents an estimate of the error on a cumulative reward function until the entrance time of a continuous-time Markov chain into a set, when the infinitesimal generator of this chain is perturbed. The derivation of an error bound constitutes the first part of the paper while the second part deals with an application where the time until saturation is considered for a circuit switched network which starts from an empty state and which is also subject to possible failures.

Keywords

Continuous-time Markov chaincumulative rewardentrance timeparameter sensitivityerror boundbias termsuniformization

MSC classification

Primary: 60J27: Continuous-time Markov processes on discrete state spaces 60G40: Stopping times; optimal stopping problems; gambling theory
Secondary: 90B22: Queues and service 90B25: Reliability, availability, maintenance, inspection

Information

Type
Research Papers
Information
Journal of Applied Probability , Volume 37 , Issue 1 , March 2000 , pp. 45 - 63
Copyright
Copyright © 2000 by The Applied Probability Trust 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Baccelli, B. and Brémaud, P. (1993). Virtual customers in sensitivity and light traffic analysis via Campbell's formula for point processes. Adv. Appl. Prob. 25, 221–234.Google Scholar
Brémaud, P., and Vasquez-Abad, J. (1992). On the pathwise computation of derivatives with respect to the rate of a point process: the phantom RPA method. Queueing Systems 10, 249–270.Google Scholar
Cao, X. R. (1992). A new method of performance sensitivity analysis for non-Markovian queueing networks. Queueing Systems 10, 313–350.Google Scholar
Decreusefond, L. (1998). Perturbation analysis and Malliavin calculus. Ann. Appl. Prob. 8, 496–523.Google Scholar
van Dijk, N. M. (1988). Simple bounds for queueing systems with breakdowns. Perf. Eval. 8, 117–128.Google Scholar
van Dijk, N. M. (1990). On the importance of bias-terms for error bounds and comparison results. In Proc. 1st Int. Conf. on Numerical Solutions of Markov Chains, Raleigh, pp. 640–654.Google Scholar
van Dijk, N. M. (1991). Transient error bound analysis for continuous-time Markov reward structures. Perf. Eval. 13, 147–158.Google Scholar
van Dijk, N. M. (1992). Approximate uniformization for continuous-time Markov chains with an application to performability analysis. Stoch. Proc. Appl. 40, 339–357.Google Scholar
van Dijk, N. M., and Puterman, M. L. (1988). Perturbation theory for Markov reward processes with applications to queueing systems. Adv. Appl. Prob. 20, 79–89.Google Scholar
van Dijk, N. M. and van der Wal, J. (1989). Simple bounds and monotonicity results for multi-server exponential tandem queues. Queuing Systems 4, 1–19.Google Scholar
Glasserman, P. (1992). Gradient Estimation via Perturbation Analysis. Kluwer, Dordrecht.Google Scholar
Glynn, P. W. (1987). Likelihood ratio gradient simulation: an overview. Proc. 1987 Winter Simulation Conf., pp. 366–375.Google Scholar
Grassi, V., and Donatiello, L. (1992). Sensitivity analysis of performability. Perf. Eval. 14, 227–239.Google Scholar
Grassmann, W. (1990). Finding transient solutions in Markovian event systems through randomization. In Proc. 1st Int. Conf. on Numerical Solutions of Markov Chains, Raleigh, pp. 375–395.Google Scholar
Heidelberger, P., and Goyal, A. (1988). Sensitivity analysis of continuous-time Markov chains using uniformization. In Computer Performance and Reliability, North-Holland, Amsterdam, pp. 93–104.Google Scholar
Hinderer, K. (1978). On approximate solutions of finite-stage dynamic programs. In Dynamic Programming and its Applications, ed. Puterman, M. Academic Press, New York, pp. 298–318.Google Scholar
Ho, Y. C., and Cao, X. R. (1983). Perturbation analysis and optimization of queueing networks. J. Optim. Theory Appl. 40, 559–582.Google Scholar
Karlin, S., and Taylor, H. M. (1981). A Second Course in Stochastic Processes. Academic Press, New York.Google Scholar
Kemeny, J. G., Snell, J. L., and Knapp, A. W. (1966). Denumerable Markov Chains. Van Nostrand, New York.Google Scholar
Liu, Z., and Nain, P. (1991). Sensitivity results in open, closed mixed product form queueing networks. Perf. Eval. 13, 237–251.Google Scholar
Meyer, C. D. Jr. (1980). The condition of a finite Markov chain and perturbation bounds for the limiting probabilities. SIAM J. Alg. Discrete Methods 1, 273–283.Google Scholar
Reibman, A., Trivedi, K. S., and Smith, R. (1989). Markov and Markov reward model transient analysis: an overview of numerical approaches. Eur. J. Operat. Res. 40, 257–267.Google Scholar
Reiman, M. I., and Weiss, A. (1989). Sensitivity analysis via likelihood ratios. Operat. Res. 37, 830–844.Google Scholar
Rubinstein, R. (1989). Sensitivity analysis and performance extrapolation for computer simulation models. Operat. Res. 37, 72–81.Google Scholar
Schweitzer, P. J. (1968). Perturbation theory and finite Markov chains. J. Appl. Prob. 5, 401–413.CrossRefGoogle Scholar
Shantikumar, J. G., and Yao, D. D. (1987). Stochastic monotonicity of the queue lengths in closed queueing networks. Operat. Res. 35, 583–588.Google Scholar
Stoyan, D. (1983). Comparison Methods for Queues and other Stochastic Models. John Wiley, New York.Google Scholar
Suri, R. (1983). Robustness of queueing network formulas. J. Assoc. Comp. Mach. 30, 564–594.Google Scholar
Tsoucas, P., and Walrand, J. (1989). Monotonicity of throughput in non-Markovian networks. J. Appl. Prob. 26, 134–141.Google Scholar
Whitt, W. (1978). Approximations of dynamic programs I. Math. Operat. Res. 3, 231–243.Google Scholar