In the present paper we study three aspects in the theory of non-homogeneous Markov systems under the continuous-time formulation. Firstly, the relationship between stability and quasi-stationarity is investigated and conditions are provided for a quasi-stationary structure to be stable. Secondly, the concept of asymptotic attainability is studied and the possible regions of asymptotically attainable structures are determined. Finally, the cyclic case is considered, where it is shown that for a system in a periodic environment, the relative structure converges to a periodic vector, independently of the initial distribution. Two numerical examples illustrate the above theoretical results.

We present a solution to a special system of Kolmogorov forward equations. We use this result to present a useful expression for the transition probabilities of the extended simple stochastic epidemic model and an epidemic model involving cross-infection between two otherwise isolated groups.

The survival distribution of a device subject to a sequence of shocks occurring randomly over time is studied by Esary, Marshall and Proschan (1973) and by A-Hameed and Proschan (1973), (1975). The present note treats the case in which shocks occur according to a homogeneous Poisson cluster process. It is shown that if [the device survives k shocks] = zk, 0 < z < 1, then the device exhibits a decreasing failure rate. A DFR preservation theorem is proved for completely monotonic . A counterexample to the IFR preservation theorem is given in which is strictly IFR while the failure rate is initially decreasing and then increasing.

This paper deals with a class of discrete-time Markov chains for which the invariant measures can be expressed in terms of generalized continued fractions. The representation covers a wide class of stochastic models and is well suited for numerical applications. The results obtained can easily be extended to continuous-time Markov chains.

We shall provide a classification of states for processes other than the Feller minimal process. In particular, we shall study the regularity, recurrence, transience, etc. of these processes. Our results are based on the observation that certain honest Q-processes exhibit regenerative behaviour. We shall also comment on the problem of determining stationary distributions directly from the transition rates.

The ‘snake' theorem, i.e., the weak convergence of the complex quantogram, is studied. The moment assumption is weakened by a strong approximation result of the author.

We consider general queueing models dealing with multiple classes of customers and address the question under what conditions and in what (stochastic) sense the marginal increase in various performance measures, resulting from the addition of a new class of customers to an existing system, is larger than if the same class were added to a system dealing with only a subset of its current customer base.

Our results enhance our understanding of the dependence of various performance measures with respect to the composition of the customer base. In addition they translate readily into convexity results in an (appropriately defined) arrival rate.

We consider a Brownian motion with time-reversible Markov-modulated speed and two reflecting barriers. A methodology depending on a certain multidimensional martingale together with some linear algebra is applied in order to explicitly compute the stationary distribution of the joint process of the content level and the state of the underlying Markov chain. It is shown that the stationary distribution is such that the two quantities are independent. The long-run average push at the two barriers at each of the states is also computed.

Homogeneous additive processes on a finite or semi-infinite interval have been studied in many forms. Wald's identity for the first passage process on the finite interval (see for example Miller, 1961), the waiting time process of Lindley (1952), and a variety of problems in the theory of queues, dams, and inventories come to mind. These processes have been treated by and large by methods in the complex plane. Lindley's discrete parameter process on the continuum, for example, described by where the ξn are independent identically distributed random variables, has been discussed by Wiener-Hopf methods in recent years by Lindley (1952), Smith (1953), Kemperman (1961), Keilson (1961), and many others. A review of earlier studies is given by Kemperman.

In this note a renewal density theorem in the multi-dimensional case is formulated and proved. Let f(x) be the density function of a p-dimensional random variable with positive mean vector μ and positive-definite covariance matrix Σ, let hn(x) be the n-fold convolution of f(x) with itself, and set Then for arbitrary choice of integers k1, …, kp–1 distinct or not in the set (1, 2, …, p), it is shown that under certain conditions as all elements in the vector x = (x1, …, xp) become large. In the above expression μ‵ is interpreted as a row vector and μ a column vector. An application to the theory of a class of age-dependent branching processes is also presented.

This paper treats stopping problems on Markov chains in which the OLA (one-step look ahead) policy is optimal. Its associated optimal value can be explicitly expressed by a potential for a charge function of the difference between the immediate reward and the one-step-after reward. As an application to the best choice problem, we shall obtain the value of three problems: the classical secretary problem, a problem with a refusal probability and a problem with a random number of objects.

The centred, periodic, stationary Gaussian process X(z), ≧ z ≧ 1 with covariances , appears when one studies the solutions of the vibrating string equation forced by noise, corresponding to the case of a finite string with the extremes tied together. The close relationship between this process and a Brownian bridge permits us to compute the distribution of the maximum excursion of the string at particular times.

Sharp upper and lower reliability bounds for the ℒ- and -classes of life distributions with a given mean are provided. Bounds are also given for a distribution which is related to a known distribution (not necessarily exponential) through the Laplace order.

The reduced Markov branching process is a stochastic model for the genealogy of an unstructured biological population. Its limit behavior in the critical case is well studied for the Zolotarev-Slack regularity parameter α ∈ (0, 1]. We turn to the case of very heavy-tailed reproduction distribution α = 0 assuming that Zubkov's regularity condition holds with parameter β ∈ (0, ∞). Our main result gives a new asymptotic pattern for the reduced branching process conditioned on nonextinction during a long time interval.

Motivated by the question of the age in a branching population we try to recreate the past by looking back from the currently observed population size. We define a new backward Galton-Watson process and study the case of the geometric offspring distribution with parameter p in detail. The backward process is then the Galton-Watson process with immigration, again with a geometric offspring distribution but with parameter 1-p, and it is also the dual to the original Galton-Watson process. We give the asymptotic distribution of the age when the initial population size is large in supercritical and critical cases. To this end, we give new asymptotic results on the Galton-Watson immigration processes stopped at zero.

In this paper we investigate a class of N-person nonconstant sum red-and-black games with bet-dependent win probability functions. We assume that N players and a gambling house are engaged in a game played in stages, where the player's probability of winning at each stage is a function f of the ratio of his bet to the sum of all the players' bets. However, at each stage of the game there is a positive probability that all the players lose and the gambling house wins their bets. We prove that if the win probability function is super-additive and it satisfies f(s)f(t)≤f(st), then a bold strategy is optimal for all players.

Consider a workstation with one server, performing jobs with a service time, Y, having distribution function, G(t). Assume that the station is unreliable, in that it occasionally breaks down. The station is instantaneously repaired, and the server restarts the uncompleted job from the beginning. Let T denote the time it takes to complete each job. If G(t) is exponential with parameter A, then because of the lack-of-memory property of the exponential, P (T > t) = Ḡ(t) =exp(−γt), irrespective of when and how the failures occur. This property also characterizes the exponential distribution.

A supercritical position-dependent Markov branching process has been used as an approximation to a model describing the initial geographical spread of a measles epidemic (Bartlett (1956)). Let α be its Malthusian parameter, ß its velocity of propagation, Z(A, t) the number of individuals in the set A at time t, and A√(ßt) = [√(ßt) r: r ∈ A]. The mean square convergence of the random variable W(A, t)= e–αtZ(A√(ßt), t) to a limit variable W(A) is established.

In this paper, the class of periodic Ornstein-Uhlenbeck processes is defined. It is shown that periodic Ornstein-Uhlenbeck processes are stationary Markov random fields and the class of stationary distributions is characterized. In particular, any self-decomposable distribution is the stationary distribution of some periodic Ornstein-Uhlenbeck process. As examples, gamma periodic Ornstein-Uhlenbeck processes and Gaussian periodic Ornstein-Uhlenbeck processes are considered.