We consider a system that gradually deteriorates. At random times Tn the system is inspected and may or may not be replaced, but it must be if its state is too bad. The process of deterioration is described by a Markovian renewal process. Under some monotonicity conditions we derive a policy for the finite and the infinite horizon which minimizes the total discounted costs. This paper is a generalization of a result of Feldman (1977).

Consider a sequence of Bernoulli trials between players A and B in which player A wins each trial with probability p∈ [0, 1]. For positive integers n and k with k ≦ n, an (n, k) contest is one in which the first player to win at least n trials and to be ahead of his opponent by at least k trials wins the contest. The (n, 1) contest is the Banach match problem and the (n, n) contest is the gambler's ruin problem. Many real contests (such as the World Series in baseball and the tennis game) have an (n, 1) or an (n, 2) format. The (n, k) contest is formulated in terms of the first-exit time of the graph of a random walk from a certain region of the state-time space. Explicit results are obtained for the probability that player A wins an (n, k) contest and the expected number of trials in an (n, k) contest. Comparisons of (n, k) contests are made in terms of the probability that the stronger player wins and the expected number of trials.

The study of simple solar energy storage models leads to the question of analyzing the equilibrium distribution of Markov chains (Harris chains), for which the state at epoch (n + 1) (i.e. the temperature of the storage tank) depends on the state at epoch n and on a controlled input, acceptance of which entails a further decrease of the temperature level. Here we study the model where the input is exponentially distributed. For all values of the parameters involved an explicit expression for the equilibrium distribution of the Markov chain is derived, and from this we calculate, as one of the possible applications, the exact values of the mean of this equilibrium distribution.

We examine the existence of limiting behavior, or stability, for storage models with shot noise input and general release rules. The shot noise feature of the input process allows the individual inputs to gradually enter the store.

We first show that a store under the unit release rule is stable if and only if the traffic intensity is less than one; this extends the classic result of Prabhu (1980) to the case of shot noise input. The stability of the unit release rule store and various stochastic orderings are then used to derive a sufficient condition for a store with a general release rule to be stable. Finally, we show that when restricted to a compact state space, our storage model is always stable.

An important component of the paper is the methodology employed: coupling and stochastic monotonicity play a key role in analyzing the non-Markov processes encountered.

Optimal stopping rules under various conditions are obtained for a proofreader who has a probability p (known or unknown) of detecting a misprint in proofsheets which contain an unknown but Poisson-distributed number of misprints.

In this paper, we study the new better than used in expectation (NBUE) and new worse than used in expectation (NWUE) properties of Markov renewal processes. We show that a Markov renewal process belongs to a more general class of stochastic processes encountered in reliability or maintenance applications. We present sufficient conditions such that the first-passage times of these processes are new better than used in expectation. The results are applied to the study of shock and repair models, random repair time processes, inventory, and queueing models.

This paper is concerned with a discrete-time population model in which a new individual entering the population at time t can produce offspring for the first time at time t + 2 and then subsequently at times t + 3, t + 4, ···. The numbers of offspring produced on each occasion are independent random variables each with the distribution of Z for which EZ = m <∞, and individuals have independent lines of descent. This model is contrasted with the corresponding Bienaymé-Galton-Watson one. If Xn denotes the number of individuals in the population at time n, it is shown that z–nXn almost surely converges to a random variable W, as n→∞, where Various properties of W are obtained, in particular W > 0 a.s. if and only if EZ | log Z | < ∞ Results are also given on the rate of convergence of to z when Var Z < ∞ and these display a surprising dependence on the size of z.

The problem of determining the asymptotic form of the stock vector n(t) in a continuous time Markovian manpower model is solved for asymptotically exponential recruitment functions {R(t)}. A new approach to the limiting behaviour of some manpower systems with given total sizes {N(t)} is then given by means of time-inhomogeneous Markov processes.

We prove that a nonstationary random field indexed by ℝn with moments at least of order 2 can always be viewed as second-order stationary in ℝ2n.

By applying a perturbation method to the structured coalescent process in population genetics theory, we obtain the approximate solutions of the moment generating functions for the total coalescence time, the number of segregating sites among sampled DNA sequences and the number of allele types in a sample in the case of strong migration.

We consider some linear regression models Y = Σα lfl(z) + X, where X is an autoregressive (AR) process. The residuals estimate the i.i.d. innovations sequence which drives the AR process. We then consider the partial sum process of the residuals and show they converge to Brownian bridges in certain cases. Some remarks are also made on similar processes when differencing is first applied to remove trends. When an AR process is differenced the residual partial sum can be asymptotically a random polynomial.

We rigorously prove that for a stochastic process, , the existence of a first regeneration time, R1, implies the existence of an infinite sequence of such times, {R1, R2, · ·· }, and hence that the definition of regenerative process need only demand the existence of a first regeneration time. Here we include very general processes up to and including processes where cycles are stationary but not necessarily independent and identically distributed.

For stationary Poisson or Poisson cluster processes ξ on R2 we study the distribution of the interpoint distances using the interpoint distance function and the nearest-neighbor indicator function . Here Sr (x) is the interior of a circle of radius r having center x, I(t) is that subset of D which has x ∊ D and St(x) ⊂ D and χ is the usual indicator function. We show that if the region D ⊂ R2 is large, then these functions are approximately distributed as Poisson processes indexed by and , where µ(D) is the Lebesgue measure of D.

Suppose that in a ballot candidate A scores a votes and candidate B scores b votes and that all the possible voting records are equally probable. Denote by α r and β r, the number of votes registered for A and B respectively among the first r votes recorded. Let µ, be a nonnegative integer. Denote by Pj(a, b) the probability that the inequality holds for exactly j subscripts r=1,2,..,a + b.

We study asymptotically optimal simulation algorithms for approximating the tail probability of P(eX1+⋯+ eXd>u) as u→∞. The first algorithm proposed is based on conditional Monte Carlo and assumes that (X1,…,Xd) has an elliptical distribution with very mild assumptions on the radial component. This algorithm is applicable to a large class of models in finance, as we demonstrate with examples. In addition, we propose an importance sampling algorithm for an arbitrary dependence structure that is shown to be asymptotically optimal under mild assumptions on the marginal distributions and, basically, that we can simulate efficiently (X1,…,Xd|Xj >b) for large b. Extensions that allow us to handle portfolios of financial options are also discussed.

We consider a polling system with two queues, exhaustive service, no switchover times, and exponential service times with rate µ in each queue. The waiting cost depends on the position of the queue relative to the server: it costs a customer c per time unit to wait in the busy queue (where the server is) and d per time unit in the idle queue (where there is no server). Customers arrive according to a Poisson process with rate λ. We study the control problem of how arrivals should be routed to the two queues in order to minimize the expected waiting costs and characterize individually and socially optimal routeing policies under three scenarios of available information at decision epochs: no, partial, and complete information. In the complete information case, we develop a new iterative algorithm to determine individually optimal policies (which are symmetric Nash equilibria), and show that such policies can be described by a switching curve. We use Markov decision processes to compute the socially optimal policies. We observe numerically that the socially optimal policy is well approximated by a linear switching curve. We prove that the control policy described by this linear switching curve is indeed optimal for the fluid version of the two-queue polling system.

Suppose that n persons each know a different piece of information, and that, whenever a pair of them talk on the telephone, each tells the other all the information he knows at that time. If the calls are made at random, we show that the expected number of calls required for everyone to know all n pieces of information is asymptotically 1.5 n log n + O(n). This sharpens an earlier result of D. W. Boyd and J. M. Steele. Some numerical comparisons are given.

Scaling of proposals for Metropolis algorithms is an important practical problem in Markov chain Monte Carlo implementation. Analyses of the random walk Metropolis for high-dimensional targets with specific functional forms have shown that in many cases the optimal scaling is achieved when the acceptance rate is approximately 0.234, but that there are exceptions. We present a general set of sufficient conditions which are invariant to orthonormal transformation of the coordinate axes and which ensure that the limiting optimal acceptance rate is 0.234. The criteria are shown to hold for the joint distribution of successive elements of a stationary pth-order multivariate Markov process.

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

In this paper we consider the stochastic wave equation in one spatial dimension driven by a two-parameter Gaussian noise which is white in time and has general spatial covariance. We give conditions on the spatial covariance of the driving noise sufficient for the string to have finite expected energy and calculate this energy as a function of time. We show that these same conditions on the spatial covariance of the driving noise are also sufficient to guarantee that the energy of the string has a version which is continuous almost surely.