We consider a family of irreducible, ergodic and aperiodic Markov chains X(ε) = {X(ε)n, n ≧0} depending on a parameter ε > 0, so that the local drifts have a critical behaviour (in terms of Pakes' lemma). The purpose is to analyse the steady-state distributions of these chains (in the sense of weak convergence), when ε↓ 0. Under assumptions involving at most the existence of moments of order 2 + γ for the jumps, we show that, whenever X(0) is not ergodic, it is possible to characterize accurately these limit distributions. Connections with the gamma and uniform distributions are revealed. An application to the well-known ALOHA network is given.

In this part we extend and particularise results developed by the author in Part I (pp. 349–375) for a class of queueing systems which can be formulated as Markov renewal processes. We examine those models where the basic transition consists of only two types: ‘arrivals' and ‘departures'. The ‘arrival lobby' and ‘departure lobby' queue-length processes are shown, using the results of Part I to be Markov renewal. Whereas the initial study focused attention on the behaviour of the embedded discrete-time Markov chains, in this paper we examine, in detail, the embedded continuous-time semi-Markov processes. The limiting distributions of the queue-length processes in both continuous and discrete time are derived and interrelationships between them are examined in the case of continuous-time birth–death queues including the M/M/1/M and M/M/1 variants. Results for discrete-time birth–death queues are also derived.

The paper is a continuation of [7]. One of the main results is as follows: if the sequence (w, v, u) is asymptotically stationary in some sense then (l, w, v, u) is asymptotically stationary in the same sense. The other main result deals with an asymptotic behaviour of the vector of the queue size and the waiting time in the heavy-traffic situation. This result resembles a formula of the Little type.

Suppose X is a multidimensional diffusion process. Assume that at time zero the state of X is fully observed, but at time $T>0$ only linear combinations of its components are observed. That is, one only observes the vector $L X_T$ for a given matrix L. In this paper we show how samples from the conditioned process can be generated. The main contribution of this paper is to prove that guided proposals, introduced in [35], can be used in a unified way for both uniformly elliptic and hypo-elliptic diffusions, even when L is not the identity matrix. This is illustrated by excellent performance in two challenging cases: a partially observed twice-integrated diffusion with multiple wells and the partially observed FitzHugh–Nagumo model.

In this work we investigate under what circumstances the equilibrium distribution of the numbers of groups of various sizes in a certain M/G/k group-arrival group-departure loss system can be obtained in a closed product form.

We introduce two stochastic chemostat models consisting of a coupled population-nutrient process reflecting the interaction between the nutrient and the bacteria in the chemostat with finite volume. The nutrient concentration evolves continuously but depends on the population size, while the population size is a birth-and-death process with coefficients depending on time through the nutrient concentration. The nutrient is shared by the bacteria and creates a regulation of the bacterial population size. The latter and the fluctuations due to the random births and deaths of individuals make the population go almost surely to extinction. Therefore, we are interested in the long-time behavior of the bacterial population conditioned to nonextinction. We prove the global existence of the process and its almost-sure extinction. The existence of quasistationary distributions is obtained based on a general fixed-point argument. Moreover, we prove the absolute continuity of the nutrient distribution when conditioned to a fixed number of individuals and the smoothness of the corresponding densities.

This paper considers Markov decision processes (MDPs) with unbounded rates, as a function of state. We are especially interested in studying structural properties of optimal policies and the value function. A common method to derive such properties is by value iteration applied to the uniformised MDP. However, due to the unboundedness of the rates, uniformisation is not possible, and so value iteration cannot be applied in the way we need. To circumvent this, one can perturb the MDP. Then we need two results for the perturbed sequence of MDPs: 1. there exists a unique solution to the discounted cost optimality equation for each perturbation as well as for the original MDP; 2. if the perturbed sequence of MDPs converges in a suitable manner then the associated optimal policies and the value function should converge as well. We can model both the MDP and perturbed MDPs as a collection of parametrised Markov processes. Then both of the results above are essentially implied by certain continuity properties of the process as a function of the parameter. In this paper we deduce tight verifiable conditions that imply the necessary continuity properties. The most important of these conditions are drift conditions that are strongly related to nonexplosiveness.

In the first part of this paper we study approximations of trajectories of piecewise deterministic processes (PDPs) when the flow is not given explicitly by the thinning method. We also establish a strong error estimate for PDPs as well as a weak error expansion for piecewise deterministic Markov processes (PDMPs). These estimates are the building blocks of the multilevel Monte Carlo (MLMC) method, which we study in the second part. The coupling required by the MLMC is based on the thinning procedure. In the third part we apply these results to a two-dimensional Morris–Lecar model with stochastic ion channels. In the range of our simulations the MLMC estimator outperforms classical Monte Carlo.

We study stationary processes given as solutions to stochastic differential equations driven by fractional Brownian motion. This rich class includes the fractional Ornstein-Uhlenbeck process and those processes that can be obtained from it by state space transformations. An explicit formula in terms of Euler's Γ-function describes the asymptotic behaviour of the covariance function of the fractional Ornstein-Uhlenbeck process near zero, which, by an application of Berman's condition, guarantees that this process is in the maximum domain of attraction of the Gumbel distribution. Necessary and sufficient conditions on the state space transforms are stated to classify the maximum domain of attraction of solutions to stochastic differential equations driven by fractional Brownian motion.

Consider a circle of circumference 1. Throw n points at random onto this circle and append to each of these points a clockwise arc of length s. The resulting random set is a union of a random number of connected components, each with specific size. Using tools designed by Steutel, we compute the joint distribution of the lengths of the connected components. Asymptotic results are presented when n goes to ∞ and s to 0 jointly according to different regimes.

In two earlier papers [6], [7] the properties of bivariate renewal processes and their associated two-dimensional renewal functions, H(x, y) were examined. By utilising the Fréchet bounds for joint distributions and the properties of univariate renewal processes, a collection of upper and lower bounds for H(x, y) are constructed. The evaluation of these bounds is carried out for the case of the family of bivariate Poisson processes. An interesting by-product of this investigation leads to a new inequality for the median of a Poisson random variable.

Results about stationary Poisson hyperplane processes and the induced hyperplane mosaics are extended to the case where, instead of stationarity, it is only assumed that the intensity measure has a (possibly continuous) density with respect to some translation-invariant measure. Intensities and quermass densities, which are constant in the stationary case, are then replaced by functions. In a similar way, the associated zonoid (Matheron's Steiner convex set) is generalized.

Discrete time-series models are commonly used to represent economic and physical data. In decision making and system control, the first-passage time and level-crossing probabilities of these processes against certain threshold levels are important quantities. In this paper, we apply an integral-equation approach together with the state-space representations of time-series models to evaluate level-crossing probabilities for the AR(p) and ARMA(1,1) models and the mean first passage time for AR(p) processes. We also extend Novikov's martingale approach to ARMA(p,q) processes. Numerical schemes are used to solve the integral equations for specific examples.

Cycle-time distribution is shown to take the form of a linear combination of M Erlang-N density functions in a cyclic queueing network of M servers and N customers. For paths of m servers in tree-like networks, the components in the more complex linear combination are convolutions of Erlang-N with at most m − 1 negative exponentials.

In this paper, we consider several stochastic models arising from environmental problems. First, we study pollution in a domain where undesired chemicals are deposited at random times and locations according to Poisson streams. The chemical concentration can be modeled by a linear stochastic partial differential equation (SPDE) which is solved by applying a general result. Various properties, especially the limit behavior of the pollution process, are discussed. Secondly, we consider the pollution problem when a tolerance level is imposed. The chemical concentration can still be modeled by a SPDE which is no longer linear. Its properties are investigated in this paper. When the leakage rate is positive, it is shown that the pollution process has an equilibrium state given by the deterministic model treated in [2]. Finally, the linear filtering problem is considered based on the data of several observation stations.

In this paper we examine the extremal tail probabilities of moving sums in a marked Poisson random field. These sums are computed by adding up the weighted occurrences of events lying within a scanning set of fixed shape and size. We also provide an alternative representation of the constants of the asymptotic formulae in terms of the occupation measure of the conditional local random field at zero, and extend these representations to the constants of asymptotic tail probabilities of Gaussian random fields.

By using properties of up- and downcrossings of the sample functions of the workload process and of the attained waiting-time process for a G/G/1 queueing model, a direct proof of a theorem proved by Sakasegawa and Wolff is given.

Corrected random walk approximations to continuous-time optimal stopping boundaries for Brownian motion, first introduced by Chernoff and Petkau, have provided powerful computational tools in option pricing and sequential analysis. This paper develops the theory of these second-order approximations and describes some new applications.

We consider the stochastic sequence {Yt}t∈ℕ defined recursively by the linear relation Yt+1=AtYt+Bt in a random environment. The environment is described by the stochastic process {(At,Bt)}t∈ℕ and is under the simultaneous control of several agents playing a discounted stochastic game. We formulate sufficient conditions on the game which ensure the existence of Nash equilibria in Markov strategies which have the additional property that, in equilibrium, the process {Yt}t∈ℕ converges in distribution to a stationary regime.

An invariant Φ of a tree T under a k-state Markov model, where the time parameter is identified with the edges of T, allows us to recognize whether data on N observed species can be associated with the N terminal vertices of T in the sense of having been generated on T rather than on any other tree with N terminals. The invariance is with respect to the (time) lengths associated with the edges of the tree. We propose a general method of finding invariants of a parametrized functional form. It involves calculating the probability f of all kN data possibilities for each of m edge-length configurations of T, then solving for the parameters using the m equations of form Φ (f) = 0. We apply this to the case of quadratic invariants for unrooted binary trees with four terminals, for all k, using the Jukes–Cantor type of Markov matrix. We report partial results on finding the smallest algebraically independent set of invariants.