This paper considers the average optimality for a continuous-time Markov decision process in Borel state and action spaces, and with an arbitrarily unbounded nonnegative cost rate. The existence of a deterministic stationary optimal policy is proved under the conditions that allow the following; the controlled process can be explosive, the transition rates are weakly continuous, and the multifunction defining the admissible action spaces can be neither compact-valued nor upper semicontinuous.

We consider a random walks system on ℤ in which each active particle performs a nearest-neighbor random walk and activates all inactive particles it encounters. The movement of an active particle stops when it reaches a certain number of jumps without activating any particle. We prove that if the process relies on efficient particles (i.e. those particles with a small probability of jumping to the left) being placed strategically on ℤ, then it might survive, having active particles at any time with positive probability. On the other hand, we may construct a process that dies out eventually almost surely, even if it relies on efficient particles. That is, we discuss what happens if particles are initially placed very far away from each other or if their probability of jumping to the right tends to 1 but not fast enough.

Reaction networks are commonly used within the mathematical biology and mathematical chemistry communities to model the dynamics of interacting species. These models differ from the typical graphs found in random graph theory since their vertices are constructed from elementary building blocks, i.e. the species. We consider these networks in an Erdös–Rényi framework and, under suitable assumptions, derive a threshold function for the network to have a deficiency of zero, which is a property of great interest in the reaction network community. Specifically, if the number of species is denoted by n and the edge probability by $p_n$ , then we prove that the probability of a random binary network being deficiency zero converges to 1 if $p_n\ll r(n)$ as $n \to \infty$ , and converges to 0 if $p_n \gg r(n)$ as $n \to \infty$ , where $r(n)=\frac{1}{n^3}$ .

Let X = {Xt: t ≥ 0} be a stationary piecewise continuous Rd-valued process that moves between jumps along the integral curves of a given continuous vector field, and let S ⊂ Rd be a smooth surface. The aim of this paper is to derive a multivariate version of Rice's formula, relating the intensity of the point process of (localized) continuous crossings of S by X to the distribution of X0. Our result is illustrated by examples relating to queueing networks and stress release network models.

We consider a sequential rule, where an item is chosen into the group, such as a university faculty member, only if his/her score is better than the average score of those already belonging to the group. We study four variables: the average score of the members of the group after k items have been selected, the time it takes (in terms of the number of observed items) to assemble a group of k items, the average score of the group after n items have been observed, and the number of items kept after the first n items have been observed. We develop the relationships between these variables, and obtain their asymptotic behavior as k (respectively, n) tends to ∞. The assumption throughout is that the items are independent and identically distributed with a continuous distribution. Though knowledge of this distribution is not needed to implement the selection rule, the asymptotic behavior does depend on the distribution. We study in some detail the exponential, Pareto, and beta distributions. Generalizations of the ‘better than average’ rule to the β better than average rules are also considered. These are rules where an item is admitted to the group only if its score is better than β times the present average of the group, where β > 0.

For a supercritical branching process x = {xn; n ≧ 0, x0 = 1} with random environments, define when xn > 0; and = 1 when xn = 0. When x is assumed to satisfy the standard regularity assumptions, under the non-extinction hypothesis, is a strongly consistent and asymptotically unbiased estimator for the criticality parameter π and is asymptotically normal. A strongly consistent estimator, is also proposed for the associated variance, σ2.

We consider general queueing models dealing with multiple classes of customers and address the question under what conditions and in what (stochastic) sense the marginal increase in various performance measures, resulting from the addition of a new class of customers to an existing system, is larger than if the same class were added to a system dealing with only a subset of its current customer base.

Our results enhance our understanding of the dependence of various performance measures with respect to the composition of the customer base. In addition they translate readily into convexity results in an (appropriately defined) arrival rate.

Sharp upper and lower reliability bounds for the ℒ- and -classes of life distributions with a given mean are provided. Bounds are also given for a distribution which is related to a known distribution (not necessarily exponential) through the Laplace order.

This paper discusses general bounds for coverage probabilities and moments of stopping rules for sequential coverage problems in geometrical probability. An approach to the study of the asymptotic behaviour of these moments is also presented.

We analyse a class of diffusion models that (i) allows an explicit expression for the likelihood function of discrete time observation, (ii) allows the possibility of heavy-tailed observations, and (iii) allows an analysis of the tails of the increments. The class simply consists of transformed Ornstein–Uhlenbeck processes and is of relevance for heavy-tailed time series. We also treat the question of the existence of an equivalent martingale measure for the class of models considered.

In the present paper we study three aspects in the theory of non-homogeneous Markov systems under the continuous-time formulation. Firstly, the relationship between stability and quasi-stationarity is investigated and conditions are provided for a quasi-stationary structure to be stable. Secondly, the concept of asymptotic attainability is studied and the possible regions of asymptotically attainable structures are determined. Finally, the cyclic case is considered, where it is shown that for a system in a periodic environment, the relative structure converges to a periodic vector, independently of the initial distribution. Two numerical examples illustrate the above theoretical results.

This paper deals with a class of discrete-time Markov chains for which the invariant measures can be expressed in terms of generalized continued fractions. The representation covers a wide class of stochastic models and is well suited for numerical applications. The results obtained can easily be extended to continuous-time Markov chains.

The survival distribution of a device subject to a sequence of shocks occurring randomly over time is studied by Esary, Marshall and Proschan (1973) and by A-Hameed and Proschan (1973), (1975). The present note treats the case in which shocks occur according to a homogeneous Poisson cluster process. It is shown that if [the device survives k shocks] = zk, 0 < z < 1, then the device exhibits a decreasing failure rate. A DFR preservation theorem is proved for completely monotonic . A counterexample to the IFR preservation theorem is given in which is strictly IFR while the failure rate is initially decreasing and then increasing.

A supercritical position-dependent Markov branching process has been used as an approximation to a model describing the initial geographical spread of a measles epidemic (Bartlett (1956)). Let α be its Malthusian parameter, ß its velocity of propagation, Z(A, t) the number of individuals in the set A at time t, and A√(ßt) = [√(ßt) r: r ∈ A]. The mean square convergence of the random variable W(A, t)= e–αtZ(A√(ßt), t) to a limit variable W(A) is established.

n applicants of similar qualification are on an interview list and their salary demands are from a known and continuous distribution. Two managers, I and II, will interview them one at a time. Right after each interview, manager I always has the first opportunity to decide to hire the applicant or not unless he has hired one already. If manager I decides not to hire the current applicant, then manager II can decide to hire the applicant or not unless he has hired one already. If both managers fail to hire the current applicant, they interview the next applicant, but both lose the chance of hiring the current applicant. If one of the managers does hire the current one, then they proceed with interviews until the other manager also hires an applicant. The interview process continues until both managers hire an applicant each. However, at the end of the process, each manager must have hired an applicant. In this paper, we first derive the optimal strategies for them so that the probability that the one he hired demands less salary than the one hired by the other does is maximized. Then we derive an algorithm for computing manager II's winning probability when both managers play optimally. Finally, we show that manager II's winning probability is strictly increasing in n, is always less than c, and converges to c as n →∞, where c = 0.3275624139 · ·· is a solution of the equation ln(2) + x ln(x) = x.

We present a solution to a special system of Kolmogorov forward equations. We use this result to present a useful expression for the transition probabilities of the extended simple stochastic epidemic model and an epidemic model involving cross-infection between two otherwise isolated groups.

A stochastic model for the competition between two species is considered and a limit theorem is obtained for the size of the surviving species. A limit theorem is also obtained for the probability that a given species should survive the other.

We propose in this paper two methods to compute Markovian bounds for monotone functions of a discrete time homogeneous Markov chain evolving in a totally ordered state space. The main interest of such methods is to propose algorithms to simplify analysis of transient characteristics such as the output process of a queue, or sojourn time in a subset of states. Construction of bounds are based on two kinds of results: well-known results on stochastic comparison between Markov chains with the same state space; and the fact that in some cases a function of Markov chain is again a homogeneous Markov chain but with smaller state space. Indeed, computation of bounds uses knowledge on the whole initial model. However, only part of this data is necessary at each step of the algorithms.

Some inequalities for moments and coefficients of variation of probability densities over the positive real line are obtained by means of simple geometrical relationships. As an illustrative application rigorous bounds are obtained for the ratio of weight average to number average molecular weight for a large class of distributions of macromolecules, giving a more precise characterization of this empirical measure of heterogeneity.