We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Dynamic programming methods for matrix permutation problems in combinatorial data analysis can produce globally-optimal solutions for matrices up to size 30×30, but are computationally infeasible for larger matrices because of enormous computer memory requirements. Branch-and-bound methods also guarantee globally-optimal solutions, but computation time considerations generally limit their applicability to matrix sizes no greater than 35×35. Accordingly, a variety of heuristic methods have been proposed for larger matrices, including iterative quadratic assignment, tabu search, simulated annealing, and variable neighborhood search. Although these heuristics can produce exceptional results, they are prone to converge to local optima where the permutation is difficult to dislodge via traditional neighborhood moves (e.g., pairwise interchanges, object-block relocations, object-block reversals, etc.). We show that a heuristic implementation of dynamic programming yields an efficient procedure for escaping local optima. Specifically, we propose applying dynamic programming to reasonably-sized subsequences of consecutive objects in the locally-optimal permutation, identified by simulated annealing, to further improve the value of the objective function. Experimental results are provided for three classic matrix permutation problems in the combinatorial data analysis literature: (a) maximizing a dominance index for an asymmetric proximity matrix; (b) least-squares unidimensional scaling of a symmetric dissimilarity matrix; and (c) approximating an anti-Robinson structure for a symmetric dissimilarity matrix.
Common outputs of software programs for network estimation include association matrices containing the edge weights between pairs of symptoms and a plot of the symptom network. Although such outputs are useful, it is sometimes difficult to ascertain structural relationships among symptoms from these types of output alone. We propose that matrix permutation provides a simple, yet effective, approach for clarifying the order relationships among the symptoms based on the edge weights of the network. For directed symptom networks, we use a permutation criterion that has classic applications in electrical circuit theory and economics. This criterion can be used to place symptoms that strongly predict other symptoms at the beginning of the ordering, and symptoms that are strongly predicted by other symptoms at the end. For undirected symptom networks, we recommend a permutation criterion that is based on location theory in the field of operations research. When using this criterion, symptoms with many strong ties tend to be placed centrally in the ordering, whereas weakly-tied symptoms are placed at the ends. The permutation optimization problems are solved using dynamic programming. We also make use of branch-search algorithms for extracting maximum cardinality subsets of symptoms that have perfect structure with respect to a selected criterion. Software for implementing the dynamic programming algorithms is available in MATLAB and R. Two networks from the literature are used to demonstrate the matrix permutation algorithms.
There are two well-known methods for obtaining a guaranteed globally optimal solution to the problem of least-squares unidimensional scaling of a symmetric dissimilarity matrix: (a) dynamic programming, and (b) branch-and-bound. Dynamic programming is generally more efficient than branch-and-bound, but the former is limited to matrices with approximately 26 or fewer objects because of computer memory limitations. We present some new branch-and-bound procedures that improve computational efficiency, and enable guaranteed globally optimal solutions to be obtained for matrices with up to 35 objects. Experimental tests were conducted to compare the relative performances of the new procedures, a previously published branch-and-bound algorithm, and a dynamic programming solution strategy. These experiments, which included both synthetic and empirical dissimilarity matrices, yielded the following findings: (a) the new branch-and-bound procedures were often drastically more efficient than the previously published branch-and-bound algorithm, (b) when computationally feasible, the dynamic programming approach was more efficient than each of the branch-and-bound procedures, and (c) the new branch-and-bound procedures require minimal computer memory and can provide optimal solutions for matrices that are too large for dynamic programming implementation.
This paper proposes an order-constrained K-means cluster analysis strategy, and implements that strategy through an auxiliary quadratic assignment optimization heuristic that identifies an initial object order. A subsequent dynamic programming recursion is applied to optimally subdivide the object set subject to the order constraint. We show that although the usual K-means sum-of-squared-error criterion is not guaranteed to be minimal, a true underlying cluster structure may be more accurately recovered. Also, substantive interpretability seems generally improved when constrained solutions are considered. We illustrate the procedure with several data sets from the literature.
The book starts out with a motivating chapter to answer the question: Why is it worthwhile to develop system theory? To do so, we jump fearlessly in the very center of our methods, using a simple and straight example in optimization: optimal tracking. Although optimization is not our leading subject– which is system theory– it provides for one of the main application areas, namely the optimization of the performance of a dynamical system in a time-variant environment (for example, driving a car or sending a rocket to the moon). The chapter presents a recursive matrix algebra approach to the optimization problem, known as dynamic programming. Optimal tracking is based on a powerful principle called “dynamic programming,” which lies at the very basis of what ”dynamical” means.
Deals with reservoir operation, including rule curves, methods of mathematical programming, optimization of reservoir operations, simulation models, reservoir operation modeling with HEC-ResSim, mass curves, and reservoir siltation.
This chapter introduces the medium-term hydrothermal planning problem. Two-stage stochastic linear programs are introduced first, and subsequently generalized to multi-stage stochastic linear programs. Various representations of multi-stage stochastic linear programs are presented, including representations on scenario trees, representations on lattices (for Markov processes), as well as representations with stagewise independent uncertainty. These models are applied to a running example of hydrothermal planning, which is used as the basis for introducing the notion of dynamic programming value functions. The value of water is defined, and demonstrated on a hydrothermal planning problem. The chapter then proceeds to focus on the performance of stochastic programs. The wait-and-see and here-and-now value are introduced for two-stage stochastic programs. These are used to define the expected value of perfect information. The expected value solution is also defined, and used in order to introduce the value of the stochastic solution. Sampling is discussed briefly in the context of sample average approximation and importance sampling.
The rapid development of new technologies such as electrification, autonomy, and other contextual factors pose significant challenges to development teams in balancing competing aspects while developing value-robust solutions. One approach for achieving value robustness is designing for changeability. This paper presents a tradespace exploration from a Systems-of-Systems perspective to facilitate changeability assessment during early design stages. The approach is further demonstrated on a fleet of haulers operating in a mining site.
This study examines the economic performance of rainfed cropping systems endemic to the Southern Great Plains under weed competition. Cropping systems include tilled and no-till wheat-fallow, wheat-soybean, and wheat-sorghum rotations. Net returns from systems are compared under different levels of weed pressure. Producers operating over longer planning horizons would choose to double-crop regardless of the tillage method used and weed pressure level. Producers operating under shorter planning horizons would implement wheat-fallow systems when weed pressure is high and double crop when weed pressure is low.
We study the time-consistent investment and contribution policies in a defined benefit stochastic pension fund where the manager discounts the instantaneous utility over a finite planning horizon and the final function at constant but different instantaneous rates of time preference. This difference, which can be motivated for some uncertainties affecting payoffs at the end of the planning horizon, will induce a variable bias between the relative valuation of the final function and the previous payoffs and will lead the manager to show time-inconsistent preferences. Both the benefits and the contribution rate are proportional to the total wage of the workers that we suppose is stochastic. The aim is to maximize a CRRA utility function of the net benefit relative to salary in a bounded horizon and to maximize a CRRA final utility of the fund level relative to the salary. The problem is solved by means of dynamic programming techniques, and main results are illustrated numerically.
The paper presents a framework for the integration of the system's design variables, state variables, control strategies, and contextual variables into a design optimization problem to assist early-stage design decisions. The framework is based on a global optimizer incorporating Dynamic Programming, and its applicability is demonstrated by the conceptual design of an electrical hauler. Pareto front of optimal design solutions, in terms of time and cost, together with optimal velocity profiles and battery state-of-charge is visualized for the given mining scenario.
Chapter 5 presents the subject matter of artificial intelligence, focusing on machine learning, where these machines are artificial agents. It presents simple examples of unsupervised learning, supervised learning, and reinforcement learning, and introduces notions of dynamic programming, Q-learning, and stochastic control. After that, it explores some links that can be established between artificial intelligence and the philosophy of mind, presenting and discussing the Turing test, the philosophical approaches of eliminativism and functionalism, and the problem of tacit knowledge.
In this chapter we present a self-contained chapter on dynamic programming in continuous time in the framework of jump diffusions driven by a marked point process. We derive the relevant HJB equation and we study some examples of standard control as well as intensity control.
This chapter is the start of a new part of the book, devoted to optimal control. The chapter consists of a self-contained course on dynamic programming in discrete time.
We consider a gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We show, under natural conditions on the system primitives, the existence of a deterministic stationary optimal policy out of a more general class of policies that allow multiple simultaneous impulses, randomized selection of impulses with random effects, and accumulation of jumps. After characterizing the value function using the optimality equation, we reduce the gradual-impulse control problem to an equivalent simple discrete-time Markov decision process, whose action space is the union of the sets of gradual and impulsive actions.
Bounded treewidth is one of the most cited combinatorial invariants in the literature. It was also applied for solving several counting problems efficiently. A canonical counting problem is #Sat, which asks to count the satisfying assignments of a Boolean formula. Recent work shows that benchmarking instances for #Sat often have reasonably small treewidth. This paper deals with counting problems for instances of small treewidth. We introduce a general framework to solve counting questions based on state-of-the-art database management systems (DBMSs). Our framework takes explicitly advantage of small treewidth by solving instances using dynamic programming (DP) on tree decompositions (TD). Therefore, we implement the concept of DP into a DBMS (PostgreSQL), since DP algorithms are already often given in terms of table manipulations in theory. This allows for elegant specifications of DP algorithms and the use of SQL to manipulate records and tables, which gives us a natural approach to bring DP algorithms into practice. To the best of our knowledge, we present the first approach to employ a DBMS for algorithms on TDs. A key advantage of our approach is that DBMSs naturally allow for dealing with huge tables with a limited amount of main memory (RAM).
Emergency search and rescue on the sea is an important part of national emergency response for marine perils. Optimal route planning for maritime search and rescue is the key capability to reduce the searching time, improve the rescue efficiency, as well as guarantee the rescue target’s safety of life and property. The main scope of the searching route planning is to optimise the searching time and voyage within the constraints of missing search rate and duplicate search rate. This paper proposes an optimal algorithm for searching routes of large amphibious aircraft corresponding to its flight characteristics and sea rescue capability. This algorithm transforms the search route planning problem into a discrete programming problem and applies the route traceback indexes to satisfy the duplicate search rate and missing search rate.
There is no silver bullet: no model can fit all data. Hence, special data requires special algorithms. In this chapter, we deal with two types of special data: sparse data and sequences that can be aligned to each other. We will not dive deep into sparsity learning, which is very complex. Rather, we introduce key concepts: sparsity inducing loss functions, dictionary learning, and what exactly the word sparsity means. For the second part in this chapter, we introduce dynamic time warping (DTW), which deals with sequences that can be aligned with each other (but there are sequences that cannot be aligned, which we will discuss in the next chapter). We use our old tricks: ideas, visualizations, formalizations, to reach the DTW solution. The key idea behind its success is divide-and-conquer and the key technology is dynamic programming.
Defined contribution (DC) pension plans have been gaining ground in the last 10–20 years as the preferred system for many countries and other agencies, both private and public. The central question for a DC plan is how to invest in order to reach the participant's retirement goals. Given the financial illiteracy of the general population, it is common to offer a default policy for members who do not actively make investment choices. Using data from the Chilean system, we discuss an investment model with fixed contribution rates and compare the results with the existing default policy under multiple objectives. Our results indicate that the Chilean default policy has good overall performance, but specific closed-loop policies have a higher probability of achieving desired retirement goals and can reduce the expected shortfall at retirement.
This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics of the individuals in a small population. The population and its individual characteristics can be represented by a point measure. We first define a PDMP on a space of locally finite measures. Then we define a sequence of random horizon optimal stopping problems for such processes. We prove that the value function of the problems can be obtained by iterating some dynamic programming operator. Finally we prove via a simple counter-example that controlling the whole population is not equivalent to controlling a random lineage.