Skip to main content Accessibility help
×
  1. Home
  2. Browse subjects
  3. Computer Science
  4. Communications, Information Theory and Security
  5. Content listing

Refine search

Refine search

Access:
Content type:
Publication date:
Apply   From and To filters
Subject: Show more
Journals Show more
Publishers: Show more
Societies Show more
Series: Show more
Collections: Show more

Actions for selected content:

Select all | Deselect all
  • View selected items
  • Export citations
  • Download PDF (zip)
  • Save to Kindle
  • Save to Dropbox
  • Save to Google Drive

Save Search

You can save your searches here and later view and run them again in "My saved searches".

Please provide a title, maximum of 40 characters.
Cancel
×

1397 results in Communications, Information Theory and Security

Page 27 of 70

  • 13 - Processing Large Data Sets in MapReduce

  • from Part III - Big Data Applications
  • Zhu Han, University of Houston, Mingyi Hong, Iowa State University, Dan Wang, Hong Kong Polytechnic University
  • Book:
    Signal Processing and Networking for Big Data Applications
    Published online:
    18 May 2017
    Print publication:
    27 April 2017, pp 283-300

  • Summary

    Introduction

    While the performance of a data parallelism structure is primarily determined by the amount of data that need to be processed, since data are processed by different machines in parallel, the completion time of an application job is determined by the last finished machine. It is a grand challenge to distribute data load evenly to different machines. In the default configuration ofMapReduce Hadoop, this is done by developing a hash function in the shuffling phase so that the intermediate results outputted by the mapper phase will be distributed evenly to different machines in the reduce phase. Such default configuration emphasizes the “common” case. Clearly, it is not optimal for various scenarios, and in many situations such configuration performs poorly.

    There are many research studies on this grand problem. There are the reactive approach and the proactive approach. In the reactive approach, the workloads of different machines are monitored and workloads can be migrated from one machine to another in case there is a large skew of the workloads. SkewTune is one good example representing reactive approach [22]. In the proactive approach, systems try to understand the possible workloads the data to be processed can generate on each machine, so that when the data are dispatched to different machines in a load aware manner.

    In this chapter, we study a proactive approach to improve performance of processing large data in MapReduce using sublinear algorithms.

    The Data Skew Problem of MapReduce Jobs

    When running MapReduce jobs, most of the studies have assumed that the input data are of uniform distribution, which, often being hashed to reduce worker nodes, naturally leads to a desirable balanced load in the later stages. The real-world data, however, are not necessarily uniform, and often exhibit a remarkable degree of skewness. For example, in PageRank, the graph commonly includes nodes with much higher degrees of incoming edges than others [21], and in Inverted Index, certain content can appear in many more documents than others [22]. Such a skewed distribution of the input or intermediate data can result in a small number of mappers or reducers taking a significantly longer time to complete than others [19]. Recent experimental studies [21, 19, 22] have shown that, in the CloudBurst application with a biology data set of a bimodal distribution [635], the slowest map task takes five times as long to complete as the fastest.

  • 7 - Deep Learning and Applications

  • from Part II - Methodology and Mathematical Background
  • Zhu Han, University of Houston, Mingyi Hong, Iowa State University, Dan Wang, Hong Kong Polytechnic University
  • Book:
    Signal Processing and Networking for Big Data Applications
    Published online:
    18 May 2017
    Print publication:
    27 April 2017, pp 126-168

  • Summary

    Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]:

    1. • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised).

    2. • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation.

    3. • They are part of the broader machine learning field of learning representations of data.

    4. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts.

    These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks.

    In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

  • 9 - Distributed Large-Scale Optimization

  • from Part III - Big Data Applications
  • Zhu Han, University of Houston, Mingyi Hong, Iowa State University, Dan Wang, Hong Kong Polytechnic University
  • Book:
    Signal Processing and Networking for Big Data Applications
    Published online:
    18 May 2017
    Print publication:
    27 April 2017, pp 196-224

  • Summary

    In this chapter, we discuss distributed algorithms for solving large-scale optimization problems over networks. In particular, the optimization problem is defined over a network of nodes/agents where each of them may have a piece of data, or may be in charge of updating a subset of variables, and they are required to collaboratively perform the desired optimization. The key question to be addressed is, under various problem settings, what protocols should the agents follow so as to efficiently optimize the overall problem. These types of distributed optimizing algorithms have recently become increasingly important in various big data problems, with applications ranging from distributed machine learning, signal processing, networking, multi-agent systems, parallel optimization, etc. There are a few driving forces behind these new trends, as we briefly list below:

    1. 1. The advances in sensor technologies makes distributed data acquisition ubiquitous.

    2. 2. The increasing size of optimization problems in big data era calls for distributed data storage.

    3. 3. Privacy issues in big data problem prevents direct data exchange among different (possibly geographically distributed) data sources.

    4. 4. Using multi-core high-performance computing systems (with either shared or distributed memory) to solve big data optimization problems can significantly reduce the solution time.

    In this chapter, we present a few recent algorithms for solving such distributed optimization problems. We will cover a wide range of algorithms and discuss their convergence behavior, as well as their performance in terms of scaling with network/problem sizes.

    The chapter is organized as follows. In Sections 9.1–9.4, we present a general problem formulation and a few popular first-order methods. In Section 9.5, we discuss a special cases of the general formulation (i.e., the global consensus problem). In Section 9.6, we compare various algorithms presented in this chapter.

    Background

    Problem Formulation

    The Optimization Problem

    Consider a network with N distributed agents, each capable of communicating with its immediate neighbors; see Figure 9.1 for an illustration.

  • 4 - Sparse Optimization

  • from Part II - Methodology and Mathematical Background
  • Zhu Han, University of Houston, Mingyi Hong, Iowa State University, Dan Wang, Hong Kong Polytechnic University
  • Book:
    Signal Processing and Networking for Big Data Applications
    Published online:
    18 May 2017
    Print publication:
    27 April 2017, pp 58-91

  • Summary

    This chapter reviews a collection of sparse optimization models and algorithms. The review focuses on introducing these algorithms along with their motivations and basic properties. The mathematical tools from the previous section will be heavily replied upon.

    This chapter is organized as follows. Section 4.1 reviews a list of (convex) sparse optimization models, which deal with different types of signals and different kinds of noise; they can also include additional features as objectives and constraints that arise in practical problems. Section 4.2 demonstrates that the convex sparse optimization problems can be transformed in equivalent cone programs and solved by off-the-shelf algorithms; yet it argues that these algorithms are usually inefficient for large-scale problems. Sections 4.3–4.12 cover a large variety of algorithms for sparse optimization. These algorithms have different strengths and fit different applications. The efficiency of these algorithms, for the most part, comes from the use of the often closed-form shrinkage-like operations, which is introduced in Section 4.4. Then, Section 4.5 presents a prox-linear framework and gives several algorithms under this framework that are based on gradient descent and take advantages of shrinkage-like operations. These algorithms can often be linked with the BSUM framework discussed in Chapter 3. Duality is a very powerful tool in modern convex optimization; sparse optimization is not an exception. Section 4.6 derives a few dual models and algorithms for sparse optimization and discusses their properties. One class of dual algorithms (i.e., the ADMM algorithm introduced in Chapter 3) is very efficient and extremely versatile, applicable to nearly all convex sparse optimization problems. The framework and applications of these algorithms are given in Section 4.8. Unlike other algorithms, the homotopy algorithms in Section 4.9 produces not just one solution but a path of solutions for the LASSO model (which is given in (4.2b) below); not only are they efficient at producing multiple solutions corresponding to different parameter values, they are especially fast if the solution is extremely sparse. All the above algorithms can be (often significantly) accelerated by appropriately setting their stepsize parameters that determine the amount of progress at each iteration. Such techniques are reviewed in Section 4.10. Unlike all previous algorithms, greedy algorithms do not necessarily correspond to an optimization model; instead of systematically searching for solutions that minimize objective functions, they build sparse solutions by constructing their supports step by step in a greedy manner.

  • 10 - Advanced MIMO Detection Techniques

  • Rakhesh Singh Kshetrimayum, Indian Institute of Technology, Guwahati
  • Book:
    Fundamentals of MIMO Wireless Communications
    Published online:
    23 July 2017
    Print publication:
    17 April 2017, pp 213-244

  • Summary

    Introduction to spatially multiplexed MIMO systems

    We have observed in chapter 2 that the capacity of wireless communication links is increased by using multiple antennas at the transmitter and the receiver. To achieve these capacities, a transmission scheme, called Diagonal-Bell Laboratories Layered Space-time (D-BLAST) has been proposed by G. J. Foschini (1996). In an i.i.d. Rayleigh scattering environment, this processing structure leads to theoretical rates which grow linearly with the number of antennas (for NR = NT ) with these rates approaching 90% of Shannon's capacity. But this has large computational complexity required for implementation of this scheme.

    A simplified version, called Vertical BLAST (V-BLAST) has been proposed by G. J. Foschini et al., (1999). They have demonstrated spectral efficiencies of 20–40 bps/Hz at average signal-to-noise ratio (SNR) ranging from 24 to 34 dB could be achieved in indoor environments. The essential difference between D-BLAST and V-BLAST lies in the vector encoding process. The D-BLAST code blocks are organized along diagonals in space-time. In V-BLAST, however, the vector encoding process is simply a demultiplexing (DMUX) operation followed by independent bit-to-symbol mapping of each sub-stream. No inter sub-stream coding, or coding of any kind, is required.

    Note that BLAST detection scheme can be done in one of the following ways:

  • 1. interference nulling to reduce the effect of the other (interfering) signals on the desired one is employed in successive interference cancellation (SIC) which will be discussed in section 10.4;

  • 2. ordering to select the sub-stream with the largest signal-to-noise ratio (SNR) or other criteria along with SIC, which will be discussed later in ordered successive interference cancellation (OSIC) (section 10.5).

    • Vertical/horizontal layered space-time transmission

    Vertical Bell Laboratories Layered Space-time Transmission (V-BLAST) suggests simultaneous transmission of independent uncoded data sub-streams. The input data is separated into several substreams with demultiplexing (DMUX) operation which are associated with each transmit antenna.

    This process is followed by independent bit-to-symbol mapping (M-ary) of each sub-stream and may be interleaved (optional) as depicted in Fig. 10.1 (a). We consider a V-BLAST system with NT transmitting antennas. At the transmitter the data is passed through a serial-to-parallel converter (S/P converter) and transformed into NT sub-streams, where each sub-stream is sent through a different transmit antenna.

  • 4 - Power Allocation in MIMO Systems

  • Rakhesh Singh Kshetrimayum, Indian Institute of Technology, Guwahati
  • Book:
    Fundamentals of MIMO Wireless Communications
    Published online:
    23 July 2017
    Print publication:
    17 April 2017, pp 70-79

  • Summary

    Introduction

    Since using SVD, we can decompose a MIMO channel into RH parallel Gaussian channels, where RH is the rank of the MIMO channel matrix, we will use the knowledge on the capacity of the parallel Gaussian channel (see Appendix C) to find the capacity of a MIMO channel for uniform and adaptive power allocation scheme. Uniform power allocation is employed when the channel state information (CSI) is available at the receiver but not at the transmitter (open loop MIMO system). We can use adaptive power allocation based on Water-filling algorithm when CSI is available at the receiver as well as the transmitter (closed loop MIMO system). We will also discuss near optimal power allocation for high and low SNR cases.

    Note that power allocation plays a significant role in deciding MIMO capacity. Power allocation was not an important issue in SISO since only single antenna was employed at the transmitter and receiver. Usually we allocate all the power to the single transmit antenna. But for MIMO it is one of the most important factors for increasing capacity. We have numerous antennas at the transmitter and receiver for MIMO case. The fundamental question is how much power we allocate to each transmit antennas. Hence if we allot power equally to all transmit antennas or unequally to each transmit antenna, capacity of the MIMO channel will be definitely different. If this is the case, then how we optimally allocate power to MIMO channels can be considered as an optimization problem to maximize capacity. To allocate power adaptively we need the CSI at the transmitter, also since power allocation is done at the transmitter. Intuitively we will allocate more power to better channels than the bad channels. We may not allocate any power at all to some of the worst channels. We will discuss these in detail in the following sections. In practical scenarios, we can allocate power near optimally for MIMO channels for two cases: high and low SNR regimes.

    Uniform power allocation

    The capacity indicates the best viable transmission data rate over the channel for miniscule probability of error. Shanon provided the expression of the achievable communication rate of a channel with noise (C. E. Shanon, 1948).

Page 27 of 70