Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-745bb68f8f-grxwn Total loading time: 0 Render date: 2025-01-11T07:10:16.207Z Has data issue: false hasContentIssue false

6 - Uncertainty Relations and Sparse Signal Recovery

Published online by Cambridge University Press:  22 March 2021

By
Erwin Riegler  and
Helmut Bökei
Edited by
Miguel R. D. Rodrigues  and
Yonina C. Eldar
Miguel R. D. Rodrigues
Affiliation:
University College London
Yonina C. Eldar
Affiliation:
Weizmann Institute of Science, Israel
Get access

Summary

This chapter provides an introduction to uncertainty relations underlying sparse signal recovery. We start with the seminal work by Donoho and Stark (1989), which defines uncertainty relations as upper bounds on the operator norm of the band-limitation operator followed by the time-limitation operator, generalize this theory to arbitrary pairs of operators, and then develop, out of this generalization, the coherence-based uncertainty relations due to Elad and Bruckstein (2002), plus uncertainty relations in terms of concentration of the 1-norm or 2-norm. The theory is completed with set-theoretic uncertainty relations which lead to best possible recovery thresholds in terms of a general measure of parsimony, the Minkowski dimension. We also elaborate on the remarkable connection between uncertainty relations and the “large sieve,” a family of inequalities developed in analytic number theory. We show how uncertainty relations allow one to establish fundamental limits of practical signal recovery problems such as inpainting, declipping, super-resolution, and denoising of signals corrupted by impulse noise or narrowband interference.

Keywords

compressed sensinguncertainty relationsparsitysignal representationsignal recoverysignal declippingsignal separationinpaintingsuper-resolution
Type
Chapter
Information
Information-Theoretic Methods in Data Science , pp. 163 - 196
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Heisenberg, W., The physical principles of the quantum theory. University of Chicago Press, 1930.Google Scholar
Faris, W. G., “Inequalities and uncertainty principles,” J. Math. Phys., vol. 19, no. 2, pp. 461–466, 1978.Google Scholar
Cowling, M. G. and Price, J. F., “Bandwidth versus time concentration: The Heisenberg–Pauli–Weyl inequality,” SIAM J. Math. Anal., vol. 15, no. 1, pp. 151–165, 1984.CrossRefGoogle Scholar
Benedetto, J. J., Wavelets: Mathematics and applications. CRC Press, 1994, ch. Frame decompositions, sampling, and uncertainty principle inequalities.Google Scholar
Folland, G. B. and Sitaram, A., “The uncertainty principle: A mathematical survey,” J. Fourier Analysis Applications, vol. 3, no. 3, pp. 207–238, 1997.Google Scholar
Donoho, D. L. and Stark, P. B., “Uncertainty principles and signal recovery,” SIAM J. Appl. Math., vol. 49, no. 3, pp. 906–931, 1989.Google Scholar
Donoho, D. L. and Logan, B. F., “Signal recovery and the large sieve,” SIAM J. Appl. Math., vol. 52, no. 2, pp. 577–591, 1992.Google Scholar
Elad, M. and Bruckstein, A. M., “A generalized uncertainty principle and sparse representation in pairs of bases,” IEEE Trans. Information Theory, vol. 48, no. 9, pp. 2558–2567, 2002.Google Scholar
Studer, C., Kuppinger, P., Pope, G., and Bölcskei, H., “Recovery of sparsely corrupted signals,” IEEE Trans. Information Theory, vol. 58, no. 5, pp. 3115–3130, 2012.Google Scholar
Kuppinger, P., Durisi, G., and Bölcskei, H., “Uncertainty relations and sparse signal recovery for pairs of general signal sets,” IEEE Trans. Information Theory, vol. 58, no. 1, pp. 263–277, 2012.CrossRefGoogle Scholar
Terras, A., Fourier analysis on finite groups and applications. Cambridge University Press, 1999.Google Scholar
Tao, T., “An uncertainty principle for cyclic groups of prime order,” Math. Res. Lett., vol. 12, no. 1, pp. 121–127, 2005.Google Scholar
Stotz, D., Riegler, E., Agustsson, E., and Bölcskei, H., “Almost lossless analog signal separation and probabilistic uncertainty relations,” IEEE Trans. Information Theory, vol. 63, no. 9, pp. 5445–5460, 2017.CrossRefGoogle Scholar
Foucart, S. and Rauhut, H., A mathematical introduction to compressive sensing. Birkhäuser, 2013.Google Scholar
Gröchenig, K., Foundations of time–frequency analysis. Birkhäuser, 2001.Google Scholar
Gabor, D., “Theory of communications,” J. Inst. Elec. Eng., vol. 96, pp. 429–457, 1946.Google Scholar
Bölcskei, H., Advances in Gabor analysis. Birkhäuser, 2003, ch. Orthogonal frequency division multiplexing based on offset QAM.Google Scholar
Fefferman, C., “The uncertainty principle,” Bull. Amer. Math. Soc., vol. 9, no. 2, pp. 129–206, 1983.Google Scholar
Evra, S., Kowalski, E., and Lubotzky, A., “Good cyclic codes and the uncertainty principle,” arXiv:1703.01080, 2017.Google Scholar
Bombieri, E., Le grand crible dans la théorie analytique des nombres. Société Mathématique de France, 1974.Google Scholar
Montgomery, H. L., Twentieth century harmonic analysis – a celebration. Springer, 2001, ch. Harmonic analysis as found in analytic number theory.Google Scholar
Baraniuk, R. G. and Wakin, M. B., “Random projections of smooth manifolds,” Found. Comput. Math., vol. 9, no. 1, pp. 51–77, 2009.CrossRefGoogle Scholar
Candès, E. J. and Recht, B., “Exact matrix completion via convex optimization,” Found. Comput. Math., vol. 9, no. 6, pp. 717–772, 2009.Google Scholar
Eldar, Y. C., Kuppinger, P., and Bölcskei, H., “Block-sparse signals: Uncertainty relations and efficient recovery,” IEEE Trans. Signal Processing, vol. 58, no. 6, pp. 3042–3054, 2010.Google Scholar
Candès, E. J. and Plan, Y., “Tight oracle inequalities for low-rank matrix recovery from a minimal number of noisy random measurements,” IEEE Trans. Information Theory, vol. 4, no. 57, pp. 2342–2359, 2011. uncertainty Relations and Sparse Signal Recovery 195Google Scholar
Alberti, G., Bölcskei, H., De Lellis, C., Koliander, G., and Riegler, E., “Lossless analog compression,” submitted to IEEE Trans. Information Theory, arXiv:1803.06887, 2018.Google Scholar
Riegler, E., Stotz, D., and Bölcskei, H., “Information -theoretic limits of matrix completion,” in Proc. IEEE International Symposium on Information Theory, 2015, pp. 1836–1840.CrossRefGoogle Scholar
Izenman, A. J., “Introduction to manifold learning,” WIREs Comput. Statist., vol. 4, pp. 439–446, 2012.Google Scholar
Lu, H., Fainman, Y., and Hecht-Nielsen, R., “Image manifolds,” in Proc. SPIE, 1998, vol. 3307, pp. 52–63.Google Scholar
Sochen, N. and Zeevi, Y. Y., “Representation of colored images by manifolds embedded in higher dimensional non-Euclidean space,” in Proc. IEEE International Conference on Image Processing, 1998, pp. 166–170.Google Scholar
Hinton, G. E., Dayan, P., and Revow, M., “Modeling the manifolds of images of handwritten digits,” IEEE Trans. Neural Networks, vol. 8, no. 1, pp. 65–74, 1997.Google Scholar
Leland, W. E., Taqqu, M. S., Willinger, W., and Wilson, D. V., “On the self-similar nature of Ethernet traffic (extended version),” IEEE/ACM Trans. Networks, vol. 2, no. 1, pp. 1–15, 1994.CrossRefGoogle Scholar
Horn, R. A. and Johnson, C. R., Matrix analysis. Cambridge University Press, 1990.Google Scholar
Cover, T. M. and Thomas, J. A., Elements of information theory, 2nd edn. Wiley, 2006.Google Scholar
Abel, J. S. and Smith III, J. O., “Restoring a clipped signal,,” in Proc. IEEE International Conference on Acoustics Speech Signal Processing, 1991, pp. 1745–1748.Google Scholar
Mallat, S. G. and Yu, G., “Super-resolution with sparse mixing estimators,” IEEE Trans. Image Processing, vol. 19, no. 11, pp. 2889–2900, 2010.CrossRefGoogle ScholarPubMed
Elad, M. and Hel-Or, Y., “Fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur,” IEEE Trans. Image Processing, vol. 10, no. 8, pp. 1187–1193, 2001.Google Scholar
Bertalmio, M., Sapiro, G., v. Caselles, and Ballester, C., “Image inpainting,” in Proc. 27th Annual Conference on Computer Graphics and Interactive Techniques, 2000, pp. 417–424.Google Scholar
Elad, M., Starck, J.-L., Querre, P., and Donoho, D. L., “Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA),” Appl. Comput. Harmonic Analysis, vol. 19, pp. 340–358, 2005.Google Scholar
Donoho, D. L. and Kutyniok, G., “Microlocal analysis of the geometric separation problem,” Commun. Pure Appl. Math., vol. 66, no. 1, pp. 1–47, 2013.Google Scholar
Tropp, J. A., “On the conditioning of random subdictionaries,” Appl. Comput. Harmonic Analysis, vol. 25, pp. 1–24, 2008.Google Scholar
Candès, E. J., Romberg, J., and Tao, T., “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Information Theory, vol. 52, no. 2, pp. 489–509, 2006.Google Scholar
Pope, G., Bracher, A., and Studer, C., “Probabilistic recovery guarantees for sparsely corrupted signals,” IEEE Trans. Information Theory, vol. 59, no. 5, pp. 3104–3116, 2013.Google Scholar
Donoho, D. L., “Compressed sensing,” IEEE Trans. Information Theory, vol. 52, no. 4, pp. 1289–1306, 2006.Google Scholar
Tropp, J. A., “On the linear independence of spikes and sines,” J. Fourier Analysis Applications, vol. 14, no. 5, pp. 838–858, 2008.CrossRefGoogle Scholar
Donoho, D. L., Johnstone, I. M., Hoch, J. C., and Stern, A. S., “Maximum entropy and the nearly black object,” J. Roy. Statist. Soc. Ser. B., vol. 54, no. 1, pp. 41–81, 1992.Google Scholar
Feng, P. and Bresler, Y., “Spectrum -blind minimum-rate sampling and reconstruction of multiband signals,” in Proc. IEEE International Conference on Acoustics Speech Signal Processing, 1996, pp. 1688–1691.Google Scholar
Mishali, M. and Eldar, Y. C., “From theory to practice: Sub-Nyquist sampling of sparse wideband analog signals,” IEEE J. Selected Areas Commun., vol. 4, no. 2, pp. 375–391, 2010.Google Scholar
Heckel, R. and Bölcskei, H., “Identification of sparse linear operators,” IEEE Trans. Information Theory, vol. 59, no. 12, pp. 7985–8000, 2013.Google Scholar
Falconer, K., Fractal geometry, 1st edn. Wiley, 1990.Google Scholar
Vaaler, J. D., “Some extremal functions in Fourier analysis,” Bull. Amer. Math. Soc., vol. 2, no. 12, pp. 183–216, 1985.Google Scholar
Heil, C., A basis theory primer. Springer, 2011.Google Scholar
Mattila, P., Geometry of sets and measures in Euclidean space: Fractals and rectifiability. Cambridge University Press, 1999.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×