Information Theory, Inference & Learning Algorithms

1st edition
  • 4.0 (1 rating) ·
  • 16 Want to read
  • 1 Currently reading
  • 1 Have read

My Reading Lists:

Create a new list

Check-In

×Close
Add an optional check-in date. Check-in dates are used to track yearly reading goals.
Today

  • 4.0 (1 rating) ·
  • 16 Want to read
  • 1 Currently reading
  • 1 Have read


Download Options

Buy this book

Last edited by MARC Bot
September 4, 2024 | History

Information Theory, Inference & Learning Algorithms

1st edition
  • 4.0 (1 rating) ·
  • 16 Want to read
  • 1 Currently reading
  • 1 Have read

Book Jacket:

This textbook introduces theory in tandem with applications. Information theory is taught alongside practical communication systems, such as arithmetic coding for data compression and sparse-graph codes for error-correction. A toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods, and variational approximations, are developed alongside applications of these tools to clustering, convolutional codes, independent component analysis, and neural networks.

Publisher Description:

This textbook offers comprehensive coverage of Shannon's theory of information as well as the theory of neural networks and probabilistic data modelling. It includes explanations of Shannon's important source encoding theorem and noisy channel theorem as well as descriptions of practical data compression systems. Many examples and exercises make the book ideal for students to use as a class textbook, or as a resource for researchers who need to work with neural networks or state-of-the-art error-correcting codes.

Publish Date
Language
English
Pages
640

Buy this book

Previews available in: Undetermined English

Edition Availability
Cover of: Information Theory, Inference and Learning Algorithms
Information Theory, Inference and Learning Algorithms
2004, University of Cambridge ESOL Examinations, TBS
in English
Cover of: INFORMATION THEORY, INFERENCE, AND LEARNING ALGORITHMS.
INFORMATION THEORY, INFERENCE, AND LEARNING ALGORITHMS.
2003, CAMBRIDGE UNIV PRESS, Cambridge University Press
in Undetermined
Cover of: Information Theory, Inference & Learning Algorithms
Information Theory, Inference & Learning Algorithms
2003, Cambridge University Press
Hardcover in English - 1st edition

Add another edition?

Book Details


First Sentence

"In this chapter we discuss how to measure the information content of the outcome of a random experiment."

Table of Contents

Preface.
Page v
Chapter 1. Introduction to information theory
Page 3
Chapter 2. Probability, entropy, and inference
Page 22
Chapter 3. More about inference
Page 48
Part I. Data Compression
Page 65
Chapter 4. The source coding theorem
Page 67
Chapter 5. Symbol codes
Page 91
Chapter 6. Stream codes
Page 110
Chapter 7. Codes for integers
Page 132
Part II. Noisy-Channel Coding
Page 137
Chapter 8. Correlated random variables
Page 138
Chapter 9. Communication over a noisy channel
Page 146
Chapter 10. The noisy-channel coding theorem
Page 162
Chapter 11. Error-correcting codes and real channels
Page 177
Part III. Further Topics in Information Theory
Page 191
Chapter 12. Hash codes: codes for efficient information retrieval
Page 193
Chapter 13. Binary codes
Page 206
Chapter 14. Very good linear codes exist
Page 229
Chapter 15. Further exercises on information theory
Page 233
Chapter 16. Message passing
Page 241
Chapter 17. Communication over constrained noiseless channels
Page 248
Chapter 18. An aside: crosswords and codebreaking
Page 260
Chapter 19. Why have sex? Information acquisition and evolution
Page 269
Part IV. Probabilities and Inference
Page 281
Chapter 20. An example inference task: clustering
Page 284
Chapter 21. Exact inference by complete enumeration
Page 293
Chapter 22. Maximum likelihood and clustering
Page 300
Chapter 23. Useful probability distributions
Page 311
Chapter 24. Exact marginalization
Page 319
Chapter 25. Exact marginalization in trellises
Page 324
Chapter 26. Exact marginalization in graphs
Page 334
Chapter 27. Laplace's method
Page 341
Chapter 28. Model comparison and Occam's razor
Page 343
Chapter 29. Monte Carlo methods
Page 357
Chapter 30. Efficient Monte Carlo methods
Page 387
Chapter 31. Ising models
Page 400
Chapter 32. Exact Monte Carlo sampling
Page 413
Chapter 33. Variational methods
Page 422
Chapter 34. Independent component analysis and latent variable modelling
Page 437
Chapter 35. Random inference topics
Page 445
Chapter 36. Decision theory
Page 451
Chapter 37. Bayesian inference and sampling theory
Page 457
Part V. Neural Networks
Page 467
Chapter 38. Introduction to neural networks
Page 468
Chapter 39. The single neuron as a classifier
Page 471
Chapter 40. Capacity of a single neuron
Page 483
Chapter 41. Learning as inference
Page 492
Chapter 42. Hopfield networks
Page 505
Chapter 43. Boltzmann machines
Page 522
Chapter 44. Supervised learning in multilayer networks
Page 527
Chapter 45. Gaussian processes
Page 535
Chapter 46. Deconvolution
Page 549
Part VI. Sparse Graph Codes
Page 555
Chapter 47. Low-density parity-check codes
Page 557
Chapter 48. Convolutional codes and turbo codes
Page 574
Chapter 49. Repeat-accumulate codes
Page 582
Chapter 50. Digital fountain codes
Page 589
Part VII. Appendices
Page 597
Appendix A. Notation
Page 598
Appendix B. Some physics
Page 601
Appendix C. Some mathematics
Page 605
Bibliography.
Page 613
Index.
Page 620

Edition Notes

Full text is online.

Published in
Cambridge, UK, New York, USA
Copyright Date
2003

Classifications

Dewey Decimal Class
003/.54
Library of Congress
Q360 .M23 2003

The Physical Object

Format
Hardcover
Pagination
xii, 628p
Number of pages
640
Dimensions
9.8 x 7.6 x 1.3 inches
Weight
3.3 pounds

ID Numbers

Open Library
OL7749839M
Internet Archive
informationtheor00mack_665
ISBN 10
0521642981
ISBN 13
9780521642989
LCCN
2003055133
OCLC/WorldCat
52377690
Amazon ID (ASIN)
0521642981
Google
AKuMj4PN_EMC
Library Thing
403618
Goodreads
201357

Excerpts

You cannot do inference without making assumptions.
Page 26, added by David.

A central theme of the book.

Links outside Open Library

Community Reviews (0)

Feedback?
No community reviews have been submitted for this work.

Lists

This work does not appear on any lists.

History

Download catalog record: RDF / JSON
September 4, 2024 Edited by MARC Bot import existing book
December 19, 2023 Edited by ImportBot import existing book
November 8, 2023 Edited by raybb merge authors
November 8, 2023 Edited by raybb Merge works
December 10, 2009 Created by WorkBot add works page