Search results for: topics-in-statistical-information-theory

Topics in Statistical Information Theory

Author : Solomon Kullback
File Size : 55.12 MB
Format : PDF, Docs
Download : 837
Read : 748
Download »
The relevance of information theory to statistical theory and its applications to stochastic processes is a unifying influence in these TOPICS. The integral representation of discrimination information is presented in these TOPICS reviewing various approaches used in the literature, and is also developed herein using intrinsically information-theoretic methods. Log likelihood ratios associated with various stochastic processes are computed by an application of minimum discrimination information estimates. Linear discriminant functionals are used in the information-theoretic analysis of a variety of stochastic processes. Sections are numbered serially within each chapter, with a decimal notation for subsections. Equations, examples, theorems and lemmas, are numbered serially within each section with a decimal notation. The digits to the left of the decimal point represent the section and the digits to the right of the decimal point the serial number within the section. When reference is made to a section, equation, example, theorem or lemma within the same chapter only the section number or equation number, etc., is given. When the reference is to a section ,equation, etc., in a different chapter, then in addition to the section or equation etc., number, the chapter number is also given. References to the bibliography are by the author's name followed by the year of publication in parentheses. The transpose of a matrix is denoted by a prime; thus one-row matrices are denoted by primes as the transposes of one-column matrices (vectors).

Topics in Statistical Information Theory

Author : Solomon Kullback
File Size : 85.20 MB
Format : PDF, ePub
Download : 448
Read : 492
Download »
The relevance of information theory to statistical theory and its applications to stochastic processes is a unifying influence in these TOPICS. The integral representation of discrimination information is presented in these TOPICS reviewing various approaches used in the literature, and is also developed herein using intrinsically information-theoretic methods. Log likelihood ratios associated with various stochastic processes are computed by an application of minimum discrimination information estimates. Linear discriminant functionals are used in the information-theoretic analysis of a variety of stochastic processes. Sections are numbered serially within each chapter, with a decimal notation for subsections. Equations, examples, theorems and lemmas, are numbered serially within each section with a decimal notation. The digits to the left of the decimal point represent the section and the digits to the right of the decimal point the serial number within the section. When reference is made to a section, equation, example, theorem or lemma within the same chapter only the section number or equation number, etc., is given. When the reference is to a section ,equation, etc., in a different chapter, then in addition to the section or equation etc., number, the chapter number is also given. References to the bibliography are by the author's name followed by the year of publication in parentheses. The transpose of a matrix is denoted by a prime; thus one-row matrices are denoted by primes as the transposes of one-column matrices (vectors).

Information Theory and Statistics

Author : Solomon Kullback
File Size : 65.76 MB
Format : PDF, Mobi
Download : 627
Read : 1055
Download »
Highly useful text studies logarithmic measures of information and their application to testing statistical hypotheses. Includes numerous worked examples and problems. References. Glossary. Appendix. 1968 2nd, revised edition.

Information Theory and Coding by Example

Author : Mark Kelbert
File Size : 22.34 MB
Format : PDF, Docs
Download : 559
Read : 598
Download »
A valuable teaching aid. Provides relevant background material, many examples and clear solutions to problems taken from real exam papers.

Information Theory and Network Coding

Author : Raymond W. Yeung
File Size : 46.23 MB
Format : PDF, Kindle
Download : 970
Read : 1069
Download »
This book is an evolution from my book A First Course in Information Theory published in 2002 when network coding was still at its infancy. The last few years have witnessed the rapid development of network coding into a research ?eld of its own in information science. With its root in infor- tion theory, network coding has not only brought about a paradigm shift in network communications at large, but also had signi?cant in?uence on such speci?c research ?elds as coding theory, networking, switching, wireless c- munications,distributeddatastorage,cryptography,andoptimizationtheory. While new applications of network coding keep emerging, the fundamental - sults that lay the foundation of the subject are more or less mature. One of the main goals of this book therefore is to present these results in a unifying and coherent manner. While the previous book focused only on information theory for discrete random variables, the current book contains two new chapters on information theory for continuous random variables, namely the chapter on di?erential entropy and the chapter on continuous-valued channels. With these topics included, the book becomes more comprehensive and is more suitable to be used as a textbook for a course in an electrical engineering department.

A First Course in Information Theory

Author : Raymond W. Yeung
File Size : 78.94 MB
Format : PDF, Docs
Download : 186
Read : 1192
Download »
This book provides an up-to-date introduction to information theory. In addition to the classical topics discussed, it provides the first comprehensive treatment of the theory of I-Measure, network coding theory, Shannon and non-Shannon type information inequalities, and a relation between entropy and group theory. ITIP, a software package for proving information inequalities, is also included. With a large number of examples, illustrations, and original problems, this book is excellent as a textbook or reference book for a senior or graduate level course on the subject, as well as a reference for researchers in related fields.

Elements of Information Theory

Author : Thomas M. Cover
File Size : 27.77 MB
Format : PDF, Mobi
Download : 944
Read : 361
Download »
The latest edition of this classic is updated with new problem sets and material The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory. All the essential topics in information theory are covered in detail, including entropy, data compression, channel capacity, rate distortion, network information theory, and hypothesis testing. The authors provide readers with a solid understanding of the underlying theory and applications. Problem sets and a telegraphic summary at the end of each chapter further assist readers. The historical notes that follow each chapter recap the main points. The Second Edition features: * Chapters reorganized to improve teaching * 200 new problems * New material on source coding, portfolio theory, and feedback capacity * Updated references Now current and enhanced, the Second Edition of Elements of Information Theory remains the ideal textbook for upper-level undergraduate and graduate courses in electrical engineering, statistics, and telecommunications.

New Developments in Statistical Information Theory Based on Entropy and Divergence Measures

Author : Leandro Pardo
File Size : 55.14 MB
Format : PDF, Mobi
Download : 432
Read : 1069
Download »
This book presents new and original research in Statistical Information Theory, based on minimum divergence estimators and test statistics, from a theoretical and applied point of view, for different statistical problems with special emphasis on efficiency and robustness. Divergence statistics, based on maximum likelihood estimators, as well as Wald’s statistics, likelihood ratio statistics and Rao’s score statistics, share several optimum asymptotic properties, but are highly non-robust in cases of model misspecification under the presence of outlying observations. It is well-known that a small deviation from the underlying assumptions on the model can have drastic effect on the performance of these classical tests. Specifically, this book presents a robust version of the classical Wald statistical test, for testing simple and composite null hypotheses for general parametric models, based on minimum divergence estimators.

Statistical Information and Likelihood

Author : D. Basu
File Size : 85.65 MB
Format : PDF, ePub
Download : 507
Read : 674
Download »
It is an honor to be asked to write a foreword to this book, for I believe that it and other books to follow will eventually lead to a dramatic change in the current statistics curriculum in our universities. I spent the 1975-76 academic year at Florida State University in Tallahassee. My purpose was to complete a book on Statistical Reliability Theory with Frank Proschan. At the time, I was working on total time on test processes. At the same time, I started attending lectures by Dev Basu on statistical inference. It was Lehmann's hypothesis testing course and Lehmann's book was the text. However, I noticed something strange - Basu never opened the book. He was obviously not following it. Instead, he was giving a very elegant, measure theoretic treatment of the concepts of sufficiency, ancillarity, and invariance. He was interested in the concept of information - what it meant. - how it fitted in with contemporary statistics. As he looked at the fundamental ideas, the logic behind their use seemed to evaporate. I was shocked. I didn't like priors. I didn't like Bayesian statistics. But after the smoke had cleared, that was all that was left. Basu loves counterexamples. He is like an art critic in the field of statistical inference. He would find a counterexample to the Bayesian approach if he could. So far, he has failed in this respect.

Information Theory for Continuous Systems

Author : Shunsuke Ihara
File Size : 68.82 MB
Format : PDF, ePub, Docs
Download : 359
Read : 1168
Download »
This book provides a systematic mathematical analysis of entropy and stochastic processes, especially Gaussian processes, and its applications to information theory.The contents fall roughly into two parts. In the first part a unified treatment of entropy in information theory, probability theory and mathematical statistics is presented. The second part deals mostly with information theory for continuous communication systems. Particular emphasis is placed on the Gaussian channel.An advantage of this book is that, unlike most books on information theory, it places emphasis on continuous communication systems, rather than discrete ones.