Description
Mathematical Theory of Bayesian Statistics introduces the mathematical foundation of Bayesian inference which is well-known to be more accurate in many real-world problems than the maximum likelihood method. Recent research has uncovered several mathematical laws in Bayesian statistics, by which both the generalization loss and the marginal likelihood are estimated even if the posterior distribution cannot be approximated by any normal distribution.
Features
- Explains Bayesian inference not subjectively but objectively.
- Provides a mathematical framework for conventional Bayesian theorems.
- Introduces and proves new theorems.
- Cross validation and information criteria of Bayesian statistics are studied from the mathematical point of view.
- Illustrates applications to several statistical problems, for example, model selection, hyperparameter optimization, and hypothesis tests.
This book provides basic introductions for students, researchers, and users of Bayesian statistics, as well as applied mathematicians.
Author
Sumio Watanabe is a professor of Department of Mathematical and Computing Science at Tokyo Institute of Technology. He studies the relationship between algebraic geometry and mathematical statistics.
Table of Contents
Definition of Bayesian Statistics
Bayesian Statistics
Probability distribution
True Distribution
Statistical model, prior, and posterior
Examples of Posterior Distributions
Estimation and Generalization
Marginal Likelihood or Partition Function
Conditional Independent Cases
Statistical Models
Normal Distribution
Multinomial Distribution
Linear regression
Neural Network
Finite Normal Mixture
Nonparametric Mixture
Basic Formula of Bayesian Observables
Formal Relation between True and Model
Normalized Observables
Cumulant Generating Functions
Basic Bayesian Theory
Regular Posterior Distribution
Division of Partition Function
Asymptotic Free Energy
Asymptotic Losses
Proof of Asymptotic Expansions
Point Estimators
Standard Posterior Distribution
Standard Form
State Density Function
Asymptotic Free Energy
Renormalized Posterior Distribution
Conditionally Independent Case
General Posterior Distribution
Bayesian Decomposition
Resolution of Singularities
General Asymptotic Theory
Maximum A Posteriori Method
Markov Chain Monte Carlo
Metropolis Method
Basic Metropolis Method
Hamiltonian Monte Carlo
Parallel Tempering
Gibbs Sampler
Gibbs Sampler for Normal Mixture
Nonparametric Bayesian Sampler
Numerical Approximation of Bayesian Observables
Generalization and Cross Validation Losses
Numerical Free Energy
Information Criteria
Model Selection
Criteria for Generalization Loss
Comparison of ISCV with WAIC
Criteria for Free Energy
Discussion for Model Selection
Hyperparameter Optimization
Criteria for Generalization Loss
Criterion for Free energy
Discussion for Hyperparameter Optimization
Topics in Bayesian Statistics
Formal Optimality
Bayesian Hypothesis Test
Bayesian Model Comparison
Phase Transition
Discovery Process
Hierarchical Bayes
Basic Probability Theory
Delta Function
Kullback-Leibler Distance
Probability Space
Empirical Process
Convergence of Expected Values
Mixture by Dirichlet Process
