内容説明
『学習物理学入門』の英語版。An introductory textbook that examines the interplay between physics and AI/machine learning. Aimed at physics students, it provides a smooth entry into machine learning and explores the collaborative relationship between the two fields. [language: English]
目次
[Table of Contents]
Preface
Introduction
A Machine Learning and Physics
A1. Linear Models
A1.1 Least Squares Method and Linear Regression
A1.1.1 Least Squares Method
A1.1.2 Convex Functions
A1.1.3 Conditions for Convexity of Multivariate Functions
A1.1.4 Linear Models
A1.1.5 Continuation of Least Squares Method
A1.2 Entropy
A1.2.1 Probability
A1.2.2 Shannon Entropy
A1.2.3 Relative Entropy and KL Divergence
A1.2.4 Jensens Inequality
A1.2.5 Gaussian Distribution
A1.3 Maximum Likelihood Estimation
A1.3.1 Likelihood Function
A1.3.2 Maximum Likelihood from KL Divergence
A1.4 Generalized Linear Models
A1.4.1 Binary Classification and Logistic Regression
A1.4.2 Origin of Cross-Entropy
A1.5 Classification of Machine Learning
A1.6 Generalization, Overfitting and Underfitting
A1.7 Random Numbers
A1.7.1 What are Random Numbers
A1.7.2 Uniform Random Numbers
A1.7.3 Gaussian Random Numbers
A2. Neural Networks (NN)
A2.1 Neural Networks
A2.2 Data Representation
A2.2.1 Vectorization of Images
A2.2.2 One-Hot Representation
A2.3 Fully Connected Neural Networks with General Number of Layers
A2.4 Gradient Descent Method
A2.5 Activation Functions and Their Derivatives
A2.6 Backpropagation
A2.7 Gradient Vanishing Problem
A3. Symmetry and Machine Learning: Convolution and Equivariant NN
A3.1 Equivariance and Convolutional Neural Networks
A3.2 Image Filters
A3.3 Convolutional Layer
A3.3.1 Two-Dimensional Convolution
A3.3.2 Pooling
A3.4 Group Theory and Symmetry
A3.5 Symmetry and Equivariance
A3.5.1 Ways to Incorporate Symmetry
A3.5.2 Group Equivariant Neural Networks
A3.5.3 Inductive Bias
A3.5.4 Gauge Symmetry and Neural Networks
A4. Classical Mechanics and Machine Learning: Neural Networks and Differential Equations
A4.1 Fundamental Equations of Physics and Machine Learning
A4.1.1 The Role of Differential Equations
A4.1.2 Embedding Physics Problems into Machine Learning
A4.2 Physics-Informed Neural Networks (PINN)
A4.3 Viewing Neural Networks as Differential Equations
A4.3.1 Methods for Handling Differential Equations in Machine Learning
A4.3.2 Locality of NN
A4.3.3 ResNet and Differential Equations
A4.3.4 Locality within Layers and Convolutional NN
A4.4 Representation of Specific Equations of Motion by NN
A4.4.1 Example of a Particle in a Potential
A4.4.2 Hamiltonian Systems
A5. Quantum Mechanics and Machine Learning: Neural Network Wave Functions
A5.1 Quantum Mechanics and Eigenvalue Problems
A5.2 Quantum Many-Body Problems on Lattices
A5.3 Variational Method and Trial Functions
A5.4 Neural Network Wave Functions in Small Quantum Systems
A5.4.1 Analytical Solution of the Two-Site Transverse-Field Ising Model
A5.4.2 Approximate Solution Using Neural NetworkWave Functions
A5.5 Neural Network Wave Functions in Larger Quantum Systems
A5.5.1 Exact Numerical Solution Using the Exact Diagonalization Method
A5.5.2 Approximate Numerical Solution Using Neural Network Wave Function
A5.6 Future Prospects
B Machine Learning Models and Physics
B1. Transformer
B1.1 Words and Embedding Vectors
B1.1.1 Use Theory of Meaning and Embedding
B1.1.2 Search from Key-Value Store and Attention Mechanism
B1.1.3 Transformer Architecture
B1.2 Transformers in NLP and Computer Vision
B1.2.1 GPT
B1.2.2 Vision Transformer
B2. Diffusion Models and Path Integrals
B2.1 Principles of Diffusion Models
B2.1.1 The Idea of Diffusion Models
B2.1.2 Diffusion Models and Langevin Equation
B2.1.3 Sampling Process of Diffusion Models
B2.1.4 Training of Diffusion Models
B2.1.5 Probability Flow ODE
B2.2 Path Integral Quantization
B2.3 Path Integral Formulation of Diffusion Models
B2.3.1 Derivation of the Reverse Process
B2.3.2 Derivation of Loss Function for Diffusion Model Training
B2.3.3 Probability Flow and Classical Limit
B3. Mechanism Behind Machine Learning: Statistical Mechanical Approach
B3.1 Infinite-Width DNN: Signal Propagation
B3.1.1 Spin Model
B3.1.2 Macroscopic Laws of Signal Propagation
B3.1.3 Mean Field Theory and Order-to-Chaos Phase Transition
B3.1.4 Macroscopic Law of Backpropagation
B3.1.5 Vanishing and Exploding Gradient Problem as Phase Transition
B3.1.6 Connection with Kernel Methods
B3.2 Infinite-Width DNN Model: Learning Regimes
B3.2.1 NTK Regime
B3.2.2 μP
B3.3 Linear Regression Model
B3.3.1 Generalization Error in Over-Parameterized Models
B3.3.2 Typical Evaluation of Generalization Error
B4. Large Language Models and Science
B4.1 Large Language Models
B4.1.1 Next Word Prediction
B4.1.2 Training of Large Language Models
B4.2 Applications of Large Language Models
B4.2.1 Arithmetic Capabilities of Large Language Models
B4.2.2 Proof Capabilities of Large Language Models
B4.2.3 Cubism in Mathematics
Afterword
Index
