- ホーム
- > 洋書
- > 英文書
- > Computer / General
Full Description
Master LLM quantization from sub-byte mathematics and algorithms to CUDA kernels, modern formats, inference runtimes, and production optimization for efficient, memory-conscious, high-performance AI systems.
Key Features
Master quantization math, sub-byte formats, GPTQ, AWQ, SmoothQuant, QLoRA, and FP8 techniques
Optimize CUDA, Triton, CUTLASS, KV-cache quantization, memory usage, and high-performance inference
Deploy quantized LLMs with GGUF, vLLM, TensorRT-LLM, ONNX Runtime, profiling, and production optimization
Book DescriptionMastering LLM Quantization: Architecture and Code is a practical engineering guide to the mathematics, algorithms, implementation techniques, and production systems behind efficient large language model inference.
The book begins with quantization basics, numerical representations, error analysis, calibration, scaling, and sub-byte math. Readers explore GPTQ, AWQ, SmoothQuant, QLoRA, quantization-aware training, FP8, and other low-bit techniques. Implementation sections connect mathematical methods with real GPU systems, covering CUDA kernel engineering, Triton, CUTLASS, memory layouts, tensor operations, kernel optimization, and hardware-aware performance analysis. It also covers activation and KV-cache quantization, memory management, batching, and techniques for reducing inference cost while preserving model quality.
Practical deployment chapters examine GGUF, vLLM, TensorRT-LLM, ONNX Runtime, and other inference environments, showing how quantized models move into production systems. Hardware profiling, benchmarking, precision trade-offs, throughput, latency, memory consumption, and deployment optimization are treated as practical engineering problems. The result is an end-to-end perspective connecting quantization mathematics, algorithms, GPU architecture, optimized kernels, inference runtimes, and production deployment.What you will learn
Understand quantization math, numerical formats, scaling, calibration, and quantization error
Apply GPTQ, AWQ, SmoothQuant, QLoRA, QAT, FP8, and low-bit techniques
Optimize CUDA, Triton, CUTLASS, memory layouts, and kernels for inference
Quantize model weights, activations, and KV caches for efficient large-model deployment
Benchmark latency, throughput, memory usage, and quality across precision formats
Deploy quantized models with GGUF, vLLM, TensorRT-LLM, and ONNX Runtime
Profile hardware and select quantization strategies for deployment constraints
Who this book is forThis book is for AI and ML engineers, LLM developers, GPU and CUDA programmers, C++ and systems engineers, inference-engine developers, MLOps and platform engineers, researchers, and technical professionals optimizing large-model deployment. A solid background in programming, machine learning, Linux, or GPU computing is recommended.
Contents
Table of Contents
The High-Performance Mandate: Modern Compute Constraints and Quantization Economics
Mathematical Anatomy of Discretization: Floating-Point Formats versus Integer Precision
Anatomy of Modern Sub-Byte Formats: FP8 (E4M3, E5M2), FP4, and NormalFloat4 (NF4)
Quantization Fundamentals: Uniform, Affine, and Symmetric Transformation Mechanics
Granularity Paradigms: Per-Tensor, Per-Channel, Group-Wise, and Vector Quantization
Scale and Zero-Point Calculation: Exact Range Mapping and Clipping Optimizations
Emergent Features and Magnitude Anomalies: The Outlier Activation Phenomenon in Scale
Optimal Brain Surgeon and Second-Order Taylor Expansion Foundations
Hessian Matrix Approximation: Empirical Fisher Information and Damping Factors
GPTQ Architecture: Mathematical Derivation and Layer-by-Layer Inverse Hessian Updates
Implementing GPTQ from Scratch: Python, PyTorch, and Cholesky Decomposition
Engineering Fast Block-Wise Updates and Lazy Quantization in GPTQ
Activation-Aware Weight Quantization (AWQ): Salience Identification and Protection
Implementing AWQ from Scratch: Optimization-Free Grid Search and Per-Channel Scaling
SmoothQuant: Mathematical Principle of Activation-to-Weight Difficulty Migration
Implementing SmoothQuant: Automated Scaling Vector Injection for Transformer Layers
SpQR: Sparse-Dense Decomposed Quantization for Ultra-Low Precision Outlier Preservation
SqueezeLLM: Non-Uniform Precision Allocation via Hessian-Driven Sensitivity Clustering
Extreme Compression: Randomized Hadamard Transforms and Vector Quantization with QuIP and QuIP Sharp
OmniQuant: Learnable Equivalent Transformations and Weight-Activation Boundary Tuning
Quantization-Aware Training (QAT) Mechanics: Overcoming Gradient Discretization with STE
Learnable Step Size Quantization (LSQ) and Differentiable Clipping Boundaries
QLoRA Foundations: 4-Bit NormalFloat Precision, Double Quantization, and Paged Optimizers
LoftQ and Initialization-Aware Low-Rank Fine-Tuning under Harsh Discretization
FP8 Native Training and Mixed-Precision Forward-Backward Dynamics
KV Cache Bottlenecks: Memory Footprint Dynamics in Ultra-Long Context Windows
FP8 and INT8 KV Caching: Implementing Dynamic Quantized Ring Buffers
Asymmetric Grouped KV Quantization with RoPE Invariant Preservation
GPU Microarchitecture Essentials: Streaming Multiprocessors, Warps, and Tensor Cores
Low-Level Arithmetic: INT4 Bit-Packing, Unpacking, and Bit-Shift Manipulation Algorithms
CUDA Kernel Engineering: Writing a Fast W4A16 Dequantization GEMM Kernel from Scratch
Writing High-Performance Triton Kernels for W8A8 and FP8 Fused Matrix Multiplication
CUTLASS 3.x Deep Dive: Tensor Core MMA Pipelines and Shared Memory Swizzling for Sub-Byte GEMM
Marlin Kernel Architecture: Near-Ideal Memory-Bandwidth Saturated Low-Bit GEMM Execution
CPU and Edge Hardware Acceleration: AVX-512, AMX, ARM NEON, and Apple Metal Pipelines
GGUF Architecture and Low-Bit Vector Dequantization in Llama.cpp
Compiling Quantized LLM Graphs with TensorRT-LLM and ONNX Runtime
Production Serving with vLLM: Integrating Marlin, AWQ, and PagedAttention at Scale
Speculative Decoding Architectures Leveraging Asymmetric Quantized Draft Models
Calibration Dataset Engineering: Token Diversity, Context Lengths, and Domain Bias Mitigation
Comprehensive Evaluation Protocols: Perplexity, MMLU, GSM8K, and MT-Bench Zero-Regression Analysis
Debugging Quantization Artifacts: Layer-by-Layer Signal-to-Noise Ratio and Cosine Distance Profiling
Hardware Profiling with NVIDIA Nsight: Roofline Analysis and Memory Throughput Verification
Enterprise Architecture Blueprint: End-to-End Quantization and Deployment Pipeline for 70B+ Parameter Models



