Reinforcement Learning for Large Language Models : Making the Hard Math Easy

個数:

Reinforcement Learning for Large Language Models : Making the Hard Math Easy

  • 提携先の海外書籍取次会社に在庫がございます。通常3週間で発送いたします。
    【重要ご説明事項】
    1. 納期遅延や、ご入手不能となる場合が若干ございます。
    2. 複数冊ご注文の場合は、ご注文数量が揃ってからまとめて発送いたします。
    3. 美品のご指定は承りかねます。

    ●3Dセキュア導入とクレジットカードによるお支払いについて
  • 【入荷遅延について】
    世界情勢の影響により、海外からお取り寄せとなる洋書・洋古書の入荷が、表示している標準的な納期よりも遅延する場合がございます。
    おそれいりますが、あらかじめご了承くださいますようお願い申し上げます。
  • ◆画像の表紙や帯等は実物とは異なる場合があります。
  • ◆ウェブストアでの洋書販売価格は、弊社店舗等での販売価格とは異なります。
    また、洋書販売価格は、ご注文確定時点での日本円価格となります。
    ご注文確定後に、同じ洋書の販売価格が変動しても、それは反映されません。
  • 製本 Paperback:紙装版/ペーパーバック版
  • 言語 ENG
  • 商品コード 9781807306793
  • DDC分類 006.35

Full Description

Learn how to train and align large language models with modern reinforcement learning methods using clear explanations and runnable code on free compute.

Key Features

Understand RL for LLMs through intuitive, practical, and mathematical explanations
Train and align language models using PPO, DPO, and modern efficiency methods
Run every example on free Colab resources with small open models
Purchase of the print or Kindle book includes a free PDF eBook

Book DescriptionReinforcement learning is at the heart of aligning modern LLMs, yet many engineers find it difficult to learn due to complex mathematics and expensive infrastructure requirements. This book bridges that gap with clear explanations and hands-on projects that show how to train capable LLMs using free compute and efficient techniques.

You'll build a solid foundation in the mathematics behind RL before progressing through supervised fine-tuning, reward modeling, PPO-based RLHF, DPO, and on-policy distillation. Each concept is reinforced with practical coding exercises and real model outputs, helping you understand the theory and its impact on model behavior.

The book emphasizes efficient training throughout, using techniques such as LoRA, quantization and memory optimization to fine-tune 1.5B-8B parameter models on Google Colab. It also explores advanced topics like process reward models, reasoning, test-time compute, continual learning and debugging.

Written by senior Google Applied AI engineers with deep experience in generative AI systems, this book focuses on practical techniques used in real-world LLM development.

By the end of this book, you can confidently train, align, and deploy LLMs using reinforcement learning, with a clear understanding of both the mathematics and the implementation choices that matter.What you will learn

Understand core reinforcement learning concepts used in LLM alignment
Build complete RLHF pipelines from SFT to policy optimization
Apply PPO, DPO, and on-policy distillation in practice
Train models efficiently using free compute and small architectures
Use process reward models to improve reasoning quality
Diagnose and fix common RL training failures
Choose RL methods based on data and compute constraints

Who this book is forMachine learning engineers, applied researchers, and motivated practitioners who want to use reinforcement learning to improve large language models but have been held back by mathematical complexity or limited compute. Readers should be comfortable with Python and basic ML concepts; no prior reinforcement learning experience is required.

Contents

Table of Contents

Essential Math Toolkit
Why LLMs Need RL: The Alignment Gap
RL Fundamentals: The Complete Picture
Setting Up Your Free Training Environment
Supervised Fine-Tuning: Your Starting Point
RLHF: The Three-Step Dance
Direct Preference Optimization: RLHF Without the RL
On-Policy Distillation: 10x Efficiency Gains
Efficient RL with Limited Compute
Reward Modeling: Automated Evaluation
Process Reward Models: Step-by-Step Excellence
Reasoning Emergence: How RL Discovers Chain-of-Thought
Test-Time Compute: Scaling at Inference
Self-Improvement: Autonomous Learning Systems
Multi-Objective RL: Balancing Multiple Goals
Domain-Specific RL: Code, Math, Tools, Dialogue
Efficient Training Infrastructure
Continual Learning and Catastrophic Forgetting
Debugging and Troubleshooting

最近チェックした商品