Monitoring and Maintaining GenAI Systems : Observability, drift detection, auditing, and MLOps for reliable production GenAI applications

  • 予約

Monitoring and Maintaining GenAI Systems : Observability, drift detection, auditing, and MLOps for reliable production GenAI applications

  • ただいまウェブストアではご注文を受け付けておりません。 ⇒古書を探す
  • 製本 Paperback:紙装版/ペーパーバック版
  • 言語 ENG
  • 商品コード 9781808655531

Full Description

Learn how to monitor and maintain production GenAI systems. Use Prometheus, Grafana, and Weights & Biases to track metrics, detect model drift and hallucinations, and apply MLOps best practices

Key Features

Monitor GenAI systems end to end with Prometheus, Grafana, and Weights & Biases
Detect model drift and hallucinations, and debug and evaluate model performance
Build audit trails and apply MLOps and DevOps practices to keep GenAI compliant

Book DescriptionDeploying a generative AI system is only the beginning. Models behave non-deterministically, quality drifts, costs creep up, and a single hallucination can erode user trust. This practical guide shows you how to monitor, debug, govern, and maintain GenAI systems once they are in production.
Following a running case study of an AI assistant in a Slack-based workflow, you'll learn how to monitor every layer of the stack, from the prompt interface to the infrastructure beneath it. You'll define and interpret key metrics such as latency, throughput, uptime, token usage, coherence, and hallucination rates, then build layered dashboards and alerts with Prometheus, Grafana, and Weights & Biases.You'll evaluate and debug model performance with perplexity, BLEU, ROUGE, A/B testing, and prompt regression testing; capture prompts and responses securely with PII redaction, encrypted storage, and role-based access; and build audit trails that support governance and compliance. You'll also learn when and how to retrain, update, and version models, and how to apply CI/CD, shadow testing, canary releases, and incident management to prompts and models.
By the end of this book, you'll have the concepts, patterns, and templates to keep GenAI systems healthy, accountable, and aligned with business outcomes.What you will learn

Interpret system and model metrics to monitor GenAI behavior
Detect and respond to model drift and hallucinations
Build dashboards and alerts with Prometheus and Grafana
Track prompts, token usage, and drift with Weights & Biases
Evaluate outputs using perplexity, BLEU, ROUGE, and A/B tests
Log and audit interactions with PII redaction and access control
Plan retraining, versioning, and CI/CD for prompts and models
Connect GenAI performance to business outcomes and governance

Who this book is forMonitoring and Maintaining GenAI Systems is for data scientists, ML engineers, and software developers who run GenAI applications in production, as well as DevOps professionals and AI system architects who need to keep them reliable. It also suits product managers and technical leaders who want to understand GenAI system health, risk, and long-term maintenance. No prior GenAI experience is required, but basic familiarity with software systems and AI concepts is helpful.

Contents

Table of Contents

Understanding GenAI System Monitoring
Measuring and Evaluating GenAI Performance
Maintaining GenAI Systems in Production
Applying and Maturing GenAI Observability

最近チェックした商品