- ホーム
- > 洋書
- > 英文書
- > Science / Mathematics
Full Description
Test-driven data analysis is the synthesis of ideas from test-driven development of software to data-intensive work including data science, data analysis, and data engineering. It is a methodology for improving the quality of data and of analytical pipelines and processes. It can be thought of as data analysis as if the answers actually matter.Test-driven data analysis can be thought of as a sibling to reproducible research, with similar concerns, but greater emphasis on automated testing, and less requirement for a human to reproduce results. Extensive checklists are provided that can be used to improve quality before,during, and after analysis.
Key Features:
Prevents costly errors in analytical processes before they reach production through automated data validation and reference testing of data pipelines.
• Provides actionable checklists for issues beyond the reach of automated testing.
• Equips readers with open-source Python tools and language-agnostic command-line interfaces.
• Addresses testing challenges for modern LLM-based systems including chat-bots and coding assistants.
• Instills in analysts an inner voice that is always asking: "How is this misleading data misleading me?"
Contents
Foreword Preface Acknowledgements Author 1 Orientation I Data Validation with Constraints 2 Data Validation 3 Textual Data 4 Profiling and Auditing Data 5 Constraint Discovery and Validation 6 Custom Constraints 7 Practical Considerations 8 Serial Data II Reference Testing 9 Introduction to Reference Tests 10 Modern Software Testing 11 Reference Tests for Analytical Pipelines 12 Testing Models and Modeling III Errors of Interpretation, of Process, & of Applicability 13 Errors of Interpretation I: Formulation 14 Errors of Interpretation II: Communication 15 Errors of Interpretation III: Graphing Sins 16 Errors of Process 17 Errors of Applicability and Errors of Judgement IV Appendices A The TDDA Library, Resources, & Tools B Glossary Bibliography
