Full Description
Radio astronomy is irreversibly moving towards the exabyte era. In the advent of all-sky radio observations, efficient tools and methods to manage the large data volume generated have become imperative.
This book brings together the knowledge of several different research fields to present an overview of current state-of-the-art methods in data-intensive radio astronomy. Its approach is comprehensive and data-centric, offering a coherent look at the four distinct parts of the data lifecycle:
Data creation, storage and archives
Data processing
Post-processing and data analysis
Data access and reuse
Large data management has been the topic of discussion within the astronomical community for decades. Some relevant areas explored in this volume are: ongoing technological innovations in interferometers and computing facilities; difficulties and possible solutions for the huge processing demands of radio telescope projects such as LOFAR, MeerKat, ASKAP; concepts for reliable and fast storage for archiving; and more.
Written by experts across astrophysics, high-energy particle physics, data science, and computer science, this volume will help researchers and advanced students better understand the current state of data-intensive radio astronomy and tackle the major problems that may arise from future instruments.
Contents
Preface.- Introduction - Concepts and challenges of data-intensive radio astronomy.- Part 1. DATA CREATION, STORAGE AND ARCHIVES.- CHAPTER 1: From the data generation to the archive.- CHAPTER 2: Storage and archives.- CHAPTER 3: Computing infrastructure.- Part 2. DATA PROCESSING.- CHAPTER 4: Challenges of radio data processing (big radio data processing).- CHAPTER 5: Implementations for specific radio observatories.- CHAPTER 6: Co-design and software architecture.- CHAPTER 7: Lesson learned from SKA pathfinders regarding processing.- PART 3. POST-PROCESSING AND DATA ANALYSIS.- CHAPTER 8: Continuum Source extraction and identification.- CHAPTER 9: Other types of source extraction and identification.- CHAPTER 10: Using AI for radio (big) data.- CHAPTER 11: Visualisation for analysis.- PART 4. DATA ACCESS AND REUSE (accessibility, VR/VO FAIR).- CHAPTER 12: Exploitation platforms & Virtual Observatory.- CHAPTER 14: Data documentation beyond provenance: metadata, Research Data Management (RDM), FAIR.- Epilogue and future outlook.- Glossary.- Index.
