Description
This book provides a comprehensive overview of music data analysis, from introductory material to advanced concepts. It covers various applications including transcription and segmentation as well as chord and harmony, instrument and tempo recognition. It also discusses the implementation aspects of music data analysis such as architecture, user interface and hardware. It is ideal for use in university classes with an interest in music data analysis. It also could be used in computer science and statistics as well as musicology.
Table of Contents
K25499 TOC
Introduction
Background and Motivation
Content, Target Audience, Prerequisites, Exercises, and Complementary Material
Book Overview
Chapter Summaries
Course Examples
Authors and Editors
Bibliography
I Music and Audio
The Musical Signal - Physically and Psychologically
Introduction
The Tonal Quality: Pitch - The First Moment
Introduction
Pure and Complex Tones on a Vibrating String
Intervals and Musical Tone Height
Musical Notation and Naming of Pitches and Intervals
The Mel Scale
Fourier Transform
Correlation Analysis
Fluctuating Pitch and Frequency Modulation
Simultaneous Pitches
Other Sounds With and Without Pitch Percepts
Volume - The Second Moment
Introduction
The Physical Basis: Sound Waves in Air
Scales for the Subjective Perception of the Volume
Amplitude Modulation
Uncertainty Principle
Gabor Transform and Spectrogram
Formants, Vowels, and Characteristic Timbres of Voices and Instruments
Sound Fluctuations and Timbre
Physical Model for the Timbre of Wind Instruments
Duration - The Fourth Moment
Integration Times and Temporal Resolvability
Time Structure in Music: Rhythm and Measure
Wavelets and Scalograms
Further Reading
Exercises
Bibliography
Musical Structures and Their Perception
Introduction
Scales and Keys
Clefs
Diatonic and Chromatic Scales
Other Scales
Gestalt and Auditory Scene Analysis
Musical Textures from Monophony to Polyphony
Polyphony and Harmony
Dichotomy of Consonant and Dissonant Intervals
Consonant and Dissonant Intervals and Tone Progression
Elementary Counterpoint
Chords
Modulations
Time Structures of Music
Note Values
Measure
Meter
Rhythm
Elementary Theory of Form
Further Reading
Bibliography
Digital Filters and Spectral Analysis
Introduction
Continuous-Time, Discrete-Time, and Digital Signals
Discrete-Time Systems
Parametric LTI Systems
Digital Filters and Filter Design
The Discrete Fourier Transform
The Discrete Fourier Transform
Frequency Resolution and Zero Padding
Short-time Spectral Analysis
The Constant-Q Transform
Filter Banks for Short-time Spectral Analysis
Uniform Filter Banks
Nonuniform Filter Banks
The Cepstrum
Fundamental Frequency Estimation
Further Reading
Bibliography
Signal-Level Features
Introduction
Timbre Features
Time-Domain Features
Frequency-Domain Features
Mel Frequency Cepstral Coefficients
Harmony Features
Chroma Features
Chroma Energy Normalized Statistics
Timbre-Invariant Chroma Features
Characteristics of Partials
Rhythmic Features
Features for Onset Detection
Phase-Domain Characteristics
Fluctuation Patterns
Further Reading
Bibliography
Auditory Models
Introduction
Auditory Periphery
The Meddis Model of the Auditory Periphery
Outer and Middle Ear
Basilar Membrane
Inner Hair Cells
Auditory Nerve Synapse
Auditory Nerve Activity
Pitch Estimation Using Auditory Models
Autocorrelation Models
Pitch Extraction in the Brain
Further Reading
Bibliography
Digital Representation of Music
Introduction
From Sheet to File
Optical Music Recognition
abc Music Notation
Musical Instrument Digital Interface
MusicXML 3.0
From Signal to File
Pulse Code Modulation and Raw Audio Format
WAVE File Format
MP3 Compression
From File to Sheet
MusicTeX Typesetting
Transcription Tools
From File to Signal
Further Reading
Bibliography
Music Data: Beyond the Signal Level
Introduction
From the Signal Level to Semantic Features
Types of Semantic Features
Deriving Semantic Features
Discussion
Symbolic Features
Music Scores
Social Web
Social Tags
Shared Playlists
Listening Activity
Music Databases
Concluding Remarks
Bibliography
II Methods
Statistical Methods
Introduction
Probability
Theory
Empirical Analogues
Random Variables
Theory
Empirical Analogues
Characterization of Random Variables
Theory
Empirical Analogues
Important Univariate Distributions
Random Vectors
Theory
Empirical Analogues
Estimators of Unknown Parameters and their Properties
Testing Hypotheses on Unknown Parameters
Modeling of the Relationship between Variables
Regression
Time Series Models
Towards Smaller and Easier to Handle Models
Further Reading
Bibliography
Optimization
Introduction
Basic Concepts
Single-Objective Problems
Binary Feasible Sets
Continuous Feasible Sets
Compound Feasible Sets
Multi-Objective Problems
Further Reading
Bibliography
Unsupervised Learning
Introduction
Distance Measures and Cluster Distinction
Agglomerative Hierarchical Clustering
Agglomerative Hierarchical Methods
Ward Method
Visualization
Partition Methods
k-Means Methods
Self-Organizing Maps
Clustering Features
Independent Component Analysis
Further Reading
Bibliography
Supervised Classification
Introduction
Supervised Learning and Classification
Targets of Classification
Selected Classification Methods
Bayes and Approximate Bayes Methods
Nearest Neighbor Prediction
Decision Trees
Support Vector Machines
Ensemble Methods: Bagging
Neural Networks
Interpretation of Classification Results
Further Reading
Bibliography
Evaluation
Introduction
Resampling
Resampling Methods
Hold-Out
Cross-Validation
Bootstrap
Subsampling
Properties and Recommendations
Evaluation Measures
Loss Based Performance
Confusion Matrix
Common Performance Measures Based on the Confusion Matrix
Measures for Imbalanced Sets
Evaluation of Aggregated Predictions
Measures Beyond Classification Performance
Hyperparameter Tuning: Nested Resampling
Tests for Comparing Classifiers
McNemar Test
Pairwise t-Test Based on B Independent Test Data Sets
Comparison of Many Classifiers
Multi-Objective Evaluation
Further Reading
Bibliography
Feature Processing
Introduction
Preprocessing
Transforms of Feature Domains
Normalization
Missing Values
Harmonization of the Feature Matrix
Processing of Feature Dimension
Processing of Time Dimension
Sampling and Order-Independent Statistics
Order-Dependent Statistics Based on Time Series Analysis
Frame Selection Based on Musical Structure
Automatic Feature Construction
A Note on the Evaluation of Feature Processing
Further Reading
Bibliography
Feature Selection
Introduction
Definitions
The Scope of Feature Selection
Design Steps and Categorization of Methods
Ways to Measure Relevance of Features
Correlation-Based Relevance
Comparison of Feature Distributions
Relevance Derived from Information Theory
Examples for Feature Selection Algorithms
Relief
Floating Search
Evolutionary Search
Multi-Objective Feature Selection
Further Reading
Bibliography
III Applications
Segmentation
Introduction
Onset Detection
Definition
Detection Strategies
Goodness of Onset Detection
Tone phases
Reasons for Clustering
The Clustering Process
Refining the Clustering Process
Musical Structure Analysis
Concluding Remarks
Further Reading
Bibliography
Transcription
Introduction
Data
Musical Challenges: Partials, Vibrato, and Noise
Statistical Challenge: Piecewise Local Stationarity
Transcription Scheme
Separation of the Relevant Part of Music
Estimation of Fundamental Frequency
Classification of Notes, Silence, and Noise
Estimation of Relative Length of Notes and Meter
Estimation of the Key
Final Transcription into Sheet Music
Software
Concluding Remarks
Further Reading
Bibliography
Instrument Recognition
Introduction
Types of Instrument Recognition
Taxonomy Design
Example of Instrument Recognition
Labeled Data
Taxonomy Design
Feature Extraction and Processing
Feature Selection and Supervised Classification
Evaluation
Summary of Example
Concluding Remarks
Further Reading
Bibliography
Chord Recognition
Introduction
Chord Dictionary
Chroma or Pitch Class Profile Extraction
Computation Using the Short-Time-Fourier-Transform
Computation Using the Constant-Q-Transform
Influence of Timbre on the Chroma/PCP
Chord Representation
Knowledge-driven Approach
Data-driven Approach
Frame-based System for Chord Recognition
Knowledge-driven Approach
Data-driven Approach
Chord Fragmentation
Hidden Markov Model-based System for Chord Recognition
Knowledge-driven Transition Probabilities
Data-driven Transition Probabilities
Joint Chord and Key Recognition
Key-Only Recognition
Joint Chord and Key Recognition
Evaluating the Performances of Chord and Key Estimation
Evaluating Segmentation Quality
Evaluating Labeling Quality
Concluding Remarks
Further Reading
Alternative Audio Signal Representations
Alternative Representations of the Chord Labels
Taking into Account other Musical Concepts
Bibliography
Tempo Estimation
Introduction
Definitions
Beat
Tempo
Metrical Levels
Automatic Rhythm Estimation
Overall Scheme of Tempo Estimation
Feature List Creation
Tempo Induction
Evaluation of Tempo Estimation
A Simple Tempo Estimation System
Applications of Automatic Rhythm Estimation
Concluding Remarks
Further Reading
Bibliography
Emotions
Introduction
What are Emotions?
Difference between Basic Emotions, Moods, and Emotional Episodes
Personality Differences and Emotion Perception
Theories of Emotions and Models
Hevner Clusters of Affective Terms
Semantic Differential
Schubert Clusters
Circumplex Word Mapping by Russell
Watson-Tellegen Diagram
Speech and Emotion
Music and Emotion
Basic Emotions
Moods and Other Affective States
Factors of Influence and Features
Harmony and Pitch
Melody
Instrumentation and Timbre
Dynamics
Tempo and Rhythm
Lyrics, Genres, and Social Data
Examples: Individual Comparison of Features
Computationally Based Emotion Recognition
A Note on Feature Processing
Future Challenges
Concluding Remarks
Further Reading
Bibliography
Similarity-based Organization of Music Collections
Introduction
Learning a Music Similarity Measure
Formalizing an Adaptable Model of Music Similarity
Modeling Preferences through Distance Constraints
Dealing with Inconsistent Constraint Sets
Learning Distance Facet Weights
Visualization: Dealing with Projection Errors
Popular Projection Techniques
Common and Unavoidable Projection Errors
Static Visualization of Local Projection Properties
Dynamic Visualization of "Wormholes"
Combined Visualization of Different Structural Views
Dealing with Changes in the Collection
Incremental Structuring Techniques
Aligned Projections
Concluding Remarks
Further Reading
Bibliography
Music Recommendation
Introduction
Common Recommendation Techniques
Collaborative Filtering
Content-based Recommendation
Further Knowledge Sources and Hybridization
Specific Aspects of Music Recommendation
Evaluating Recommender Systems
Laboratory Studies
Offline Evaluation and Accuracy Metrics
Beyond Accuracy – Additional Quality Factors
Current Topics and Outlook
Context-Aware Recommendation
Incorporating Social Web information
Playlist Generation
Concluding Remarks
Further Reading
Bibliography
Automatic Composition
Introduction
Composition
What Composers Do
Why Automatic Composition?
A Short History of Automatic Composition
Principles of Automatic Composition
Basic Methods
Advanced Methods
Evaluation of Automatically Composed Music
Concluding Remarks
Further Reading
Bibliography
IV Implementation
Implementation Architectures
Introduction
Architecture Variants and their Evaluation
Personal Player Device Processing
Network Server Based Processing
Distributed Architectures
Applications
Music Recommendation
Music Recognition
Novel Applications and Future Development
Concluding Remarks
Further Reading
Bibliography
User Interaction
Introduction
User Input for Music Applications
Haptic Input
Audio Input
Visual and Other Sensor Input
Multi-Modal Input
Coordination of Inputs from Multiple Users
User Interface Output for Music Applications
Audio Presentation
Visual Presentation
Haptic Presentation
Multi-Modal Presentation
Factors Supporting the Interpretation of User Input
Role of Context in Music Interaction
Impact of Implementation Architectures
Influence of Social Interaction and Machine Learning
Concluding Remarks
Bibliography
Hardware Architectures for Music Classification
Introduction
Evaluation Metrics for Hardware Architectures
Cost Factors
Combined Cost Metrics
Specific Methods for Feature Extraction for Hardware Utilization
Architectures for Digital Signal Processing
General Purpose Processor
Graphics Processing Unit
Digital Signal Processor
Application Specific Instruction Set Processor
Dedicated Hardware
Design Space Exploration
Concluding Remarks
Further Reading
Bibliography
Index
