Did AI Just End Human-Made Music? (Cold Fusion ft. Rick Beato, 04/30/2024)

                 So it begins ... Is this a real band or AI? (Rick Beato, 06/30/2025)
 

Methods used in AI Music Generation (source)
 

Typical Flowchart for AI Music Genre Classification (source)

SYMBOLIC AI  (1950s - 1970s)

Symbolic AI is based on the idea that intelligence can be represented using symbols and explicit rules.
It involves creating detailed models of knowledge and logic to process information using predefined rules, "if-then-else" decision trees, and knowledge representations assembled in Semantic Networks [ i.e. graphical representations of knowledge, where nodes represent concepts or objects, and links between nodes represent their relationships ].

In the 1950s, computer scientists John McCarthy, Allen Newell, and others created foundational symbolic AI systems such as the Logic Theorist, the General Problem Solver, and LISP.
The 1970s saw the development of expert systems [ i.e. rule-based programs designed to mimic human specialists ], such as MYCIN and DENDRAL.
 

Key Concepts

• Knowledge Representation: Encoding real-world knowledge into symbols that a machine can process (e.g. logical propositions, frames, or rules), closely related to the previously discussed concepts of schematization, chucking, and pattern recognition.

• Reasoning: Using deductive (i.e. moving from general to specific) or inductive (i.e. moving from specific to general) reasoning to draw conclusions from existing data
  [ see deductive vs. inductive vs. abductive reasoning ].

• Expert Systems: Creating specialized symbolic AI systems informed by thematic experts, to emulate human expertise in specific domains.
   

Challenges

• Scalability: Handling the complexity of real-world situations with explicit rules becomes very difficult as the number of options and rules increases (combinatorial explosion problem, exemplified in the tower of hanoi puzzle).

• Knowledge Acquisition: Building comprehensive knowledge bases is time-consuming, ongoing, and requires human experts. Knowledge elicitation problem: the challenge of extracting knowledge from an expert and doing so in a way that it can be converted into a machine readable format.

• Inflexibility: Symbolic systems struggle to generalize beyond the predefined rules and are less adaptable to new or unforeseen situations.
   

Music Applications (limited scope)

• Music can be represented symbolically as a sequence of notes, chords, rhythms, and other musical elements. This can be accomplished via MIDI (Musical Instrument Digital Interface) or sheet music notation / MusicXML, which encode musical information (e.g. pitch, duration, velocity/dynamics), and sound source generator (i.e. instrument type or timbre).

• Digital format audio signal representations (e.g. WAV, MP3) are analyzed using multi-channel Fourier transform techniques to extract key perceptual features (beat, pitch, loudness, timbre, spatial spread).

• Systems recommending new music based on music you like are supported by probabilistic (inductive) reasoning algorithms.

• Instrument selection based on a set of timbre specifications (e.g. richness, nasality, roughness) is supported by deductive reasoning algorithms.

  

PROBABILISTIC MODELS & BAYESIAN INFERENCE (1970s-1990s)

Probabilistic models are able to represent uncertainty and make decisions in situations where information is incomplete or noisy. Since the mid 1980s, they've been used in conjunction with Bayesian networks, which provide a formal framework for updating results as new data becomes available.
 

Key Concepts

• Bayesian Networks: Graphical models that represent the probabilistic relationships among variables using directed acyclic graphs or DAGs (i.e. graphs that link nodes with unidirectional arrows without forming closed loops) to form cognitive models of propositional reasoning under uncertainty [ e.g. Pearl & Russell, 2004 ].
  Bayesian networks model probability-based causal relationships and handle noise and partial information using local, distributed algorithms for inference & learning.

• Hidden Markov Models (HMMs): Statistical models used to link time-series data that are assumed to follow a Markov process (reminder), where a given state is only influenced by the one preceding it, but with hidden states (i.e. where the preceding state is unknown/hidden and can only be inferred by its effect on the given state).

• Markov Decision Processes (MDPs): Mathematical frameworks used to model decision-making problems where outcomes are partially random and partially controllable, using rewards to produce optimal decisions within a dynamic system. For example: a robot navigating a room has several movement options and the goal to reach a specific target, while avoiding obstacles, with each movement providing a reward or penalty depending on its relationship to the goal.
   

Challenges:

• Complexity: Probabilistic models can become computationally expensive as the number of variables increases, particularly in high-dimensional spaces (combinatorial explosion).

• Data Sensitivity: These models are sensitive to the quality and quantity of data available for estimating the underlying probability distributions.

• Interpretability: Understanding the decisions made by the network can be challenging, necessitating further research into explainable AI techniques. Common challenge to all non-symbolic AI processes.
   

Music Applications

• Analysis of existing compositions to identify patterns supports modeling of relationships among musical elements into Bayesian networks, which, in turn, help composers generate new pieces that adhere to specific styles or genres.

• Bayesian networks can adapt to the evolving context of a performance in real time, allowing for dynamic improvisation.

• Bayesian networks support automatic music generation models e.g.:
       _ Fang, C. et al. (2022). "Automatic Music Creation Based on Bayesian Networks."
       _ Jing, E. et al. (2024). "Emotion-aware Personalized Music Recommendation with a Heterogeneity-aware Deep Bayesian Network."
       _ Marcos et al., (2025). "Random rotational embedding Bayesian optimization for human-in-the-loop personalized music generation."

 

Unsupervised Learning Clustering for Spotify Playlist Generation (source; code)

Visualization of k-means clustering (Johan Hagelbäck, 2018)

Sample Convolutional Neural Network (source)

Recurrent Neural Network (source)

Sample Generative Adversarial Network (source)

ARTIFICIAL NEURAL NETWORKS & MACHINE LEARNING (ML) (1950s - 1990s)

Artificial Neural Networks consist of interconnected nodes (neurons) that process information. Early (1950s) work includes Frank Rosenblatt's perceptron, a single-layer neural network that takes one or more inputs, processes them, and produces a single output. Perceptrons are used for supervised learning of binary classifiers. They learn from examples and adjust their parameters to improve their accuracy in classifying new examples.

Machine Learning focuses on neural networks that learn from data and improve over time without explicit programming.. This approach has become the dominant paradigm in AI development due to its flexibility and ability to scale with large datasets. Machine learning algorithm types include:
    a) decision trees: multi-option "if-then-else" node sets (1980s)
    b) nearest neighbors: probable outcome choices based on context-dependent proximity (1980s).
 

Key Concepts

• Supervised Learning: Models are trained on labeled data -where the correct output per input is given- and learns to predict outputs for new, unseen inputs.

• Unsupervised Learning: Models are given unlabeled data and must find patterns or structures within it.
   

Challenges

• Data Dependency: ML algorithms require large amounts of high-quality data for training, which may be scarce.

• Overfitting & Underfitting: ML models may either fit the training data too closely (overfitting) or fail to capture important patterns (underfitting), leading to poor generalization on unseen/new data.
   

Music Applications

• AI models can be trained on labeled datasets of music to learn mappings between input features and output targets. For example, a supervised learning model might be trained to classify music genres, recognize chords and predict chord progressions, or identify instrument types.

• Clustering and dimensionality reduction techniques (e.g., k-means*; PCA**) can be used to identify the underlying structure of music, without labeled data, and perform
     _stylistic similarity classification of musical pieces based on shared sonic and/or structural features;
     _timbral similarity classification of sources based on shared average and time-variant spectral
       features.

[ * k-means: a machine learning algorithm that groups unlabeled data-points into clusters based on some emergent features. The algorithm randomly selects a few central points, called centroids (not to be confused with spectral centroid), and assigns each data point to the nearest centroid. It then recalculates the centroids based on the new groupings and repeats the process until the clusters are "sensible" (e.g. according to gestalt principles). ]
[ ** PCA: (Principal Component Analysis) a dimensionality reduction technique that transforms a large dataset with many variables into a smaller set of variables, called principal components, while still preserving most of the important information from the original data. It is a linear transformation method that reduces the number of dimensions, while (ideally) capturing most of the original variance in the new, lower-dimensional space. ]

• For machine learning models, music is often represented through feature vectors or embeddings (i.e. mathematical expressions of musical features) that capture important attributes such as tempo, beat, key, pitch chroma, and mel-frequency cepstral coefficients or MFCCs (i.e. numerical representations of spectral envelope periodicities using a log-frequency scale).
   

DEEP LEARNING / REINFORCEMENT LEARNING (2000s -)

Deep Learning (DL) is a subset of machine learning that uses multi-layered artificial neural networks to model complex patterns in large datasets. It has been particularly successful in tasks such as image recognition, speech recognition, and natural language processing.
Deep learning became a dominant field in the 2000s, with breakthroughs like the use of deep convolutional neural networks (CNN) for image recognition [ e.g. AlexNet by Alex Krizhevsky et al., 2012 - review ] and with the rise of frameworks that support CNN building [ e.g. TensorFlow and PyTorch ].

Reinforcement Learning (RL) is a deep learning paradigm where an agent* learns to make decisions by interacting with an environment in order to maximize a cumulative reward over time. Rooted in behaviorism, RL's early developments included
_ 1950's dynamic programming [ i.e. solving complex problems in stages; e.g. Bellman, 1954 ];
_ 1960's Monte Carlo methods [ i.e. evaluation of blind solutions on a random sample; e.g. Ulam, 1946 - summary ];
_ 1970's temporal difference learning [ i.e. combination of the previous two with the inclusion of weighted value
    functions; e.g. Sutton, 1988 ].

Recent developments include deep-learning-based tools by Deep Mind, OpenAI, Anthropic, and others.

[ *Agent: a multi-layered neural network that interacts with its environment by taking actions based on observations, receiving rewards, and learning to optimize its behavior over time to achieve a specific goal. ]
 

Key Concepts

• Multi-layered Neural Networks: Layered architecture of artificial neurons that enables the network to learn hierarchical representations of data.

• Convolutional Neural Networks (CNNs): Specialized multi-layered neural networks used primarily to detect patterns in images, via layered convolution operations. A convolution operation is a mathematical process that filters spatially distributed data to extract local features.
  A CNN performs this process repeatedly, rectifying [ i.e. removing negative values from ] and pooling
  [ i.e. reducing complexity by selecting the maximum or average value within a pre-specified range ] the results at every step.

• Recurrent Neural Networks (RNNs): Specialized multi-layered neural networks used primarily to detect patterns in sequential data --such as time series, natural language, or a piece of music-- and can maintain a form of memory through backpropagation* feedback loops. An RNN feeds the output from the previous step as input to the current step, allowing it to capture temporal dependencies within a sequence. RNNs include hidden layers, 'short-term memory' components that allow the network to understand a current element in context. These components support a special kind of memory called Long (i.e. lasting) Short-Term (i.e. related to immediate neighbors in time) Memory, or LSTM.

• *Backpropagation: Iterative adjustment of a probabilistic model's decision weights to reduce the difference between predicted and actual outputs [ Hinton in Lin, 2024;  LeCun, 1988 ].

• Generative Models: NNs that output new data resembling but not identical to the input data. e.g.:
     a) Generative Adversarial Networks (GANs). Two neural networks compete during training:
         a "generator" that creates new data, and a "discriminator" that tries to distinguish between
         real data and the generated fake data, essentially creating a "game" where each network
         is trying to outsmart the other.
     b) Variational Autoencoders (VAEs). Two neural networks, where one encodes input data into a
         probability distribution within a latent space*, and another decodes a portion of the encoded
         data space to generate new data samples that resemble the input data it was trained on.
         A VAE essentially learns the underlying patterns of input data and can create variations
         based on that understanding. [ *Latent Space: a compressed, abstract map of complex data that
         highlights its key features and underlying patterns. ]

• Large Language Model (LLM): Any generative AI process that requires the manipulation and creation of natural language based on training on large amounts of text data [ e.g. ChatGPT by OpenAI ].
  LLMs are built on machine learning that uses transformer models**.

• **Transformer Model: a special type of neural network architecture fit to understand context and relationships within sequential data, like words in a sentence. It employs a "self-attention" mechanism to focus on the most relevant elements of an input sequence and can parallel process multiple sequences. It learns "meaning" by tracking how different parts of the data relate to each other and is particularly powerful for natural language processing tasks.

• Reinforcement Learning (RL): A paradigm that involves learning from trial and error, with an agent receiving feedback in the form of rewards or penalties for its actions. This feedback enables the agent to improve its policy (i.e. its mapping of states or observations to actions), maximize long-term rewards, and facilitate decision-making in dynamic and uncertain environments.
     RL involves 3 stages:
     a) Initialization: The agent starts with little or no knowledge about the environment. Its policy is
         often random, with value functions, Q (i.e. action weights, rewards, and punishments) that
         are initialized arbitrarily.
     b) Interaction with the Environment: The agent takes actions based on its current policy, interacts
         with the environment, and receives feedback (rewards) and new states.
     c) Knowledge Update: Based on the feedback received, the agent updates its policy and value
         function settings to improve future decision-making.
   

Challenges

• Computational Cost: Deep learning requires a large number of interactions with the environment to learn effective policies. It relies on specialized hardware and is computationally expensive and time-consuming.

• Data Requirements: Deep learning models typically need vast amounts of labeled data to train effectively, which might not be available for all applications.

• Interpretability: Deep learning models lack transparency (i.e. they do not support understanding of why a particular decision was made) and act as "black boxes".

• Exploration-Exploitation Dilemma: How to best balance between exploring new actions and exploiting known ones.

• Reward Shaping: Designing a suitable reward function that only leads to intended behaviors.

• Human Validation of Machine-Learning Data: Data validation and deep learning guidance continues to benefit from and require human intervention. This is provided in the form of “Human Intelligence Tasks” or HITs, work that involves dubious labor practices and employees in low income countries.

One of the largest platforms for such employment is Amazon's Mechanical Turk (MTurk), described as "...a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually. This could include anything from conducting simple data validation and research to more subjective tasks like survey participation, content moderation, and more." 

Critics have likened MTurk to a "digital sweatshop," where knowledge is produced by a class of exploited and unprotected laborers, a largely invisible workforce whose labor is actually powering what we call “artificial” intelligence. Some of this work is performed by the general public interacting daily with the ubiquitously placed AI tools, training them with our digital actions. However, most of the work is done by a globally dispersed army of data workers who are paid poorly and have no recourse if they’re mistreated.
 

Music Applications

Most successful music AI tools use deep and reinforcement learning methodologies.
Convolutional Neural Networks, for example, have been successfully used since the 2010s for stem separation. This and other reinforcement learning methodologies support:
Music genre classification; chord recognition/classification; theme variation generation; automatic orchestration/harmonization; music generation by style; generation of musical accompaniment to a vocal line; text-to-music audio signal generation; and much more.
 

Sample Generative Genetic Algorithm Flowchart for Music Generation (source)

EVOLUTIONARY ALGORITHMS & GENETIC PROGRAMMING (1960s - 1990s)

Evolutionary Algorithm (EA) and Genetic Programming (GP) AI methods that have been inspired by the processes of natural selection and genetic evolution. They are used to optimize solutions to complex problems by iteratively and in parallel evolving a population of candidate solutions. The concept was introduced by John Holland in the 1960s and was developed and popularized by John Koza in the 1990s for solving complex optimization problems.
EA and GP methods are complementary to deep and reinforcement learning, providing best-fit mathematical formulas to data and exploring complex search spaces where gradient-based methods* struggle.
           [ *Gradient-based methods: DL and RL optimization algorithms that use the gradient (i.e. the direction of the
              steepest ascent or descent) of a function to iteratively zero in on its minimum or maximum value. ]

Key Concepts

• Evolutionary Algorithms (EAs): Algorithms that simulate the process of natural selection by evolving a population of candidate solutions through operations such as crossover (recombination), mutation, and selection.

• Genetic Programming (GP): A specific form of evolutionary computation where tree-structured programs "evolve" during a problem-solving process.

Challenges

• Convergence Speed/Cost: Evolutionary algorithms are computationally expensive and take a long time to converge on an optimal solution.

• Premature Convergence: If the population loses diversity too early, the algorithm may get stuck in local optima and fail to find the global optimum.

• Scalability: For very large-scale problems, EAs and GP do not scale efficiently enough to complement deep learning or other methods.

Music Applications

• EAs are used to evolve musical compositions by treating musical pieces as "genes" in a population, where each gene represents a particular feature (e.g. melody, harmony, rhythm).
  The population evolves over generations (i.e. iterative crossovers/mutations/selections) to optimize the resulting piece of music based on a user-defined fitness function (i.e. operational definitions of target harmony, emotional tone, degree of complexity, or even composer style).

• GP is a building block of EAs. It conceives musical compositions as tree-like structures, where each node in the tree represents a musical element (e.g. pitch, rhythm, or dynamics).
  Musical operations (e.g. note generation, rhythm patterns, harmonic progressions) are encoded as functions and are fed to EAs that evolve to create novel pieces of music.
  [ e.g. Kilb & Ellis, 2024; Campobello et al., 2020; Farzaneh & Toroghi, 2020. ]

• EAs and GP can support musical applications of all other machine learning methods.

  

COGNITIVE & COMPUTATIONAL NEUROSCIENCE APPROACHES (1980s - )

In addition to providing the foundation knowledge for all AI modeling approaches, principles of neuroscience and cognitive science can directly drive the design of intelligent systems that aim to replicate human thinking. Cognitive modeling and the simulation of human cognition in AI began in the 1980s, with architectures that continue to evolve
[ e.g. Carnegie Mellon University's ACT-R (Adaptive Control of Thought-Rational) and Soar (State, Operator, and Result ].
Spiking Neural Networks, mimicking the brain’s neuron spiking activity, were developed in the 2000s and led to neuromorphic computing, which attempts to fully model the structure and function of the brain. [ See Aghav-Palwe, S. & Gunjal, A. (2021). "Introduction to cognitive computing and its various applications." ].

Key Concepts

• Cognitive Modeling: Computational models that simulate human mental processes (e.g. memory, learning, attention, perception, interpretation, and decision making).

• Brain-Inspired Computing: AI system designs that mimic the brain's structure and functioning.

Challenges

• Evasiveness of Cognitive Processes: We attempt to understand the cognitive processes related to musical experience primarily via empirical studies that rely on listener responses. The vague and multidimensional nature of the study results impede computational replication of the underlying processes.

• Complexity of the Brain: Validly and accurately modeling the complex electrochemical processes of the human brain is an ongoing challenge.

Music Applications

• Brain-to-Music Interfaces: Brain-to-Music research explores neural decoding methods to help AI systems interpret brain activity (e.g. via EEG and/or fMRI) and convert it into musical compositions or modifications. In more advanced, future scenarios, brain-computer interfaces may help users generate music directly from their brain activity, potentially offering new ways to create music for people with disabilities [ read about a sample attempt ], as well as to automate emotion recognition through analysis of neural and electrophysiological responses to music [ e.g. Su et al., 2024 ].

• Source Separation (Stem Splitting): Using models inspired by the brain’s ability to perform auditory scene analysis, AI systems can separate individual audio sources (e.g., vocals, drums, guitar) from a mixed track.

  When paired with symbolic AI methods, cognitive & computational neuroscience approaches can automatically transcribe polyphonic music.
  Automatic music transcription (ATM) has had a long history, ending up being considered an impossible task by all involved, up until the early 2010s and the digital technology advancements that allowed AI to come to its own.
  From Miller's Phonodeik (1910s), and Metfessel's Phonophotographer (1920s), to Seeger's Melograph (1950s), Moorer's efforts at IRCAM (1970s), and MIDI advancements (1980s-90s), no automatic method had been able to successfully transcribe polyphonic music.
  As is the case with several other musical "impossibilities" (e.g. automatic composition, recording, mixing & mastering; stylistic classification; stem splitting; mood recognition; etc.) AI can soon claim success in ATM as well.
  [ E.g. Holzapfel et al., 2022 & Liu & Benetos, 2021. See also the full bibliography of Emmanouil Benetos, a top researcher in AI and music ].

 

HYBRID & NEURO-SYMBOLIC APPROACHES (2000s - )

Given the limitations of purely symbolic or purely statistical methods, latest research focuses on hybrid AI systems that combine the strengths of both approaches.
Rule-based reasoning is combined with learning-based techniques, enabling the use of organized logic that is able to adapt to new information and changing circumstances. This has led many to dub Hybrid AI "the next big thing in tech" [ e.g. Marr, 2024; Velante, 2025 ].
 

Key Concepts

• Neuro-Symbolic Systems:  Neuro-Symbolic AI integrates the reasoning and logical inference capabilities of symbolic AI with the data-driven and pattern recognition power of machine learning and neural networks, on a pathway towards artificial general intelligence.

• Cognitive Architectures: Computational models designed to simulate human-like cognition combine elements of symbolic reasoning, machine learning, and problem-solving. Examples include:
     a) CLARION (Connectionist Learning with Adaptive Rule Induction On-line): It is formed by integrating connectionist, reinforcement, and symbolic
         learning methods, and performs real-time reactive learning by adopting a framework of layered implicit and explicit cognitive processes.
     b) LIDA (Learning Intelligent Distribution Agent): It is structured around a central "attention" mechanism, which combines action selection, motivation
         via emotions, and three types of learning (multimodal, rules-based, and inferential).

Challenges

• Integration: Combining symbolic reasoning and machine learning in a way that leverages the strengths of both while minimizing their weaknesses and enhances transparency/explainability and trust remains an open challenge.

• Scalability: Integrating these approaches can lead to scalability problems when working with large, multimodal datasets and real-time processing contexts.

Music Applications

Hybrid AI models support:
   _ enhanced expressiveness (richer palette of sounds and styles) and automatic affective music (i.e. music designed for specific emotional
      impact) generation [ e.g. Dash & Agres, 2024 ];
   _ improved symbolic representation of structural elements, helping compositions adhere to musical norms and conventions;
   _ generation of a wider variety of musical genres with applications in film scoring, game music, choral music and personalized playlist generation
      [ e.g. Chen et al., 2024; Szelogowski, 2024 ].
   _ text-to-music generation models [ e.g. Denk et al., 2025 ].