What is inside the black box of AI? Paradigms for math operators

J. Philippe Blankert, 4 May 2025

1. Matrices and Tensors

Matrices (2-dimensional tensors) and higher-dimensional tensors are the primary backbone of deep learning.

Use-cases:
- Linear transformations, convolutions, pooling.
- Attention mechanisms.
- Embedding layers (word embedding, position embedding).
- Tensor decomposition and factorization.
- Tensor contractions and Einstein summation (einsum operations).
Operators involved:
- Multiplication, dot products, Hadamard products, convolutions, tensor operations, slicing, and reshaping.

AI operators acting through nodes typically appear in graph-based representations such as Graph Neural Networks (GNNs).

Use-cases:
- Graph convolutions.
- Node aggregation operators (mean, sum, max pooling).
- Attention-based node relationships.
- Graph diffusion processes (operators based on spectral graph theory).
Operators involved:
- Message-passing operations, Laplacian operations, diffusion operators, attention operators defined over nodes and edges.

Symbolic AI uses operators over symbolic or logical expressions, rules, and predicates.

Use-cases:
- Logic programming.
- Symbolic reasoning.
- Automated theorem proving.
- Expert systems.
Operators involved:
- Logical connectives (AND, OR, NOT, XOR).
- Quantifiers (existential ∃, universal ∀).
- Inference operators (modus ponens, resolution).

AI increasingly utilizes differentiable functions and programming constructs as first-class operators.

Use-cases:
- Differentiable programming.
- Automatic differentiation.
- Neural architecture search.
- Hypernetwork generation (networks generating other networks).
Operators involved:
- Higher-order differentiation (grad, Jacobian, Hessian).
- Composable differentiable modules/functions.
- Functional composition, mapping, filtering, and folding.

Probabilistic operators work with distributions and random variables, allowing stochastic AI methods.

Use-cases:
- Bayesian networks.
- Variational autoencoders (VAEs).
- Generative models.
- Reinforcement learning policies and value estimations.
Operators involved:
- Sampling (sample), marginalization (integrate, sum), expectation (E[x]), KL-divergence, entropy, conditional probability (P(A|B)).

Quantum computing principles inspire new operators applied in quantum AI algorithms.

Use-cases:
- Quantum machine learning algorithms.
- Quantum embedding of data.
- Quantum neural networks.
Operators involved:
- Quantum gates (Hadamard, CNOT).
- Quantum tensor product (⊗), quantum measurements.
- Quantum amplitude encoding.

Set-based and algebraic operations underpin reasoning systems, clustering, and relational representations.

Use-cases:
- Fuzzy logic systems.
- Concept lattices.
- Relational learning and databases.
Operators involved:
- Set operators (union ∪, intersection ∩, difference \).
- Algebraic closures, lattices, equivalence classes.

Operators acting on geometric spaces enable geometric deep learning.

Use-cases:
- Point cloud processing.
- Manifold learning.
- Shape analysis.
Operators involved:
- Rotations, translations, projections.
- Geodesic computations.
- Laplace-Beltrami operators.

Operators handling temporal dynamics and sequential dependencies.

Use-cases:
- Recurrent neural networks (RNN, LSTM, GRU).
- Continuous-time models (Neural ODEs).
- Time-series forecasting.
Operators involved:
- Differential equations.
- Integral operators.
- Discrete temporal updates (state(t+1) = operator(state(t))).