Model descriptions#
AlphaCSC is a library to perform shift-invariant sparse dictionary learning, also known as convolutional sparse coding (CSC), on time-series data. It includes a number of different models:
univariate CSC
multivariate CSC
multivariate CSC with a rank-1 constraint 1
univariate CSC with an alpha-stable distribution 2
1. Univariate CSC#
Framed mathematically, if we are giving a signal \(x^n \in \mathbb{R}^T\), we want to learn \(k\) atoms \(d^k \in \mathbb{R}^{L}\) and their associated activations \(z_k^n \in \mathbb{R}^{T - L + 1}\). The optimization problem boils down to minimizing an \(\ell_2\) reconstruction loss, which corresponds to a Gaussian noise model, with a sparsity-inducing \(\ell_1\) penalty term:
subject to \(z_k^n \ge 0\) and \(||d_k||_2 \le 1\). The shift invariance is encoded by the convolution operator \(*\) which is why these methods are called convolutional sparse coding.
2. Multivariate CSC#
In the multivariate case, we are giving a multivariate signal \(X^n \in \mathbb{R}^{T \times P}\), where \(P\) is the dimension of the signal, and we want to learn \(k\) multivariate atoms \(D_k \in \mathbb{R}^{L \times P}\) and their associated activations \(z_k^n \in \mathbb{R}^{T - L + 1}\). The objective function reads:
subject to \(z_k^n \ge 0\) and \(||D_k||_2 \le 1\).
3. Multivariate CSC with a rank-1 constraint#
A variant of the multivariate CSC model considers an additional constraint on the multivariate atoms, stating that they need to be rank-1. In other words, each atom can be written as a product of univariate vectors \(D_k = u_k v_k^\top\), where \(u_k^\top \in \mathbb{R}^{P}\) is the pattern over channels, and \(v_k^\top \in \mathbb{R}^{L}\) is the pattern over time. The objective function reads:
subject to \(z_k^n \ge 0\), \(||u_k||_2 \le 1\), and \(||v_k||_2 \le 1\). This rank-1 formulation is particularly meaningful in the case of magnetoencephalography (MEG), due to the physical properties of electromagnetic waves, which propagate instantaneously and add up linearly. It also brings robustness to the atom estimates, since less parameters are estimated.
This model is described in details in the original NIPS paper 1.
4. Univariate CSC with an alpha-stable distribution#
The name alphaCSC originally referred to a particular model which uses an
alpha-stable distribution
for the noise, instead of the more classical Gaussian distribution. Note that
the alphaCSC library is not limited to this particular model.
This model leads to a weighted formulation of univariate CSC, the weights being used to downweight noisy portions of the data:
subject to \(z_k^n \ge 0\) and \(||d_k||_2 \le 1\), and where \(w_n\) are the weights which are learned using an EM algorithm.
This model is described in details in the original NIPS paper 2.
Cite our work#
If you use this code in your project, please consider citing our work:
- 1(1,2)
Dupré La Tour, T., Moreau, T., Jas, M., & Gramfort, A. (2018). Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals. Advances in Neural Information Processing Systems (NIPS).
- 2(1,2)
Jas, M., Dupré La Tour, T., Şimşekli, U., & Gramfort, A. (2017). Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding. Advances in Neural Information Processing Systems (NIPS), pages 1099–1108.