Model descriptions

AlphaCSC is a library to perform shift-invariant sparse dictionary learning, also known as convolutional sparse coding (CSC), on time-series data. It includes a number of different models:

  1. univariate CSC
  2. multivariate CSC
  3. multivariate CSC with a rank-1 constraint [1]
  4. univariate CSC with an alpha-stable distribution [2]

1. Univariate CSC

Framed mathematically, if we are giving a signal \(x^n \in \mathbb{R}^T\), we want to learn \(k\) atoms \(d^k \in \mathbb{R}^{L}\) and their associated activations \(z_k^n \in \mathbb{R}^{T - L + 1}\). The optimization problem boils down to minimizing an \(\ell_2\) reconstruction loss, which corresponds to a Gaussian noise model, with a sparsity-inducing \(\ell_1\) penalty term:

\[\min_{d,z} \sum_n \left(\|x^n - \sum_k d^k * z_k^n \|_2^2 + \lambda \sum_k z_k^n \right)\]

subject to \(z_k^n \ge 0\) and \(||d_k||_2 \le 1\). The shift invariance is encoded by the convolution operator \(*\) which is why these methods are called convolutional sparse coding.

2. Multivariate CSC

In the multivariate case, we are giving a multivariate signal \(X^n \in \mathbb{R}^{T \times P}\), where \(P\) is the dimension of the signal, and we want to learn \(k\) multivariate atoms \(D_k \in \mathbb{R}^{L \times P}\) and their associated activations \(z_k^n \in \mathbb{R}^{T - L + 1}\). The objective function reads:

\[\min_{D,z} \sum_n \left(\|X^n - \sum_k D_k * z_k^n \|_2^2 + \lambda \sum_k z_k^n \right)\]

subject to \(z_k^n \ge 0\) and \(||D_k||_2 \le 1\).

3. Multivariate CSC with a rank-1 constraint

A variant of the multivariate CSC model considers an additional constraint on the multivariate atoms, stating that they need to be rank-1. In other words, each atom can be written as a product of univariate vectors \(D_k = u_k v_k^\top\), where \(u_k^\top \in \mathbb{R}^{P}\) is the pattern over channels, and \(v_k^\top \in \mathbb{R}^{L}\) is the pattern over time. The objective function reads:

\[\min_{u, v, z} \sum_n \left(\|X^n - \sum_k (u_k v_k^\top) * z_k^n \|_2^2 + \lambda \sum_k z_k^n \right)\]

subject to \(z_k^n \ge 0\), \(||u_k||_2 \le 1\), and \(||v_k||_2 \le 1\). This rank-1 formulation is particularly meaningful in the case of magnetoencephalography (MEG), due to the physical properties of electromagnetic waves, which propagate instantaneously and add up linearly. It also brings robustness to the atom estimates, since less parameters are estimated.

This model is described in details in the original NIPS paper [1].

4. Univariate CSC with an alpha-stable distribution

The name alphaCSC originally referred to a particular model which uses an alpha-stable distribution for the noise, instead of the more classical Gaussian distribution. Note that the alphaCSC library is not limited to this particular model.

This model leads to a weighted formulation of univariate CSC, the weights being used to downweight noisy portions of the data:

\[\min_{d,z} \sum_n \left( \|\sqrt{w_n} \odot (x^n - \sum_k d_k * z_k^n) \|_2^2 + \lambda \sum_k z_k^n \right)\]

subject to \(z_k^n \ge 0\) and \(||d_k||_2 \le 1\), and where \(w_n\) are the weights which are learned using an EM algorithm.

This model is described in details in the original NIPS paper [2].

Cite our work

If you use this code in your project, please consider citing our work:

[1](1, 2) Dupré La Tour, T., Moreau, T., Jas, M., & Gramfort, A. (2018). Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals. Advances in Neural Information Processing Systems (NIPS).
[2](1, 2) Jas, M., Dupré La Tour, T., Şimşekli, U., & Gramfort, A. (2017). Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding. Advances in Neural Information Processing Systems (NIPS), pages 1099–1108.