Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Sparse Hebbian Learning is efficient with egalitarian homeostasis

Abstract : Following the work of \citet{Olshausen98}, a number of unsupervised learning algorithms were proposed for receptive field formation in the primary visual cortex (V1). We describe here the theoretical formulation of a novel and simple algorithm aiming at achieving the shortest representation in an over-complete dictionary of unknown features using the L$_0$ norm, a "hard" NP-complete problem. Inspired by parallel computing solutions with spiking events, we propose here a solution based on correlation-based inhibition with a tuned egalitarian homeostasis. We present results of the simulation of this dynamical neural network with natural images and compare it to the {\sc SparseNet} solution. This algorithm exhibited similarly the formation of edge-like components as is observed in the input layer of V1 and to assess the quality of the different sets of filters, we additionally compared their relative efficiencies and show that our sub-optimal solution to the hard problem performs better than the optimal solution to the relaxed problem. We also show that homeostasis, by tuning the competition and cooperation, may yield solutions of different qualities which coding efficiency drastically depended on the goal assigned to the model by the cost function. This model provides a bridge between the non-linearity of the neural response and optimal use of distributed probabilistic representations of information. It is suggesting the importance of the role of interactions within neural assemblies \citep{Hebb49} to efficiently build representations in the neural code.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-00156610
Contributor : Laurent Perrinet Connect in order to contact the contributor
Submitted on : Wednesday, September 5, 2007 - 2:22:41 PM
Last modification on : Thursday, January 11, 2018 - 6:16:39 AM
Long-term archiving on: : Tuesday, September 21, 2010 - 1:44:51 PM

Files

perrinet07shl.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00156610, version 2
  • ARXIV : 0706.3177

Citation

Laurent Perrinet. Sparse Hebbian Learning is efficient with egalitarian homeostasis. 2007. ⟨hal-00156610v2⟩

Share

Metrics

Record views

2

Files downloads

121