Effect of Top-Down Connections in Hierarchical Sparse Coding - Aix-Marseille Université Accéder directement au contenu
Article Dans Une Revue Neural Computation Année : 2020

Effect of Top-Down Connections in Hierarchical Sparse Coding

Victor Boutin
Franck Ruffier

Résumé

Hierarchical sparse coding (HSC) is a powerful model to efficiently represent multidimensional, structured data such as images. The simplest solution to solve this computationally hard problem is to decompose it into independent layer-wise subproblems. However, neuroscientific evidence would suggest interconnecting these subproblems as in predictive coding (PC) theory, which adds top-down connections between consecutive layers. In this study, we introduce a new model, 2-layer sparse predictive coding (2L-SPC), to assess the impact of this interlayer feedback connection. In particular, the 2L-SPC is compared with a hierarchical Lasso (Hi-La) network made out of a sequence of independent Lasso layers. The 2L-SPC and a 2-layer Hi-La networks are trained on four different databases and with different sparsity parameters on each layer. First, we show that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers. Second, we demonstrate that the inference stage of the 2L-SPC is faster to converge and generates a refined representation in the second layer compared to the Hi-La model. Third, we show that the 2L-SPC top-down connection accelerates the learning process of the HSC problem. Finally, the analysis of the emerging dictionaries shows that the 2L-SPC features are more generic and present a larger spatial extension.
Fichier principal
Vignette du fichier
boutin-franciosini-ruffier-perrinet-20-feedback.pdf (5.61 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-02986570 , version 1 (18-12-2020)

Identifiants

Citer

Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent Perrinet. Effect of Top-Down Connections in Hierarchical Sparse Coding. Neural Computation, 2020, 32 (11), pp.2279-2309. ⟨10.1162/neco_a_01325⟩. ⟨hal-02986570⟩
84 Consultations
96 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More