Service interruption on Monday 11 July from 12:30 to 13:00: all the sites of the CCSD (HAL, EpiSciences, SciencesConf, AureHAL) will be inaccessible (network hardware connection).
Skip to Main content Skip to Navigation
Journal articles

Effect of Top-Down Connections in Hierarchical Sparse Coding

Victor Boutin 1 Angelo Franciosini 1 Franck Ruffier 2 Laurent Perrinet 1 
2 BIOROB - Biorobotique
ISM - Institut des Sciences du Mouvement Etienne Jules Marey
Abstract : Hierarchical sparse coding (HSC) is a powerful model to efficiently represent multidimensional, structured data such as images. The simplest solution to solve this computationally hard problem is to decompose it into independent layer-wise subproblems. However, neuroscientific evidence would suggest interconnecting these subproblems as in predictive coding (PC) theory, which adds top-down connections between consecutive layers. In this study, we introduce a new model, 2-layer sparse predictive coding (2L-SPC), to assess the impact of this interlayer feedback connection. In particular, the 2L-SPC is compared with a hierarchical Lasso (Hi-La) network made out of a sequence of independent Lasso layers. The 2L-SPC and a 2-layer Hi-La networks are trained on four different databases and with different sparsity parameters on each layer. First, we show that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers. Second, we demonstrate that the inference stage of the 2L-SPC is faster to converge and generates a refined representation in the second layer compared to the Hi-La model. Third, we show that the 2L-SPC top-down connection accelerates the learning process of the HSC problem. Finally, the analysis of the emerging dictionaries shows that the 2L-SPC features are more generic and present a larger spatial extension.
Document type :
Journal articles
Complete list of metadata
Contributor : Laurent Perrinet Connect in order to contact the contributor
Submitted on : Friday, December 18, 2020 - 2:30:52 PM
Last modification on : Wednesday, November 3, 2021 - 5:52:27 AM


Publisher files allowed on an open archive




Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent Perrinet. Effect of Top-Down Connections in Hierarchical Sparse Coding. Neural Computation, Massachusetts Institute of Technology Press (MIT Press), 2020, 32 (11), pp.2279-2309. ⟨10.1162/neco_a_01325⟩. ⟨hal-02986570⟩



Record views


Files downloads