Skip to Main content Skip to Navigation
Journal articles

Effect of Top-Down Connections in Hierarchical Sparse Coding

Abstract : Hierarchical sparse coding (HSC) is a powerful model to efficiently represent multidimensional, structured data such as images. The simplest solution to solve this computationally hard problem is to decompose it into independent layer-wise subproblems. However, neuroscientific evidence would suggest interconnecting these subproblems as in predictive coding (PC) theory, which adds top-down connections between consecutive layers. In this study, we introduce a new model, 2-layer sparse predictive coding (2L-SPC), to assess the impact of this interlayer feedback connection. In particular, the 2L-SPC is compared with a hierarchical Lasso (Hi-La) network made out of a sequence of independent Lasso layers. The 2L-SPC and a 2-layer Hi-La networks are trained on four different databases and with different sparsity parameters on each layer. First, we show that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers. Second, we demonstrate that the inference stage of the 2L-SPC is faster to converge and generates a refined representation in the second layer compared to the Hi-La model. Third, we show that the 2L-SPC top-down connection accelerates the learning process of the HSC problem. Finally, the analysis of the emerging dictionaries shows that the 2L-SPC features are more generic and present a larger spatial extension.
Document type :
Journal articles
Complete list of metadata

https://hal-amu.archives-ouvertes.fr/hal-02986570
Contributor : Laurent Perrinet <>
Submitted on : Friday, December 18, 2020 - 2:30:52 PM
Last modification on : Monday, March 29, 2021 - 1:44:08 PM

File

boutin-franciosini-ruffier-per...
Publisher files allowed on an open archive

Identifiers

Collections

Citation

Victor Boutin, Angelo Franciosini, Franck Ruffier, Laurent Perrinet. Effect of Top-Down Connections in Hierarchical Sparse Coding. Neural Computation, Massachusetts Institute of Technology Press (MIT Press), 2020, 32 (11), pp.2279-2309. ⟨10.1162/neco_a_01325⟩. ⟨hal-02986570⟩

Share

Metrics

Record views

109

Files downloads

16