Neural Representations of Dialogical History for Improving Upcoming Turn Acoustic Parameters Prediction

Simone Fuscone (1, 2, 3) , Benoit Favre (1, 3, 4) , Laurent Prevot (5, 2, 1, 6)

1 AMU - Aix Marseille Université
2 LPL - Laboratoire Parole et Langage
3 LIS - Laboratoire d'Informatique et des Systèmes (LIS) (Marseille, Toulon)
4 TALEP - Traitement Automatique du Langage Ecrit et Parlé
5 IUF - Institut universitaire de France
6 ILCB - Institute of Language, Communication and the Brain

Predicting the acoustic and linguistic parameters of an upcoming conversational turn is important for dialogue systems aiming to include low-level adaptation with the user. It is known that during an interaction speakers could influence each other speech production. However, the precise dynamics of the phenomena is not well-established, especially in the context of natural conversations. We developed a model based on an RNN architecture that predicts speech variables (Energy, F0 range and Speech Rate) of the upcoming turn using a representation vector describing speech information of previous turns. We compare the prediction performances when using a dialogical history (from both participants) vs. monological history (from only upcoming turn's speaker). We found that the information contained in previous turns produced by both the speaker and his interlocutor reduce the error in predicting current acoustic target variable. In addition the error in prediction decreases as increases the number of previous turns taken into account.

Mots clés: convergence; prediction; acoustic features; prosody

Domaines: Linguistique; Informatique

Simone Fuscone, Benoit Favre, Laurent Prevot. Neural Representations of Dialogical History for Improving Upcoming Turn Acoustic Parameters Prediction. Interspeech 2020, Oct 2020, Virtual (Shangai), China. pp.4203-4207, ⟨10.21437/interspeech.2020-2785⟩. ⟨hal-03224194⟩