Crowdsourcing Dialect Characterization through Twitter

Bruno Goncalves (1) , David Sanchez (2)

1 CPT - Centre de Physique Théorique - UMR 7332
2 IFISC

We perform a large-scale analysis of language diatopic variation using geotagged microblogging datasets. By collecting all Twitter messages written in Spanish over more than two years, we build a corpus from which a carefully selected list of concepts allows us to characterize Spanish varieties on a global scale. A cluster analysis proves the existence of well defined macroregions sharing common lexical properties. Remarkably enough, we find that Spanish language is split into two superdialects, namely, an urban speech used across major American and Spanish citites and a diverse form that encompasses rural areas and small towns. The latter can be further clustered into smaller varieties with a stronger regional character.

Mots clés: Big Data; Machine Learning; Linguistics; Dialectology; language; variation; microblogging; datasets

Domaines: Physique [physics]

Bruno Goncalves, David Sanchez. Crowdsourcing Dialect Characterization through Twitter. PLoS ONE, 2014, 9 (e112074 ), ⟨10.1371/journal.pone.0112074⟩. ⟨hal-01242109⟩