Somali dialect identification in low-resource settings using machine learning and deep learning

Abstract This study investigates automatic dialect identification for the Somali language, focusing on its two primary dialects: MAXAA TIRI and MAAY. Somali exhibits substantial dialectal variation, which poses challenges for natural language processing (NLP) applications in low-resource settings. To support dialect-aware NLP research, we construct and manually annotate a dataset of 8947 Somali text samples collected from heterogeneous sources, including social media, news outlets, blogs, and formal documents. The study evaluates a range of traditional machine learning and deep learning models, including Naive Bayes, Support Vector Machines (SVM), and Bidirectional Long Short-Term Memory (BiLSTM) networks, for dialect classification. Experimental results show that Naive Bayes and BiLSTM achieve high classification performance under controlled evaluation settings. To mitigate overfitting and source bias, we apply source-aware data splitting, duplicate removal, and ablation analyses. However, results should be interpreted in light of dataset construction constraints, including expert-assisted translation for portions of the MAAY data. This work contributes a linguistically validated Somali dialect dataset and provides empirical insights into the effectiveness of machine learning and deep learning approaches for dialect identification in low-resource contexts.