Scopus Indexed Publications

Paper Details


Title
Automatic Classification of Banglish and English Text Using TF-IDF and Machine Learning Models

Author
Nitta Nando Roy, ABDULLAH-AL-MAMUN SAIF, ANMAY PAUL ARPAN, Md Mustafijur Rahman, Prosenjit Sarker, Sohanur Rahman,

Email

Abstract

The increasing use of code-mixed and transliterated languages on social media has introduced new challenges in natural language processing (NLP). Among these, Banglish—Bangla written using the Latin script and frequently interwoven with English—has become a prominent form of digital communication in Bangladesh. The accurate classification of Banglish and English text is a critical preprocessing step for downstream NLP applications such as sentiment analysis, topic modeling, machine translation, and content moderation. Traditional language identification tools fail in this setting due to the non-standard orthography and heavy lexical borrowing that characterize Banglish. In this paper, we present a comparative evaluation of ten machine learning models for the task of Banglish–English classification. A balanced dataset containing 8,000 labeled samples was obtained from Kaggle and preprocessed through normalization, emoji removal, and stratified splitting into training, validation, and test subsets. Feature extraction was performed using Term Frequency–Inverse Document Frequency (TFIDF) with unigrams and bigrams, limited to the 5,000 most frequent n-grams. Models spanning probabilistic, linear, tree-based, and neural approaches were trained and evaluated, with Multinomial Naïve Bayes emerging as the best-performing classifier, achieving a validation accuracy of 0.9931 with negligible training time. The findings suggest that lightweight statistical models can outperform more complex architectures in efficiency without compromising accuracy, offering promising directions for practical applications in multilingual NLP.


Keywords

Journal or Conference Name
2026 International Conference on Artificial Intelligence for Sustainable Engineering and Innovation, AISEI 2026

Publication Year
2026

Indexing
scopus