Classifying closely related languages or dialects is challenging due to strong linguistic similarities, and limited research exists in this area. This study focuses on distinguishing Chittagonian and Standard Bengali by analyzing their textual patterns and context. A dataset of 6,150 sentences was created, divided into two classes: Standard Bengali and Chittagonian, to enable proper identification. We evaluate five machine learning models, three deep learning models, and five transformer-based approaches for this task. Among them, the proposed BanglaBERT model achieves the highest accuracy of 98.86%, outperforming all others. This demonstrates its effectiveness in capturing subtle linguistic differences and handling low-resource language classification tasks. To the best of our knowledge, this is the first publicly available study on Bangla-Chittagonian language classification using a BERT-based approach.