Sentiment analysis is one of the major tasks in NLP that involves understanding user emotions and opinions from textual data. While sentiment analysis has seen extensive research in English and other widely spoken languages, the exploration of the same in the Bengali language is limited. In this paper, a novel approach for the sentiment analysis of Bengali texts is proposed, using two pre-trained transformer models, namely BanglaBERT and mBERT (bert-base-multilingual-cased). BanglaBERT and mBERT both fine tunes on our custom dataset to achieve superior performance in the sentiment classification task. The effectiveness of the proposed method is demonstrated on a manually labelled dataset of Facebook comments, where class imbalance is handled using a two-step resampling strategy. These results show that the BanglaBERT and mBERT achieved accuracies as high as 86% and 87% on our dataset after balancing using random oversampler and undersampler technique to outperform the current results in Bengali sentiment analysis and state-of-the-art performances. This therefore forms a clear indication that a model such as BERT will have its effectiveness considerably enriched by a language-specific counterpart, especially for settings where classes are highly sensitive to imbalance and generalization is across diverse sentiment categories.