Artificial Intelligence (AI) is increasingly being employed in critical decision-making processes such as medical diagnosis, credit approval, criminal justice, and many more. However, many AI models exploit complex algorithms that are difficult for humans to see through, which can lead to concerns about accountability, bias, and the ability to trust the outcomes. With the increasing demand for AI systems to be transparent, interpretable, and reliable, the field of Explainable AI (XAI) has gained attention of the researchers. This paper presents a robust hybrid architecture that combines Bidirectional Gated Recurrent Units (BiGRU) and Artificial Neural Networks (ANN) for the classification of texts and sentiment analysis. Interpretable Model Agnostic Explanation (LIME) has been employed with our proposed model to enhance confidence in the outcomes. The proposed architecture is found to be effective for sentiment analysis from texts, and classifying images containing handwrit- ten characters. It leverages the BiGRU to model the sequential dependencies in the data, while the ANN is used for the final classification. Evaluations on both Bengali and English datasets show that the proposed architecture outperforms state-of-the-art models in various performance metrics, providing meaningful and interpretable explanations for its predictions. The model can be used in systems that require the architectures to be computationally less demanding, yet a decent accuracy is secured.