The rapid rise of generative AI tools for health advice has created serious trustworthiness concerns, particularly in Bangladesh, where limited medical literacy and cyberchondria, heightened anxiety from unverified symptom checks, pose significant risks. This study proposes a robust framework for evaluating AI-generated Bengali health suggestions through expert validation and explainable machine learning. A novel dataset, MedBanglaTrust3, was developed, containing 6,660 suggestions from ChatGPT and Google’s GEMINI on symptoms such as headaches, insomnia, and fatigue. Suggestions were categorized as Highly Relevant, Partially Relevant, or Not Relevant using Fleiss’ kappa for inter-rater reliability. Four models were tested: Random Forest, BiLSTM, sagorsarker/bangla-bert-base, and an attention-enhanced Bangla-BERT, with the latter achieving 88.34% accuracy, 82.62% precision, and 82.51% recall, demonstrating strong multi-class performance. Evaluation included confusion matrices, ROC-AUC curves (up to 0.96), and LIME, which highlighted contextual cues and domain-specific keywords shaping decisions. The findings show that attention-based transformers effectively assess trustworthy Bengali health suggestions, addressing risks of unverified AI-generated advice in low-access rural contexts. This framework advances multilingual medical NLP, supports safe AI deployment in low-literacy settings, and provides policy insights for the Bangladesh Telecommunication Regulatory Commission (BTRC) to mitigate health misinformation.