Artificial intelligence (AI) is becoming increasingly influential in the academic sector, which is why it is important to explore the ethical dilemmas and concerns surrounding AI chatbots’ design, development, and deployment in educational contexts. Conducted as a thematic literature review, this paper explores existing research on AI in education, AI chatbots, and their integration with higher education to gather evidence and insights that discuss ethical implications and challenges. The study has analyzed several articles on AI chatbots and their integration into academic fields. Significant gaps have been identified, such as the need for more practical implications and the recognition of AI chatbots as a collaborative tool for academic purposes. More AI chatbots should be explicitly trained on data relevant to the learners’ study to examine their usefulness properly. The paper discusses the ethical dilemmas and concerns about the design, development, and deployment of AI chatbots in higher education. It seeks to provide insights and recommendations to ensure the ethical use of AI chatbots in higher education by identifying significant gaps in the existing literature and providing scenarios to expect in the development of AI in education.