In the domain of modern deep learning and classification techniques, the convolutional neural network (CNN) stands out as a highly successful and preferred method for image classification in artificial intelligence. Especially in the medical field, CNN has proven to be an ideal approach for analyzing medical data and accurately identifying diseases. Over the recent years, CNN has demonstrated significant potential and success in various computer vision tasks, with medical image classification being one of the prominent applications. In our study, we introduce a novel custom CNN model called MedvCNN, designed for classifying different types of classes. We conduct experiments with various image sizes to explore their versatility. In addition, long short-term memory (LSTM), a type of recurrent neural network (RNN), is incorporated into our approach. LSTM is specifically tailored to handle sequential data, making it ideal for time series analysis. However, its capabilities extend beyond time series data and are effectively applied to various sequential data types, including sequential vectors derived from image data. One of the key advantages of utilizing LSTM for image classification is its ability to effectively memorize and capture important features in the image data. This feature is particularly advantageous in medical image processing, where precise and accurate identification of key attributes is crucial for successful diagnosis and analysis. Furthermore, our experiments reveal that the hybrid custom LSTM model, MedvLSTM, a RNN algorithm, surpasses other methods in the domain of medical image classification. Our study places significant emphasis on attaining robust classification performance for medical image data through a sophisticated, parameter free approach, complemented by an ablation study, and comprehensive statistical analysis. This comprehensive analysis and evaluation allow us to gain a deeper understanding of the model’s effectiveness and its potential impact in the field of medical image analysis. We compare these two approaches to a baseline CNN architecture, aiming to streamline the classification process, reduce time consumption, and improve cost efficiency. Additionally, we present a real-time web-based AutoML framework along with a practical demonstration. Ultimately, our research provides a thorough investigation of the current state-of-the-art in medical image analysis accuracy, focusing on the utilization of neural networks and LSTM.