Knee osteoarthritis (KOA) is a prevalent condition characterized by gradual progression, resulting in observable bone alterations in X-ray images. X-rays are the preferred diagnostic tool for their ease of use and cost-effectiveness. Physicians use the Kellgren and Lawrence (KL) grading system to understand the severity of an individual condition of KOA. This system categorizes the disease from normal to a severe stage. Early detection of the condition with this approach enables knee deterioration to be slowed down with therapy. In this study, we aggregated four datasets to generate an extensive dataset comprising 110,232 raw images by applying an augmentation technique called deep convolutional generative adversarial network (DCGAN). We employed advanced image pre-processing methods (adaptive histogram equalization (AHE), fast non-local means), including image resizing, to generate a substantial dataset and enhance image quality. Our proposed approach involved developing a modified compact convolutional transformer (CCT) model known as KOA-CCTNet as the foundational model. We further investigated optimal configurations by adjusting various parameters and hyperparameters in the final model to handle large datasets and address training time concerns efficiently. We investigated optimizing its configurations by adjusting numerous parameters and hyperparameters to efficiently manage extensive data and address concerns related to training time. Simulation results indicated that our proposed model outperforms other transfer learning models (Swin Transformer, Vision Transformer, Involutional Neural Network) in terms of accuracy. The test accuracy for the ResNet50, MobileNetv2, DenseNet201, InceptionV3, and VGG16 was 80.77%, 79.98%, 80.23%, 76.89%, and 79.58%, respectively. All of them were surpassed by our proposed KOA-CCTNet model, which had a test accuracy of 94.58% while classifying KOA X-ray images. Furthermore, we reduced the number of images to assess the model’s performance and compared it to existing models. However, by employing a large datahub, our proposed approach provides a unique and effective way to diagnose KOA grades with satisfying results.