Diseases during the growth phases have a significant impact on the production of citrus fruits, degrading the quality of the fruits. To prevent significant losses, early detection of the disease severity and an accurate diagnosis are crucial. In this study, we emphasized the citrus fruit disease classification, then we tested it on a non-citrus apple leaf dataset for better confirmation. The research uses deep learning models (VGG16, VGG19, ResNet50) along with machine learning algorithms (KNN, Naive Bayes, Random Forest, SVM, Logistic Regression) for disease classification to increase accuracy. Accordingly, we achieved the highest accuracy for disease classification, with 99.69% for orange using ResNet50 paired with Logistic Regression, and 95% for lemon and 99.20% for apple leaf using ResNet50 combined with SVM, along with Recall of 96.60%, Precision of 95.80%, F1-Score of 95.90%, MCC of 96.70%, Kappa of 96.60% and GDR of 97.40% for lemon, and Recall of 99.20%, Precision of 99.10%, F1-Score of 99.10%, MCC of 98.80%, Kappa of 98.80% and GDR of 99.10% for apple leaf, while the ResNet50 with Logistic Regression model achieved Recall of 99.60%, Precision of 99.60%, F1 score 99.60%, MCC of 99.20%, Kappa of 99.20%, and GDR of 99.30% for orange. The proposed model also outperforms the existing models in which most of them classified the diseases using the Softmax classifier without using any individual classifiers. Furthermore, k-means clustering is used to find the infected region of fruits, and a ResNet50-based fuzzy logic control system is used to evaluate degrees of severity, especially of lemon and orange diseases. Moreover, K-fold cross-validation has been employed to ensure the model's robustness and validate its performance.