Finding rotten fruits and vegetables has been important, especially in the agricultural industry. Computer vision has significant applications in the automation of damaged, freshness detection of fruits and vegetables. In recent decades, the farming sector has discovered computer machine vision and image processing technology to be more and more beneficial, particularly for implementations in quality control by identifying rotten and freshness. Farmers cannot contribute effectively between fresh and rotten fruits, vegetables because this is mainly done by people. People tire out after performing the same task for several days, whereas robots do not. By identifying weaknesses in agricultural product, the study suggested a technique for minimizing human effort and worktime. Vegetables and fruits with defects might affect healthy fruits if they are not identified in time. As an outcome, we put up a methodology to stop rottenness from spreading. The suggested model detects between fresh and decaying fruits and vegetables depending on the input fruit and vegetable photos. In this work, we used six different types of fruits and vegetables like carrot, potato, calabash, cucumber, eggplant, and cauliflower, as well as fruits likes mango, banana, star fruit, jackfruit, guava, and papaya. This study discusses multiple image processing methods for rottenness categorization of fruits and vegetables. A Convolutional Neural Network (CNN), KNN, and SVM are used to gather the features from the data fruit and vegetable photos. On Google and Kaggle datasets, the efficiency of the suggested model is evaluated, and CNN model shows the greatest accuracy which is 95 percent.