Abstract: Breast cancer is a malignant tumor that affects women. It is the most prevalent cancer in women, affecting about 10% of all women at any point in their lives. The development of breast cancer begins in the lobules or ducts of the cells. Early detection and prevention are the best ways to stop this cancer from spreading. In this study, five Convolution Neural Network (CNN) models are used to process image data of breast cells. AlexNet, InceptionV3, GoogLeNet, VGG19 and Xception models are used for the classification of Invasive Ductal Carcinoma, IDC and Non-Invasive Ductal Carcinoma (Non-IDC) cells. The models are trained and tested at different epochs to record the learning rate. It is observed from the study that with higher epochs, the data loss decreases and accuracy increases. The accuracy of InceptionV3 and Xception is 92.48% and 90.72% respectively. Likewise, VGG19 and AlexNet have fairly close accuracy of 94.83% and 96.74%. However, GoogLeNet dominates over the other implemented models with the highest accuracy of 97.80%. The GoogLeNet model performs with high accuracy and precision in detecting IDC cells responsible for breast cancer.