Convolutional neural networks are current times state-of-art algorithms widely used in image classification. This paper has explored the image classification of social gatherings with state-of-the-art neural network models. We introduce image classification with the modified VGG16 model and the modified InceptionV3 model. Images are first pre-processed and then given input to the models for multi-class classification. We have modified layers in the models, resulting in the best accuracy for our dataset. Data augmentation and layer modification schemes are applied in this paper. The algorithm learns to identify the classes of an image by performing feature extraction and data augmentations of each image. Throughout this research, we discovered that the approaches suggested in this paper improve the performance of the models. Our task was based on four classes of social gathering images. We concluded that the layer-modified VGG16 model with augmentation gives us the best results with a training accuracy of 90.99% and validation accuracy of 87.18%.