Road surface monitoring is mostly done manually in cities which is an intensive process of time consuming and labor work. The intention of this paper is to research on road damage detection and classification from road surface images using object detection method. This paper applied multiple convolutional neural network (CNN) algorithm to classify road damage and discovered which algorithm performs better in road damage detection and classification. The damages are classified in three categories pothole, crack and revealing. For this research data was collected from street of Dhaka city using smartphone camera and prepossessed the data like image resize, white balance, contrast transformation, labeling. This study applies R-CNN and faster R-CNN for object detection of road damages and apply Support Vector Machine (SVM) for classification and gets a better result from previous studies. Then losses are calculated using different loss functions. The results demonstrate the highest 98.88% accuracy and the lowest loss is 0.01.