Handwritten digit recognition is one of the most novel topics from last few years. The complexity of recognition handwriting are differ in languages because of their shapes, character numbers and streak. Albeit Bangla is the 7th most popular language in order to the number of first language speakers. Remaining approaches use discrete feature expulsion methods and algorithms to recognize handwritten digits. Recently, Deep learning and convolutional neural network is used to solve the classification problem, it gives better accuracy for image classification with its distinct features. In this paper, we have proposed a Convolutional Neural Network referred as “ByannoNet”, to identify Bangla hand-written digits. We worked with the richest and popular dataset called NumtaDB generated and published by the Bengali.ai community. Our proposed model has achieved 97 percent accuracy with a very low cross-entropy rate.