In the present world, one of the most interesting topics is Handwritten Recognition due to its academic and commercial interest in different research fields. But deal with it a little bit tough because of different size and style. There are many works have been accomplished base in handwritten recognition including Bangla. Here proposed a model which is classified Bangla handwritten numeral using capsule net (a new type of neural network represents activity vector as parameters). The Model is trained and valid with ISI handwritten database [1], BanglaLekha Isolated [2], CMATERdb 3.1.1 [3] and all database together that was achieved 99.28% validation accuracy on ISI handwritten character database, 97.62% validation accuracy on BanglaLekha Isolated, 98.33% validation accuracy on CMATERdb 3.1.1 dataset and 98.90% validation accuracy combination mixed dataset. This model gives satisfactory recognition accuracy compared to other existing models.