Offline English handwritten text recognition is a very challenging research field because different people write differently; it is among the most significant research projects and fields of study in artificial intelligence and computer vision. It is difficult to identify the English handwriting because the characters have a similar look. This research aims to identify and translate the handwritten text into digital form. The process of identifying handwritten text from pictures, papers, and other sources and transforming it into a machine-readable format for additional processing is known as Handwritten text recognition (HTR). Convolutional neural networks (CNNs) with different architectures are used in this system to build a model that can precisely classify words. We employed two primary methods to enhance handwritten text detection: direct word classification and character segmentation. For the latter, we create bounding boxes for every character using contour with convolution. After segmenting the characters, we feed the data to a CNN for classification. Based on the segmentation and classification findings, we reconstruct each word next. The Handwritten Character and Handwritten Recognition datasets have been used in this system. It contains 206,924 surnames and 209,809 first names overall. We achieved favorable outcomes with multiclass categorization. To create the suggested CNN model for handwritten recognition which consists of three convolutional layers, three max-pooling layers, one flatten layer, and two dense layers that are important to avoid overfitting. Nonetheless, this study demonstrates the role of various learning rates and optimizers in models’ performances. We test our CNN implementation using the two datasets to determine the accuracy of the handwritten text. When using the unseen test image, the trained system achieves an average accuracy greater than 94.72%.