Globalization has enhanced the transportation system resulting in the increased mobility of the people all around the world. More people are now travelling outside their own border for self-refreshment or business purpose. The task of the immigration officers in the airport thus has become more challenging now-a-days. Moreover, there are many countries whose people are restricted from travelling to other certain countries. Hence, the country identification by analyzing only the passport cover can reduce both time and hassle greatly by detecting those unauthorized travelers. In our paper, we present a model for country recognition by analyzing any passport cover using a deep Convolutional Neural Network (CNN) model based on the Residual Network architecture with 50 layers termed as ResNet50. We have experimented our model with images of seven passport covers of seven different countries from different angles and found an average accuracy rate of 98.56%. Our model can also be enhanced to detect fake and forged passports.