The globe has reached a critical juncture in the last recent years. According to data we collected from an official internet source, Bangladesh recorded 913,258 confirmed cases, 14,646 death cases with a 1.60% mortality rate, and 85% recovery rate as of June 30, 2021. Furthermore, the delta variant currently has a significant impact on improving the current COVID situation in Bangladesh. So, one of the efficient ways to prevent this outbreak is to building multiple Bangladesh outbreak prediction models to analyze historical data and predict with it for making decisions and implementing appropriate COVID-19 control measures. In this study, a machine learning model, an Auto-Regressive Integrated Moving Average (ARIMA) and Prophet, were developed using time series analysis to forecast new cases in Bangladesh in the coming days. This study examined the model outputs, compared their performance, and created predicted values from these models using the Python programming language. The ARIMA model is the best fit model among the algorithms used to predict the new COVID-19 situation in Bangladesh. The primary goals of this paper are to analyze COVID-19 trends and predict the new upcoming cases and assist decision-makers in controlling the Bangladesh outbreak.