Animal source foods (ASF) are nutrient-dense and essential for the growth and development of children. The Bangladesh Demographic and Health Survey (BDHS) 2022 reported that approximately two-thirds of children aged 6–23 months consumed eggs/flesh foods. However, overall consumption patterns, trends, and factors influencing ASF intake among children in Bangladesh were not well-documented. Therefore, the study aimed to assess the trends and associated factors of ASF consumption among children aged 6–23 months in Bangladesh. A total of 9401 children were extracted from four consecutive BDHS (2011, 2014, 2017/18, and 2022). The Cochran-Armitage test was conducted to assess the trends in ASF consumption, while a two-stage multilevel mixed-effects logistic regression was performed to identify the associated factors. The consumption of ASF significantly increased to 79.1% in 2017/18 from 67% in 2011 but decreased to 73.3% in 2022. ASF consumption was found to be higher among children whose mothers were educated (AOR = 1.60, 95% CI = 1.30–1.98), employed in either agricultural (AOR = 1.27, 95% CI = 1.04–1.54) or non-agricultural (AOR = 1.36, 95% CI = 1.07–1.72) activities, pregnant (AOR = 2.54, 95% CI = 1.66–3.87), had received ANC 1–3 times (AOR = 1.43, 95% CI = 1.20–1.72) or ≥4 times (AOR = 1.59, 95% CI = 1.29–1.95), and was exposed to media (AOR = 1.21, 95% CI = 1.04–1.39). Furthermore, consumption increased with increasing the age of children, and the wealth of their families. However, children who experienced illness were less likely to consume ASF (AOR = 0.76, 95% CI = 0.68–0.86). The recent declines in ASF consumption emphasize the need for targeted interventions to increase ASF consumption among children in Bangladesh.