Faecal contamination (by Escherichia coli [E. coli]) of household drinking water can have adverse effects on child health, particularly increasing the episodes of childhood diarrhea; however, the evidence is scanty in Bangladesh. This study utilised data from the most recent nationally representative 2019 Multiple Indicator Cluster Survey to investigate the relationship between E. coli concentration in household drinking water and diarrheal episodes among children aged under-5 years in Bangladesh. Childhood diarrhea was identified by asking the children’s mothers or caregivers if they had a diarrheal episode in the 2 weeks preceding the survey. E. coli colonies were counted as colony-forming units (CFUs) per 100 ml of water and classified into three risk groups (low: < 1 CFU/100 ml; moderate: 1–10 CFU/100 ml; and high: > 10 CFU/100 ml). The design-adjusted logistic regression was used to estimate the association between drinking water E. coli risk groups and childhood diarrhea, adjusting for potential confounders. We observed a significant association between household drinking water E. coli contamination and diarrheal episodes among under-5 children. Compared to the children from households with a low risk of E. coli contamination in drinking water, children from households with a moderate risk of E. coli contamination were 1.68 times more likely to have diarrhea, which was 2.28 times among children from households with a high risk of E. coli contamination. Findings of the study have significant policy implications and urge to ensure safe water supplies, improve water management practices and modify hygiene behaviours to reduce episodes of childhood diarrhea.