Toxic metal(loid)s (TMLs) in agricultural soils cause detrimental effects on ecosystem and human health. Therefore, source-specific health risk apportionment is very crucial for the prevention and control of TMLs in agricultural soils. In this study, 149 surface soil samples were taken from a coal mining region in northwest Bangladesh and analyzed for 12 TMLs (Pb, Cd, Ni, Cr, Mn, Fe, Co, Zn, Cu, As, Se, and Hg). Positive matrix factorization (PMF) and absolute principal component score-multiple linear regression (APCS-MLR) receptor models were employed to quantify the pollution sources of soil TMLs. Both models identified five possible sources of pollution: agrochemical practice, industrial emissions, coal-power-plant, geogenic source, and atmospheric deposition, while the contribution rates of each source were calculated as 28.2%, 17.2%, 19.3%, 19% and 16.3% in APCS-MLR, 22.2%, 13.4%, 24.3%, 15.1% and 25.1% in PMF, respectively. Agrochemical practice was the major source of non-carcinogenic risk (NCR) (adults: 32.37%, children: 31.54%), while atmospheric deposition was the highest source of carcinogenic risk (CR) (adults: 48.83%, children: 50.11%). NCR and CR values for adults were slightly higher than for children. However, the trends in NCR and CR between children and adults were similar. As a result, among the sources of pollution, agrochemical practices and atmospheric deposition have been identified as the primary sources of soil TMLs, so prevention and control strategies should be applied primarily for these pollution sources in order to protect human health.