Increased exposure to arsenic (As) has been reported in many arid and semi-arid areas, where drinking water and agricultural irrigation strongly rely on groundwater. High-As groundwater occurred widely in the Northern Henan Plain, which belongs to the lower Yellow River alluvial plain area, posing a new threat to the health of local residents. Based on 77 groundwater samples collected from different hydrogeological units, the sources and optical characteristics of dissolved organic matter (DOM) components in groundwater were analyzed using a three-dimensional fluorescence spectroscopy excitation-emission matrix (3D-EEM) and parallel factor analysis (PARAFAC), to reveal the influence of organic matter on As mobilization and enrichment in Quaternary alluvial aquifers. The results showed that high-As groundwater was concentrated in the areas of the piedmont alluvial depressions and the Yellow River crevasse splay, which was closely related to the richness of organic matter in the buried sediments of the Yellow River paleochannels. Vertically, the variability of groundwater As was governed by the redox conditions and anthropogenic activities, leading to elevated As levels in deep aquifers. High-As groundwater was characterized by high content of humic-like components (C2+C3), strong humification, and weak authigenic sources, which can be attributed to the effects of frequent diversion of the Yellow River and the introduction of more terrestrial humic-like components by intensive agricultural activities (e.g., irrigation). The significant correlations between Fe2+ and As and NH4+ (r = 0.65, 0.48, p < 0.05) as well as the maximum fluorescence intensity of C2 and C3 with As (r = 0.59, 0.74, p < 0.05) reflected the dominant impact of DOM on As migration. The terrestrial-derived high molecular weight organic matter C2 facilitated As mobilization through complexation reactions, while the labile humic-like component C3 triggered the reductive dissolution of iron oxides/hydroxides through microbial metabolic processes, which together contributed to the enrichment of As in groundwater from the Northern Henan Plain.