Convolutional neural networks (CNNs) have shown significant potential for automated whole-slide image (WSI) analysis for breast cancer diagnosis. However, most existing CNN models are computationally intensive, making multi-resolution WSI analysis challenging, while current lightweight methods often lack comprehensive evaluation across image magnifications, external datasets, and interpretability. To address these limitations, we propose DSNet, a lightweight CNN architecture that balances diagnostic accuracy, computational efficiency, cross-magnification robustness, and interpretability. DSNet comprises only 1.31 million parameters (5.01 MB), making it 2–17 more compact than widely used CNN models while achieving the fastest inference time (51.8 ms) and the highest accuracy-to-GFLOPs ratio (81.3), demonstrating superior computational efficiency. Extensive evaluation on multiple public datasets achieved accuracies of 99.8% (BreakHis-400), 94.9% (200), 92.7% (100), 88.0% (40), and 90.5% (BACH), demonstrating robust performance across different image magnifications and an independent external dataset. Statistical analysis further confirmed that these improvements over widely used CNN models were significant (). Furthermore, Score-CAM consistently highlighted diagnostically relevant tissue regions across magnifications, supporting the interpretability and reliability of the proposed model. Compared with existing lightweight breast cancer classification methods, DSNet achieves comparable or superior diagnostic performance while requiring over 20 fewer parameters and providing comprehensive validation across multiple magnifications, external datasets, and explainable predictions. Overall, DSNet provides an efficient, robust, and interpretable framework for practical AI-assisted WSI analysis.