Scopus Indexed Publications

Paper Details


Title
A Data-Centric AI Framework for Intelligent Data Cleaning and Quality Scoring

Author
, Md. Sadekur Rahman,

Email

Abstract

Data-driven AI emphasizes the importance of high quality input data over complex models in ML. We introduce a hybrid data cleaning framework that combines several approaches in a multi-stage process: (1) a rule-based layer that satisfies domain constraints, [2] a statistical layer (z-score and IQR) for simple outlier removal, and [7] an unsupervised ML layer (such as Isolation Forest) for complex anomaly detection. These layers are then combined into a composite Data Quality Score (DQS), which measures the overall health of the dataset. The important aspect of our work is the utilization of a nature inspired optimizer (PSO) to dynamically update the weights and thresholds of the DQS, which helps to focus each successive cleaning step on optimizing accuracy. We test the framework on typical tabular datasets (e.g. UCI Adult) with four levels of cleaning: Raw, Basic, Advanced, and Aggressive. Classifiers (Random Forest, Gradient Boosting, SVM, Logistic Regression) are tested using 5x5 cross-validation; we measure accuracy, AUC, and stability scores, with statistical tests on all comparisons. Our results demonstrate huge accuracy gains from cleaning: e.g. Random Forest accuracy increases from 0.87 (raw data) to 0.9857 with advanced cleaning (a highly significant gain, pi 0.01). In general, systematic cleaning achieves 5-15 percentage absolute accuracy gains across all models. These results confirm the observation that “AI performance is bounded by data quality”. Our work offers a tangible, automated preprocessing solution for data-driven AI, requiring less human labor while also ensuring a standardized level of quality. Our approach is particularly well-suited for high-risk applications (healthcare, finance) that demand accurate predictions, as it combines interpretability (through the use of rules/scores) with the strength of automated optimization.


Keywords

Journal or Conference Name
2026 International Conference on Artificial Intelligence for Sustainable Engineering and Innovation, AISEI 2026

Publication Year
2026

Indexing
scopus