Timely detection of heart disease is a crucial factor in making effective clinical decisions, especially in disadvantaged regions due to the scarcity of healthcare facilities. This research aims to investigate the performance of some supervised machine learning techniques in predicting heart diseases. In this study, regression-based classification, tree-structured machine learning approaches, probabilistic approaches, and approaches using margin values were investigated. Furthermore, the proposed study evaluated some machine learning techniques derived from random bagging and boosting. In addition, class imbalance handling techniques based on the Synthetic Minority Oversampling Technique (SMOTE) were integrated as a solution to the imbalance problem identified in the dataset. Above all, some ensemble machine learning approaches using probability averaging and integration were developed to improve the machine learning performance. Experimental results proved that machine learning approaches based on the random ensemble performed better. In other words, some machine learning approaches based on stacking achieved better performance by reaching a maximum performance with 93.44% accuracy, 0.93 for F1-measure, and 0.90 for precision.