Two-Step Approaches to Overcome Data Imbalance in the Development of an Electrocardiography Data Quality Assessment Algorithm: A Real-World Data Challenge

Abstract Automation of electrocardiography (ECG) signal quality assessment is indispensable for the development of artificial intelligence-based decision support systems. We developed machine and deep learning algorithms to classify the quality of ECG data automatically. A total of 31,127 twenty-second ECG segments of 250 Hz were used as the training/validation dataset. Data qualities were categorized into three classes: acceptable, unacceptable, and uncertain. In the training/validation dataset, 29,606 segments (95%) were in the acceptable class. Two 1-step 3-class approaches and two 2-step binary sequential approaches were developed using random forest (RF) and 2-dimensional convolutional neural network (2D CNN) classifiers. Four approaches were tested on 9,779 test samples from another hospital. On the test dataset, the 2-step 2D CNN approach showed the best overall accuracy (0.85), and the 1-step 3-class 2D CNN approach showed the worst overall accuracy (0.54). The most important parameter, precision in the acceptable class, was greater than 0.9 for all approaches but recall in the acceptable class was better for the 2-step approaches: 1-step RF (0.77) and 2D CNN (0.51) vs. 2-step RF (0.89) and 2D CNN (0.94). When the acceptable and uncertain classes were merged, all four approaches showed comparable performance, but the 2-step approaches had higher precision in the unacceptable class: 1-step RF (0.47) and 2D CNN (0.37) vs. 2-step RF (0.72) and 2D CNN (0.71). For ECG quality classification, where substantial data imbalance exists, the 2-step approaches showed more robust performance than the 1-step approach.