Ensemble of Plug-in Modules for Knee Osteoarthritis Severity Classification Using Radiographs

Abstract Fine-grained classification deals with data with a large degree of similarity such as cat or bird species and similarly, knee osteoarthritis (KOA) severity classification (Kellgren-Lawrence grading) is one of the fine-grained classification tasks. Recently, plug-in module (PIM) that can be integrated to CNN-based or Transformer-based networks has been proposed to provide strongly discriminative regions for fine-grained classification and the results have outperformed the previous deep learning (DL) models. Therefore, a DL model that classifies KOA severity of a knee radiograph was developed utilizing PIMs. The dataset used for the study was a combination of two different open source datasets, Osteoarthritis Initiative and Multicenter Osteoarthritis Study (test set size: 13038). The final DL model was an ensemble of four different PIMs that used Swin and EfficientNet as the backbones (two each). The overall accuracy of the model was 82.252%, the lowest for KL grade 1 (65%) and the highest for KL grade 0 & 4 (90%). An ensemble of PIMs could classify KOA severity using simple radiographs with a fine accuracy, better than most of the previously proposed CNN-based models.