Enhancing ReliefF for multi-label text classification via high-order label correlation

Document Type

Article

Publication Date

3-1-2026

Abstract

ReliefF is a widely used feature selection algorithm due to its effectiveness in high-dimensional feature spaces and its non-parametric nature. However, existing ReliefF-based approaches often rely on the similarity between the label sets of two instances. They ignore global and high-order label correlations, which lower the accuracy performance of these approaches. In multi-label data, labels can correlate with more than one label and tend to form a group based on their similarities. Therefore, this study introduces HLC-ReliefF, a novel feature selection method based on ReliefF by integrating high-order label correlations for multi-label text classification. This allows scalable and innovative knowledge discovery in technological innovation systems. First, mutual information is used to extract the non-linear relationship between labels and features. It is employed to cluster the label space for capturing global and high-order label correlations. Second, imbalanced labels are mitigated by assigning a proportion of labels to determine the weight of each cluster. Finally, the hybrid metric for ReliefF’s instance-distance computation is adapted by implementing feature distance and cluster-aware label distance. Cluster similarity is integrated into the feature update weights to increase the influence of neighbors with more similar cluster structures to the target instance. The effectiveness of HLC-ReliefF is validated on nine open-access benchmark multi-label datasets using five standard evaluation metrics. The results demonstrate that HLC-ReliefF outperforms other methods across most metrics and datasets. Ablation studies also confirm the major contribution of label clustering and cluster weighting. This highlights their impact on improving ReliefF for capturing label correlations.

Keywords

High-order label correlations, Multi-label feature selection, Mutual information, ReliefF, Text classification

Publication Title

Results in Engineering

ISSN

2590-1230

DOI

10.1016/j.rineng.2026.109411

Funders

Universitas Multimedia Nusantara, UCSI University

Volume

29

Publisher

Elsevier B.V.

Share

COinS