Leveraging local and global word context for multi-label document classification

Abstract

With the increasing volume of text documents, it is crucial to identify the themes and topics contained within. Labelling documents with the identified topics is called multi-label classification. Interdependencies exist between not just words, but sentences and paragraphs. These longer sequences and more complex relationships increase the label identification challenge. Five novel deep neural networks are proposed and evaluated for their performance classifying longer documents. The RCLNN applies the RCL to NLP, combining that model with a CNN which has demonstrated success on short text. The QRCNN similarly extends a CNN in addition to implementing it with a QRNN. The remaining three models build on these base models, integrating them in a novel pseudo-Siamese approach. Experiments find QRCNN highest performing overall, with the PSRCNNA model a close second, indicating that the pseudo-Siamese approach can be performant when married with attention.