Abstract
Convolution kernels, such as sequence and tree kernels, are advantageous for both the concept and accuracy of many natural language processing (NLP) tasks. Experiments have, however, shown that the over-fitting problem often arises when these kernels are used in NLP tasks. This paper discusses this issue of convolution kernels, and then proposes a new approach based on statistical feature selection that avoids this issue. To enable the proposed method to be executed efficiently, it is embedded into an original kernel calculation process by using sub-structure mining algorithms. Experiments are undertaken on real NLP tasks to confirm the problem with a conventional method and to compare its performance with that of the proposed method.
Keywords
Computer scienceKernel (algebra)Convolution (computer science)Artificial intelligenceSelection (genetic algorithm)Feature selectionProcess (computing)Feature (linguistics)Pattern recognition (psychology)Natural language processingKernel methodMachine learningAlgorithmSupport vector machineMathematicsProgramming languageArtificial neural network
Affiliated Institutions
Related Publications
Supervised Learning of Universal Sentence Representations from Natural\n Language Inference Data
Many modern NLP systems rely on word embeddings, previously trained in an\nunsupervised manner on large corpora, as base features. Efforts to obtain\nembeddings for larger chunk...
DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
Recurrent neural nets (RNN) and convolutional neural nets (CNN) are widely used on NLP tasks to capture the long-term and local dependencies, respectively. Attention mechanisms ...
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Deep learning methods employ multiple processing layers to learn hierarchical representations of data, and have produced state-of-the-art results in many domains. Recently, a va...
Extensions of marginalized graph kernels
Positive definite kernels between labeled graphs have recently been proposed. They enable the application of kernel methods, such as support vector machines, to the analysis and...
A Discriminative Kernel-Based Approach to Rank Images from Text Queries
This paper introduces a discriminative model for the retrieval of images from text queries. Our approach formalizes the retrieval task as a ranking problem, and introduces a lea...
Publication Info
- Year
- 2004
- Type
- article
- Pages
- 119-es
- Citations
- 37
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
37
OpenAlex
Cite This
Jun Suzuki,
Hideki Isozaki,
Eisaku Maeda
(2004).
Convolution kernels with feature selection for natural language processing tasks.
, 119-es.
https://doi.org/10.3115/1218955.1218971
Identifiers
- DOI
- 10.3115/1218955.1218971