How to Fine-Tune BERT for Text Classification?

Chi Sun; Xipeng Qiu; Yige Xu; Xuanjing Huang

doi:10.1007/978-3-030-32381-3_16

Keywords

Computer scienceLanguage modelTransformerEncoderArtificial intelligenceNatural language processingTask (project management)Machine learningSpeech recognitionElectrical engineeringVoltage

Affiliated Institutions

Fudan University CN

Related Publications

TinyBERT: Distilling BERT for Natural Language Understanding

Xiaoqi Jiao , Yichun Yin , Lifeng Shang +5 more

Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usuall...

2020 Findings of the Association for Compu... 1491 citations

Supervised Multimodal Bitransformers for Classifying Images and Text

Douwe Kiela , Suvrat Bhooshan , Hamed Firooz +2 more

Self-supervised bidirectional transformer models such as BERT have led to dramatic improvements in a wide variety of textual classification tasks. The modern digital world is in...

2019 arXiv (Cornell University) 163 citations

Attention Is All You Need

Ashish Vaswani , Noam Shazeer , Niki Parmar +5 more

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also co...

2025 6466 citations

Understanding the Behaviors of BERT in Ranking

Yifan Qiao , Chenyan Xiong , Zhenghao Liu +1 more

This paper studies the performances and behaviors of BERT in ranking tasks. We explore several different ways to leverage the pre-trained BERT and fine-tune it on two ranking ta...

2019 arXiv (Cornell University) 145 citations

Publicly Available Clinical

Emily Alsentzer , John R. Murphy , William Boag +4 more

Contextual word embedding models such as ELMo and BERT have dramatically improved performance for many natural language processing (NLP) tasks in recent months. However, these m...

2019 Proceedings of the 2nd Clinical Natur... 1422 citations

Publication Info

Year: 2019
Type: book-chapter
Pages: 194-206
Citations: 1275
Access: Closed

External Links