End-To-End Multi-Task Learning With Attention

Shikun Liu; Edward Johns; Andrew J. Davison

doi:10.1109/cvpr.2019.00197

Abstract

We propose a novel multi-task learning architecture, which allows learning of task-specific feature-level attention. Our design, the Multi-Task Attention Network (MTAN), consists of a single shared network containing a global feature pool, together with a soft-attention module for each task. These modules allow for learning of task-specific features from the global features, whilst simultaneously allowing for features to be shared across different tasks. The architecture can be trained end-to-end and can be built upon any feed-forward neural network, is simple to implement, and is parameter efficient. We evaluate our approach on a variety of datasets, across both image-to-image predictions and image classification tasks. We show that our architecture is state-of-the-art in multi-task learning compared to existing methods, and is also less sensitive to various weighting schemes in the multi-task loss function. Code is available at https://github.com/lorenmt/mtan.

Keywords

Computer scienceTask (project management)WeightingFeature (linguistics)Artificial intelligenceCode (set theory)Artificial neural networkEnd-to-end principleMulti-task learningNetwork architectureArchitectureTask analysisFunction (biology)Feature extractionMachine learningEngineeringSet (abstract data type)

Affiliated Institutions

Imperial College London GB

Related Publications

Multi-Stage Progressive Image Restoration

Syed Waqas Zamir , Aditya Arora , Salman Khan +4 more

Image restoration tasks demand a complex balance between spatial details and high-level contextualized information while recovering images. In this paper, we propose a novel syn...

2021 2021 IEEE/CVF Conference on Computer ... 1696 citations

Fully Convolutional Instance-Aware Semantic Segmentation

Yi Li , Haozhi Qi , Jifeng Dai +2 more

We present the first fully convolutional end-to-end solution for instance-aware semantic segmentation task. It inherits all the merits of FCNs for semantic segmentation [29] and...

2017 1120 citations

Uformer: A General U-Shaped Transformer for Image Restoration

Zhendong Wang , Xiaodong Cun , Jianmin Bao +3 more

In this paper, we present Uformer, an effective and efficient Transformer-based architecture for image restoration, in which we build a hierarchical encoder-decoder network usin...

2022 2022 IEEE/CVF Conference on Computer ... 1731 citations

FFA-Net: Feature Fusion Attention Network for Single Image Dehazing

Qin Xu , Zhilin Wang , Yuanchao Bai +2 more

In this paper, we propose an end-to-end feature fusion at-tention network (FFA-Net) to directly restore the haze-free image. The FFA-Net architecture consists of three key compo...

2020 Proceedings of the AAAI Conference on... 1458 citations

Hybrid Task Cascade for Instance Segmentation

Kai Chen , Wanli Ouyang , Chen Change Loy +9 more

Cascade is a classic yet powerful architecture that has boosted performance on various tasks. However, how to introduce cascade to instance segmentation remains an open question...

2019 1411 citations

Publication Info

Year: 2019
Type: article
Pages: 1871-1880
Citations: 1042
Access: Closed

External Links

View on DOI.org

Social Impact

Altmetric

End-To-End Multi-Task Learning With Attention

PlumX Metrics

Social media, news, blog, policy document mentions

Citation Metrics

1042

OpenAlex

Cite This

APA Style

                            
                                    Shikun Liu, 
                                
                                    Edward Johns, 
                                
                                    Andrew J. Davison
                                
                            (2019). 
                            End-To-End Multi-Task Learning With Attention. 
                            
                            , 1871-1880.
                            https://doi.org/10.1109/cvpr.2019.00197

Identifiers

DOI: 10.1109/cvpr.2019.00197