Abstract

The effectiveness of knowledge transfer using classification algorithms depends on the difference between the distribution that generates the training examples and the one from which test examples are to be drawn. The task can be especially difficult when the training examples are from one or several domains different from the test domain. In this paper, we propose a locally weighted ensemble framework to combine multiple models for transfer learning, where the weights are dynamically assigned according to a model's predictive power on each test example. It can integrate the advantages of various learning algorithms and the labeled information from multiple training domains into one unified classification model, which can then be applied on a different domain. Importantly, different from many previously proposed methods, none of the base learning method is required to be specifically designed for transfer learning. We show the optimality of a locally weighted ensemble framework as a general approach to combine multiple models for domain transfer. We then propose an implementation of the local weight assignments by mapping the structures of a model onto the structures of the test domain, and then weighting each model locally according to its consistency with the neighborhood structure around the test example. Experimental results on text classification, spam filtering and intrusion detection data sets demonstrate significant improvements in classification accuracy gained by the framework. On a transfer learning task of newsgroup message categorization, the proposed locally weighted ensemble framework achieves 97% accuracy when the best single model predicts correctly only on 73% of the test examples. In summary, the improvement in accuracy is over 10% and up to 30% across different problems.

Keywords

Computer scienceTransfer of learningWeightingArtificial intelligenceMachine learningDomain (mathematical analysis)CategorizationConsistency (knowledge bases)Ensemble learningData miningTest dataTask (project management)Mathematics

Affiliated Institutions

Related Publications

Publication Info

Year
2008
Type
article
Pages
283-291
Citations
328
Access
Closed

Social Impact

Social media, news, blog, policy document mentions

Citation Metrics

328
OpenAlex
32
Influential

Cite This

Jing Gao, Wei Fan, Jing Jiang et al. (2008). Knowledge transfer via multiple model local structure mapping. Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining , 283-291. https://doi.org/10.1145/1401890.1401928

Identifiers

DOI
10.1145/1401890.1401928

Data Quality

Data completeness: 81%