Abstract
In this paper the problem of learning appropriate bias for an environment of related tasks is examined from a Bayesian perspective. The environment of related tasks is shown to be naturally modelled by the concept of an {\em objective} prior distribution. Sampling from the objective prior corresponds to sampling different learning tasks from the environment. It is argued that for many common machine learning problems, although we don't know the true (objective) prior for the problem, we do have some idea of a set of possible priors to which the true prior belongs. It is shown that under these circumstances a learner can use Bayesian inference to learn the true prior by sampling from the objective prior. Bounds are given on the amount of information required to learn a task when it is simultaneously learnt with several other tasks. The bounds show that if the learner has little knowledge of the true prior, and the dimensionality of the true prior is small, then sampling multiple tasks is highly advantageous.
Keywords
CitationNinthComputer scienceBayesian probabilityBayesian inferenceLibrary scienceArtificial intelligenceInformation retrieval
Affiliated Institutions
Related Publications
Making Large-Scale SVM Learning Practical
Training a support vector machine SVM leads to a quadratic optimization problem with bound constraints and one linear equality constraint. Despite the fact that this type of pro...
What makes a good model of natural images?
Many low-level vision algorithms assume a prior probability over images, and there has been great interest in trying to learn this prior from examples. Since images are very non...
A Survey on Policy Search for Robotics
<p>Policy search is a subfield in reinforcement learning which focuses onfinding good parameters for a given policy parametrization. It is wellsuited for robotics as it ca...
Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modeling
In the past ten years there has been a dramatic increase of interest in the Bayesian analysis of finite mixture models. This is primarily because of the emergence of Markov chai...
A comparison of Bayesian and likelihood-based methods for fitting multilevel models
We use simulation studies, whose design is realistic for educational and medical\nresearch (as well as other fields of inquiry), to compare Bayesian and likelihood-based\nmethod...
Publication Info
- Year
- 1996
- Type
- preprint
- Pages
- 77-88
- Citations
- 17
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
17
OpenAlex
Cite This
Jonathan Baxter
(1996).
A Bayesian/information theoretic model of bias learning.
, 77-88.
https://doi.org/10.1145/238061.238071
Identifiers
- DOI
- 10.1145/238061.238071