A theory of learning from different domains

Теория обучения на данных из различных областей
Jennifer Wortman Vaughan, Fernando M. B. Pereira, Shai Ben-David, John C. Blitzer, Koby Crammer, Alex Kulesza
2009-10-22

classifier-induced divergencedomain adaptationempirical source and target errorssource and target domainstarget error bounds
Discriminative learning methods for classification perform well when training and test data are drawn from the same distribution. Often, however, we have plentiful labeled training data from a source domain but wish to learn a classifier which performs well on a target domain with a different distribution and little or no labeled training data. In this work we investigate two questions. First, under what conditions can a classifier trained from source data be expected to perform well on target data? Second, given a small amount of labeled target data, how should we combine it during training with the large amount of labeled source data to achieve the lowest target error at test time? We address the first question by bounding a classifier’s target error in terms of its source error and the divergence between the two domains. We give a classifier-induced divergence measure that can be estimated from finite, unlabeled samples from the domains. Under the assumption that there exists some hypothesis that performs well in both domains, we show that this quantity together with the empirical source error characterize the target error of a source-trained classifier. We answer the second question by bounding the target error of a model which minimizes a convex combination of the empirical source and target errors. Previous theoretical work has considered minimizing just the source error, just the target error, or weighting instances from the two domains equally. We show how to choose the optimal combination of source and target error as a function of the divergence, the sample sizes of both domains, and the complexity of the hypothesis class. The resulting bound generalizes the previously studied cases and is always at least as tight as a bound which considers minimizing only the target error or an equal weighting of source and target errors.
1
Assuming a hypothesis performs well in both domains, empirical source error plus the domain divergence characterizes the target error of a source-trained classifier.
2
For limited labeled target data, the paper derives bounds for minimizing a convex combination of empirical source and target errors.
3
It introduces a classifier-induced divergence that can be estimated from finite unlabeled samples drawn from both domains.
4
The optimal source–target weighting depends on domain divergence, sample sizes, and hypothesis-class complexity, and yields bounds at least as tight as target-only or equal-weighting approaches.
5
The paper bounds target-domain classification error using source-domain error and a divergence between source and target distributions.

classification learning across source and target domains with different data distributions

conditions and optimal source–target error weighting governing target-domain generalization performance

Publication Details
Publication Date
2009-10-22
Journal
Publisher
ISSN
Cited by
3711
Access Type
Author Information
Authors
Jennifer Wortman Vaughan
Fernando M. B. Pereira
Shai Ben-David
John C. Blitzer
Koby Crammer
Alex Kulesza
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%