Computer Science Thesis Proposal
- Gates Hillman Centers
- Traffic21 Classroom 6501
- PETAR STOJANOV
- Ph.D. Student
- Computer Science Department
- Carnegie Mellon University
Towards More Efficient and Data-Driven Domain Adaptation
In recent years with the fast progress made in neural networks research, supervised machine learning approaches have become increasingly powerful predicting target variable Y from input features X. However, most of these complex models require large amounts of data to train, and often work under the assumption that the data points are i.i.d. In reality these assumptions are very likely to be violated. A simplified notion of this violation is when the labeled training and test data come from different joint distributions. Domain adaptation is the process of accounting for this distributional difference under specific assumptions, for the purpose of better prediction performance in the test data. In this thesis, we focus on three main sub-problems of domain adaptation:
The first sub-problem is single-source domain adaptation, in which we have one labeled source domain and one unlabeled target domain, specifically under the covariate shift setting. In this setting, the assumption is that the marginal distribution of the features X changes across domains, while the conditional distribution of the target variable Y given X stays the same. We study this problem in detail and develop a low-dimensional density ratio estimation method for covariate shift correction, which makes use of the relationships between X and Y.
The second sub-problem is multiple-source domain adaptation, in which we are given multiple labeled source domains, with multiple respective joint distributions, where we only observe the features X in the target domain. We develop techniques, which make use of kernel methods to extract the change of the distribution across domains and make use of it for prediction in the target domain. We plan to extend this methodology to make use of latent features produced by deep architectures.
The third sub-problem is heterogeneous domain adaptation, in which the source and the target domain are not in the same feature space. However, the different domains can share a common latent structure that can be used to align the domains in order to perform prediction in the target domain. In this thesis, we explore this reasoning for addressing the problem of heterogeneous domain adaptation, along with some potential applications.
Jaime Carbonell (Chair)
Aapo Hyvärinen (University of Helsinki, Finland)
Copy of Thesis Summary