| dc.description.abstract | With rapid development in technology and ubiquitous presence of diverse types of sensors, a large
amount of data from different modalities (e.g., text, audio, images etc.) describing the same person/
object/event has become easily available. Similarly, multiple datasets targeted towards the same
task but exhibiting different data distributions are often available. To be able to learn and utilize
the complementary information present across diverse domains can be immensely valuable towards
building more intelligent models. Cross-modal learning and domain adaptation techniques are closely
related to learning under such scenarios. In this thesis, we investigate and provide novel algorithms
for two applications of learning across domains - namely Text-based Person Search and Multi-Source
Domain Adaptation.
Person search in a camera network is an important problem in the field of intelligent video surveillance.
Often the search query comes in the form of unstructured textual description of the target of
interest, and the goal is to retrieve the pedestrian images that best match this description. In the first
part of the thesis, we investigate methods for this cross-modal retrieval problem of Text-based Person
Search. Existing methods utilize class-id information to get discriminative and identity-preserving
features. However, it is not well-explored whether it is beneficial to explicitly ensure that the semantics
of the data are also retained. In the proposed work, we aim to create semantics-preserving
embeddings through an additional task of attribute prediction. Since attribute annotation is typically
unavailable in text-based person search, we first mine them from the text corpus. These attributes are
then used as a means to bridge the modality gap between the image-text inputs, as well as to improve
the representation learning. In summary, we propose an approach for text-based person search by
learning an attribute-driven space along with a class-information driven space, and utilize both for
obtaining the retrieval results. Our experiments show that learning the attribute space not only helps
in improving performance but also yields humanly-interpretable features.
In the second part of the thesis, we worked on Multi-Source Domain Adaptation, a problem involving
multiple data sources, which are of the same modality but follow different distributions. Domain
adaptation is a field of machine learning that aims at learning a model from a labelled source dataset,
such that the model performs well on samples drawn from an unlabelled target domain which has
iv
Abstract
a related but different distribution. The problem of single-source unsupervised domain adaptation
has been explored quite extensively. However, in practice, labelled data is often available from multiple,
differently distributed sources - giving rise to the problem of multi-source domain adaptation
(MSDA). Recent works in MSDA propose to learn a domain-invariant space for the sources and the
target. However, such methods treat each source to be equally relevant and are not sensitive to the
intrinsic relations amongst domains. In this work, we provide a novel algorithm for multi-source domain
adaptation which utilizes the multiple sources based on their relative importance to the target.
Our objective is to dynamically explore the relevance of sources, and then to perform weighted alignment
of domains. We experimentally validate the performance of our method on benchmark datasets,
and achieve state-of-the-art results on Office-Home and Office-Caltech. | en_US |
