State-of-the-art approaches in computer vision community are mainly rely on Deep-Nets, which is highly data-driven. However, high quality dataset is hard to obtain as the annotation effort is quite expensive and in some special case, even the required image is hard to find, like images of rare wild animals.
In a more realistic scenario , if we want to know whether model trained in sunny days are still robust for images taken in rainy days or during late night. To answer this question, we would need datasets taken in different weather conditions along with the annotation, which is quite expensive to obtain.
Therefore, we plan to use image-to-image translation technique to generate such images, the goal is to build an generative model that maps an image from sunny day to late night but keep the objects unchanged, including locations and poses, therefore, the annotation is the same as the original one.
We can start from several widely-used approaches, including CycleGAN [2], UNIT [1] and so on, example images are shown at above figure. The challenge here is that, current training dataset for cross-season image-to-image translation are mainly street views, in order to make it work for images from different domains, we may need to solve this problem with domain adaption techniques, this would be the contribution of our work.
References:
[1] M. Liu, T. Breuel, and J. Kautz. Unsupervised Image-to-Image Translation Networks. In Advances in
Neural Information Processing Systems, 2017.
[2] J.-Y. Zhu, T. Park, P. Isola, and A.A. Efros. Unpaired Image-To-Image Translation Using
Cycle-Consistent Adversarial Networks. In International Conference on Computer Vision, 2017.
Back to the project list.
Proficiency in Python, especially PyTorch. Knowledge of machine learning, deep-learning. Basic knowledge in generative model and domain adaption is a plus.
20% Theory, 30% Implementation, 50% Research and Experiments