Bo Wang
ABSTRACT
Deep neural network based transfer learning has been widely used to leverage information from the domain with rich data to help domain with insufficient data. When the source data distribution is different from the target data, transferring knowledge between these domains may lead to negative transfer. To mitigate this problem, a typical way is to select useful source domain data for transferring. However, limited studies focus on selecting high-quality source data to help neural network based transfer learning. To bridge this gap, we propose a general Minimax Game based model for selective Transfer Learning (MGTL). More specifically, we build a selector, a discriminator and a TL module in the proposed method. The discriminator aims to maximize the differences between selected source data and target data, while the selector acts as an attacker to selected source data that are close to the target to minimize the differences. The TL module trains on the selected data and provides rewards to guide the selector. Those three modules play a minimax game to help select useful source data for transferring. Our method is also shown to speed up the training process of the learning task in the target domain than traditional TL methods. To the best of our knowledge, this is the first to build a minimax game based model for selective transfer learning. To examine the generality of our method, we evaluate it on two different tasks: item recommendation and text retrieval. Extensive experiments over both public and real-world datasets demonstrate that our model outperforms the competing methods by a large margin. Meanwhile, the quantitative evaluation shows our model can select data which are close to target data. Our model is also deployed in a real-world system and significant improvement over the baselines is observed.
- Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil. 2007. Multitask feature learning. In NIPS.Google Scholar
- John Blitzer, Mark Dredze, and Fernando Pereira. 2007. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. ACL (2007).Google Scholar
- Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Michael I. Jordan. 2017. Partial Transfer Learning with Selective Adversarial Networks. CoRR (2017).Google Scholar
- Minmin Chen, Kilian Q. Weinberger, and John C. Blitzer. 2011. Co-training for Domain Adaptation. In NIPS. Google ScholarDigital Library
- Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah. 2016. Wide & Deep Learning for Recommender Systems. CoRR abs/1606.07792 (2016).Google Scholar
- Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. In RecSys '16. 191--198. Google ScholarDigital Library
- Wenyuan Dai, Qiang Yang, Gui-Rong Xue, and Yong Yu. 2007. Boosting for Transfer Learning. In ICML. 193--200. Google ScholarDigital Library
- Hal Daume III. 2007. Frustratingly Easy Domain Adaptation. In ACL.Google Scholar
- Yang Fan, Fei Tian, Tao Qin, Jiang Bian, and Tie-Yan Liu. 2017. Learning What Data to Learn. CoRR (2017).Google Scholar
- Meng Fang, Yuan Li, and Trevor Cohn. 2017. Learning how to Active Learn: A Deep Reinforcement Learning Approach. In EMNLP.Google Scholar
- Jun Feng, Minlie Huang, Li Zhao, Yang Yang, and Xiaoyan Zhu. 2018. Reinforcement Learning for Relation Classification From Noisy Data. In AAAI.Google Scholar
- Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial Training of Neural Networks. J. Mach. Learn. Res. 17, 1 (Jan. 2016), 2096--2030. Google ScholarDigital Library
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In NIPS. Google ScholarDigital Library
- Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. CoRR abs/1703.04247 (2017).Google Scholar
- Xiangnan He and Tat-Seng Chua. 2017. Neural Factorization Machines for Sparse Predictive Analytics. In Proceedings of SIGIR. 355--364. Google ScholarDigital Library
- Xiangnan He, Zhankui He, Xiaoyu Du, and Tat-Seng Chua. 2018. Adversarial personalized ranking for recommendation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 355--364. Google ScholarDigital Library
- Jiayuan Huang, Alexander J. Smola, Arthur Gretton, Karsten M. Borgwardt, and Bernhard. Scholkopf. 2006. Correcting Sample Selection Bias by Unlabeled Data. In NIPS. (2006). Google ScholarDigital Library
- Ferenc Huszar. 2015. How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary? arXiv:1511.05101. (2015).Google Scholar
- Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. SciTail: A Textual Entailment Dataset from Science Question Answering. In AAAI.Google Scholar
- Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial Multi-task Learning for Text Classification. In Proceedings of ACL. (2017).Google ScholarCross Ref
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin A. Riedmiller, Andreas Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature (2015).Google Scholar
- Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. 2016. Natural Language Inference by Tree-Based Convolution and Heuristic Matching. In ACL.Google Scholar
- Lili Mou, Zhao Meng, Rui Yan, Ge Li, Yan Xu, Lu Zhang, and Zhi Jin. 2016. How Transferable are Neural Networks in NLP Applications?. In EMNLP.Google Scholar
- Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on knowledge and data engineering (2010), 1345--1359. Google ScholarDigital Library
- Ankur P. Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A Decomposable Attention Model for Natural Language Inference. In EMNLP.Google Scholar
- Yash Patel, Kashyap Chitta, and Bhavan Jasani. {n. d.}. Learning Sampling Policies for Domain Adaptation. CoRR, abs/1805.07641, 2018. ({n. d.}).Google Scholar
- Chen Qu, Feng Ji, Minghui Qiu, Liu Yang, Zhiyu Min, Haiqing Chen, Jun Huang, and W. Bruce Croft. 2019. Learning to Selectively Transfer: Reinforced Transfer Learning for Deep Text Matching. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (WSDM '19). Google ScholarDigital Library
- Michael T. Rosenstein, Zvika Marx, Leslie Pack Kael-bling, and Thomas G.Dietterich. 2005. To Transfer or Not To Transfer. NIPS Workshop on Inductive Transfer (2005).Google Scholar
- Sebastian Ruder and Barbara Plank. 2017. Learning to select data for transfer learning with Bayesian Optimization. In EMNLP. (2017).Google ScholarCross Ref
- Gavin A. Rummery and Mahesan Niranjan. 1994. OnLine Q-Learning Using Connectionist Systems. Technical Report. University of Cambridge.Google Scholar
- Tobias Schnabel and Hinrich SchuÌtze. 2014. FLORS: Fast and Simple Domain Adaptation for Part-of-Speech Tagging. TACL, 2:15-26. (2014).Google Scholar
- Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. 2018. Wasserstein Distance Guided Representation Learning for Domain Adaptation. In AAAI. AAAI Press.Google Scholar
- Richard S. Sutton and Andrew G. Barto. 1998. Reinforcement Learning - An Introduction. MIT Press. Google ScholarDigital Library
- Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial Discriminative Domain Adaptation. CoRR abs/1702.05464 (2017).Google Scholar
- ChangWang and Sridhar Mahadevan. 2008. Manifold alignment using procrustes analysis. In ICML.Google Scholar
- Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. 2017. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of SIGIR. ACM, 515--524. Google ScholarDigital Library
- RuoxiWang, Bin Fu, Gang Fu, and MingliangWang. 2017. Deep & Cross Network for Ad Click Predictions. In Proceedings of the ADKDD'17. 12:1--12:7. Google ScholarDigital Library
- TianyangWang, Jun Huan, and Michelle Zhu. 2018. Instance-based Deep Transfer Learning. In WACV.Google Scholar
- Junfeng Wen, Chun-Nam Yu, and Russell Greiner. 2014. Robust Learning Under Uncertain Test Distributions: Relating Covariate Shift to Model Misspecification (ICML'14). JMLR.org, II--631--II--639. Google ScholarDigital Library
- Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In NAACL.Google Scholar
- Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning (1992) (1992). Google ScholarDigital Library
- Jiawei Wu, Lei Li, and William Yang Wang. 2018. Reinforced Co-Training. In NAACL.Google Scholar
- Zhilin Yang, Ruslan Salakhutdinov, and William W. Cohen. 2017. Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks. In ICLR (2017).Google ScholarDigital Library
- Wenpeng Yin, Hinrich Schütze, Bing Xiang, and Bowen Zhou. 2016. ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs. TACL (2016).Google Scholar
- Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. In AAAI. (2017). Google ScholarDigital Library
- Fuzhen Zhuang, Lang Huang, Jia He, Jixin Ma, and Qing He. 2017. Transfer Learning with Manifold Regularized Convolutional Neural Network, Gang Li, Yong Ge, Zili Zhang, Zhi Jin, and Michael Blumenstein (Eds.). Springer International Publishing, Cham, 483--494.Google Scholar
Index Terms
- A Minimax Game for Instance based Selective Transfer Learning
Recommendations
Learning to Selectively Transfer: Reinforced Transfer Learning for Deep Text Matching
WSDM '19: Proceedings of the Twelfth ACM International Conference on Web Search and Data MiningDeep text matching approaches have been widely studied for many applications including question answering and information retrieval systems. To deal with a domain that has insufficient labeled data, these approaches can be used in a Transfer Learning (...
Double-bootstrapping source data selection for instance-based transfer learning
Instance-based transfer is an important paradigm for transfer learning, where data from related tasks (source data) are combined with the data for the current learning task (target data) to train a learner for the current (target) task. However, in most ...
Instance-based transfer learning method via modified domain-adversarial neural network with influence function: Applications to design metamodeling and fault diagnosis
AbstractThe availability of a large amount of high-quality data is critical to the performance of machine-learning models. It is challenging to obtain a training dataset because data collection is costly and time-consuming. However, data ...
Graphical abstractDisplay Omitted
Highlights- This study explores an instance-based transfer learning method for surrogate-model and fault diagnosis.
Comments