Skip to main content
SpringerLink
Log in

Patch Mix Augmentation with Dual Encoders for Meta-Learning

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13623))

Included in the following conference series:

  • 1294 Accesses

Abstract

Meta-learning aims to learn models that can make quick adaptations to new tasks. However, due to the lack of data, the further improvement of meta-learning can be severely constrained. Since, data augmentation has been a commonly used method to help models reach state-of-art performance in various image classification tasks. It is wise to use data augmentation methods in meta-learning. Different strategies for applying data augmentation to meta-learning have emerged. One common combination of data augmentation and meta-learning is performing different transformations on images. Other methods use generative models, such as GAN, VAE, or AE, to generate samples and expand the data set. In this paper, we proposed a novel data augmentation method aiming to enlarge the number of samples in the support sets. Our approach uses wavelet transform, a widely used method in signal analysis and processing and style mix from AdaIn. Furthermore, we use both ResNet and ViT as our feature encoder. Combining with the idea of contrastive learning, we train our ViT in an unsupervised way. Experimental results show that we achieve a decent performance improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. Artif. Intell. Rev. 18(2), 77–95 (2002)

    Article  Google Scholar 

  2. Vanschoren, J.: Meta-learning: A survey. arXiv preprint arXiv:1810.03548 (2018)

  3. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural. Inf. Process. Syst. 25, 1097–1105 (2012)

    Google Scholar 

  4. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  5. Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  7. Ni, R., Goldblum, M., Sharaf, A., Kong, K., Goldstein, T.: Data augmentation for meta-learning. In: International Conference on Machine Learning, pp. 8152–8161. PMLR (2021)

    Google Scholar 

  8. Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: Cutmix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019)

    Google Scholar 

  9. Zhang, H., Cissé, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net (2018)

    Google Scholar 

  10. Verma, V., et al.: Manifold mixup: Better representations by interpolating hidden states. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning, ICML 2019. Proceedings of Machine Learning Research, vol. 97, pp. 6438–6447. PMLR (2019)

    Google Scholar 

  11. Zhang, R., Che, T., Ghahramani, Z., Bengio, Y., Song, Y.: Metagan: an adversarial approach to few-shot learning. In: NeurIPS, vol. 2, p. 8 (2018)

    Google Scholar 

  12. Lee, D.B., Min, D., Lee, S., Hwang, S.J.: Meta-GMVAE: mixture of gaussian VAE for unsupervised meta-learning. In: International Conference on Learning Representations (2020)

    Google Scholar 

  13. Huang, X., Belongie, S.J.: Arbitrary style transfer in real-time with adaptive instance normalization. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24–26, 2017, Workshop Track Proceedings (2017)

    Google Scholar 

  14. Fu, Y., Xie, Y., Fu, Y., Chen, J., Jiang, Y.G.: Wave-san: Wavelet based style augmentation network for cross-domain few-shot learning. arXiv preprint arXiv:2203.07656 (2022)

  15. Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In: 9th International Conference on Learning Representations, ICLR 2021 (2021)

    Google Scholar 

  16. van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. CoRR abs/1807.03748 (2018)

    Google Scholar 

  17. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.B.: Momentum contrast for unsupervised visual representation learning. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13–19, 2020, pp. 9726–9735 (2020)

    Google Scholar 

  18. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR (2017)

    Google Scholar 

  19. Rusu, A.A., et al.: Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.05960 (2018)

  20. Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning (2016)

    Google Scholar 

  21. Li, Z., Zhou, F., Chen, F., Li, H.: Meta-SGD: learning to learn quickly for few-shot learning. arXiv preprint arXiv:1707.09835 (2017)

  22. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. Adv. Neural. Inf. Process. Syst. 29, 3630–3638 (2016)

    Google Scholar 

  23. Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1199–1208 (2018)

    Google Scholar 

  24. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017)

    Google Scholar 

  25. Ni, R., Goldblum, M., Sharaf, A., Kong, K., Goldstein, T.: Data augmentation for meta-learning. CoRR abs/2010.07092 (2020)

    Google Scholar 

  26. Schwartz, E., et al.: Delta-encoder: an effective sample synthesis method for few-shot object recognition. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  27. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset. Technical report CNS-TR-2011-001, California Institute of Technology (2011)

    Google Scholar 

  28. Wang, H., Deng, Z.H.: Cross-domain few-shot classification via adversarial task augmentation. arXiv preprint arXiv:2104.14385 (2021)

  29. Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3D object representations for fine-grained categorization. In: 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13). Sydney, Australia (2013)

    Google Scholar 

  30. Sun, Q., Liu, Y., Chua, T.S., Schiele, B.: Meta-transfer learning for few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 403–412 (2019)

    Google Scholar 

  31. Mishra, N., Rohaninejad, M., Chen, X., Abbeel, P.: A simple neural attentive meta-learner. arXiv preprint arXiv:1707.03141 (2017)

  32. Lee, K., Maji, S., Ravichandran, A., Soatto, S.: Meta-learning with differentiable convex optimization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10657–10665 (2019)

    Google Scholar 

  33. Chen, Y., Wang, X., Liu, Z., Xu, H., Darrell, T.: A new meta-baseline for few-shot learning. arXiv preprint arXiv:2003.04390 (2020)

  34. Tseng, H.Y., Lee, H.Y., Huang, J.B., Yang, M.H.: Cross-domain few-shot classification via learned feature-wise transformation. In: International Conference on Learning Representations (2020)

    Google Scholar 

  35. Sun, J., Lapuschkin, S., Samek, W., Zhao, Y., Cheung, N.M., Binder, A.: Explanation-guided training for cross-domain few-shot classification. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 7609–7616 (2021)

    Google Scholar 

  36. Wang, H., Deng, Z.H.: Cross-domain few-shot classification via adversarial task augmentation. In: Zhou, Z.H. (ed.) Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp. 1075–1081, August 2021. main Track

    Google Scholar 

Download references

Acknowledgment

This work is supported by the National Key R &D Program of China (2018YFA0701700; 2018YFA0701701), and the National Natural Science Foundation of China under Grant No. 61672364, No. 62176172 and No. 61902269.

Author information

Authors and Affiliations

  1. School of Computer Science and Technolgy, Soochow University, Suzhou, China

    Hong Yu & Fanzhang Li

Authors
  1. Hong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fanzhang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fanzhang Li .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Indore, Indore, India

    Mohammad Tanveer

  2. Indian Institute of Information Technology - Allahabad, Prayagraj, India

    Sonali Agarwal

  3. Kobe University, Kobe, Japan

    Seiichi Ozawa

  4. Indian Institute of Technology Patna, Patna, India

    Asif Ekbal

  5. University of Innsbruck, Innsbruck, Austria

    Adam Jatowt

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yu, H., Li, F. (2023). Patch Mix Augmentation with Dual Encoders for Meta-Learning. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13623. Springer, Cham. https://doi.org/10.1007/978-3-031-30105-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-30105-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-30104-9

  • Online ISBN: 978-3-031-30105-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation