Abstract
Current unsupervised domain adaptation methods for intelligent fault diagnosis focus on learning domain-invariant representations under covariate shift assumption. However, the covariate shift assumption is usually unsatisfied when each fault class in different domains consists of multiple modes with skewed proportions, which is common in industrial scenarios. Imbalanced data from multiple modes cause the presence of within-domain class imbalance and between-domain label distributional shift. This paper introduces a novel subpopulation shift that further considers the domain shift from a subpopulation perspective, i.e., that covariate shift and label distributional shift across domains are caused by shifts in the multiple modes. To address this issue, a label propagation-based unsupervised domain adaptation is proposed based on a realistic expansion assumption. We apply the theoretical analysis of the proposed method with a bi-level optimization strategy adapted from meta-learning. Using joint optimization of a teacher model and a student model, the label propagation-based model-agnostic meta-learning (LPMAML) not only propagates supervision information from the source to the target but also adjusts the teacher’s strategy throughout the student’s learning process. To alleviate the noise caused by label distributional shift, we integrate a sampling-based alignment method that aligns the empirical label distributions across the two domains into LPMAML. Experimental results on three bearing datasets show that the proposed method has impressive generalization ability under covariate, label distributional, and subpopulation shifts. The proposed method offers consistent improvements to unsupervised domain adaptation (UDA) methods. Compared with the vanilla UDA methods, the average diagnosis accuracies of the proposed method on the label distributional shift benchmark and subpopulation shift benchmark are improved by 8.21% and 7.63%, respectively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets analyzed during the current study are partly available from https://mb.uni-paderborn.de/kat/forschung/kat-datacenter/bearing-datacenter/data-sets-and-download, https://engineering.case.edu/bearingdatacenter/download-data-file, and partly available from the corresponding author on reasonable request.
References
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010). A theory of learning from different domains. Machine Learning, 79(1), 151–175. https://doi.org/10.1007/s10994-009-5152-4
Cai, T., Gao, R., Lee, J., & Lei, Q. (2021). A theory of label propagation for subpopulation shift. In International Conference on Machine Learning, PMLR (Vol. 139, pp. 1170–1182).
Chen, Q., Liu, Y., Wang, Z., Wassell, I., & Chetty, K. (2018). Re-weighted adversarial adaptation network for unsupervised domain adaptation. In: IEEE Conference on Computer Vision and Pattern Recognition (pp. 7976–7985). https://doi.org/10.1109/CVPR.2018.00832
Chen, X., Zhang, B., & Gao, D. (2021). Bearing fault diagnosis base on multi-scale CNN and LSTM model. Journal of Intelligent Manufacturing, 32(4), 971–987. https://doi.org/10.1007/s10845-020-01600-2
Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, PMLR, (Vol. 70, pp. 1126–1135).
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2016). Domain-adversarial training of neural networks. Journal of Machine Learning Research, 17(1), 2096–2030.
Huang, Z., Shao, J., Zhu, J., Zhang, W., & Li, X. (2023). Tool wear condition monitoring across machining processes based on feature transfer by deep adversarial domain confusion network. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-023-02088-2
Iscen, A., Tolias, G., Avrithis, Y., & Chum, O. (2019). Label propagation for deep semi-supervised learning. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 5065–5074). https://doi.org/10.1109/CVPR.2019.00521
ISO 15243. (2004). Rolling bearings-damages and failures-terms characteristics and causes.
Jin, X., He, T., Shen, X., Wu, S., Liu, T., Wang, X., Huang, J., Chen, Z., & Hua, X. S. (2022). Unleash the potential of adaptation models via dynamic domain labels. In International Conference on Learning Representations.
Kingma, D. P., & Ba, J. (2015). Adam: a method for stochastic optimization. In International Conference on Learning Representations.
LeCun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., & Jackel, L. (1989). Handwritten digit recognition with a back-propagation network. Advances in Neural Information Processing Systems, 2.
Lessmeier, C., Kimotho, J. K., Zimmer, D., & Sextro, W. (2016). Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: a benchmark data set for data-driven classification. In PHM Society European Conference (Vol. 3).
Long, M., Cao, Y., Wang, J., & Jordan, M. (2015). Learning transferable features with deep adaptation networks. In International Conference on Machine Learning, PMLR, (Vol. 37, pp. 97–105).
Long, M., Cao, Z., Wang, J., & Jordan, M. I. (2018). Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems, 31, 1640–1650.
Long, M., Zhu, H., Wang, J., & Jordan, M. I. (2017). Deep transfer learning with joint adaptation networks. In International Conference on Machine Learning, PMLR (pp. 2208–2217).
Lvd, Maaten, & Hinton, G. (2008). Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov), 2579–2605.
Müller, R., Kornblith, S., & Hinton, G. E. (2019). When does label smoothing help? Advances in Neural Information Processing Systems, 32.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., & Antiga, L., et al. (2019). Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (pp. 8024–8035). Curran Associates, Inc.
Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747.
Smith, W. A., & Randall, R. B. (2015). Rolling element bearing diagnostics using the case western reserve university data: A benchmark study. Mechanical Systems and Signal Processing, 64–65, 100–131. https://doi.org/10.1016/j.ymssp.2015.04.021
Wang, Z., He, X., Yang, B., & Li, N. (2022). Subdomain adaptation transfer learning network for fault diagnosis of roller bearings. IEEE Transactions on Industrial Electronics, 69(8), 8430–8439. https://doi.org/10.1109/TIE.2021.3108726
Wei, C., Shen, K., Chen, Y., & Ma, T. (2020). Theoretical analysis of self-training with deep networks on unlabeled data. In International Conference on Learning Representations.
Zhao, H., Des Combes, R. T., Zhang, K., & Gordon, G. (2019). On learning invariant representations for domain adaptation. In International Conference on Machine Learning, PMLR (pp. 7523–7532).
Zhao, Z., Li, T., Wu, J., Sun, C., Wang, S., Yan, R., & Chen, X. (2020). Deep learning algorithms for rotating machinery intelligent diagnosis: An open source benchmark study. ISA Transactions, 107, 224–255. https://doi.org/10.1016/j.isatra.2020.08.010
Zhao, Z., Zhang, Q., Yu, X., Sun, C., Wang, S., Yan, R., & Chen, X. (2021). Applications of unsupervised deep transfer learning to intelligent fault diagnosis: A survey and comparative study. IEEE Transactions on Instrumentation and Measurement, 70, 1–28. https://doi.org/10.1109/TIM.2021.3116309
Acknowledgements
This work was supported in part by the National Key Research and Development Program of China under Grant 2022YFB3304602 and the National Nature Science Foundation of China under Grant 62003344.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, H., Li, Y., Bai, X. et al. Label propagation-based unsupervised domain adaptation for intelligent fault diagnosis. J Intell Manuf (2023). https://doi.org/10.1007/s10845-023-02186-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10845-023-02186-1