volume | PIER Journals

Abstract

In the ever-evolving landscape of engineering and technology, the optimization of complex systems is a perennial challenge. Cavity filters, pivotal in Radio Frequency (RF) systems, demand precise tuning for optimal performance. This article introduces an innovative approach to automate cavity filter tuning using Q-learning, enhanced with epsilon decay. While reinforcement learning algorithms like Q-learning have shown effectiveness in complex decision-making, the exploration-exploitation trade-off remains a crucial challenge. The study conducts a thorough investigation into the application of epsilon decay in conjunction with Q-learning, employing the well-established epsilon-greedy strategy. This research focuses on systematically decaying the exploration rateε over time, aiming to strike a balance between exploring new actions and exploiting acquired knowledge. This strategic shift serves to not only refine the convergence of the Q-learning model but also remarkably elevate the overall tuning performances. Impressively, this optimization is achieved with a notable reduction in the number of tuning steps, demonstrating an efficiency boost of up to 45 steps.

1. Xie, Ya, Fu-Chang Chen, and Qing-Xin Chu, "Tunable cavity filter and diplexer using in-line dual-post resonators," IEEE Transactions on Microwave Theory and Techniques, Vol. 70, No. 6, 3188-3199, Jun. 2022.
doi:10.1109/TMTT.2022.3163420

2. Rehman, Abdul and Cristiano Tomassoni, "Spurious self-suppression method: Application to TM cavity filters," IEEE Transactions on Microwave Theory and Techniques, Vol. 71, No. 3, 1201-1215, Mar. 2023.
doi:10.1109/TMTT.2022.3222328

3. Laplanche, Etienne, Nicolas Delhote, Aurelien Perigaud, Olivier Tantot, Serge Verdeyme, Stephane Bila, Damien Pacaud, and Ludovic Carpentier, "Tunable filtering devices in satellite payloads: A review of recent advanced fabrication technologies and designs of tunable cavity filters and multiplexers using mechanical actuation," IEEE Microwave Magazine, Vol. 21, No. 3, 69-83, Mar. 2020.
doi:10.1109/MMM.2019.2958706

4. Mansour, R. R., "RF filters and diplexers for wireless system applications: State-of-the-art and trends," Radio and Wireless Conference, 2003. RAWCON’03. Proceedings, 373-376, IEEE, Boston, Massachusetts, USA, Aug. 2003.

5. Mansour, R. R., "Filter technologies for wireless base stations," IEEE Microwave Magazine, Vol. 5, No. 1, 68-74, Mar. 2004.
doi:10.1109/MMW.2004.1284945

6. Zhao, Botao and Fan Yang, "Compatibility evaluation and technical analysis of C-band broadcasting satellite receiving stations against 5G base station," Second International Conference on Digital Signal and Computer Communications (DSCC 2022), Vol. 12306, 162-168, SPIE, Changchun, China, Aug. 2022.

7. Zhao, Junyu, Chunguang Ma, Jinghan Zhou, Junhui Li, Jiaming Liu, and Yong Luo, "Design of wide stopband and high suppression cavity filter," 2023 24th International Vacuum Electronics Conference (IVEC), 1-2, IEEE, Chengdu, China, 2023.

8. Varikuntla, Krushna Kanth and Raghavan Singaravelu, "Review on design of frequency selective surfaces based on substrate integrated waveguide technology," Advanced Electromagnetics, Vol. 7, No. 5, 101-110, Nov. 2018.
doi:10.7716/aem.v7i5.751

9. Li, Mengze, Yang Yang, Francesca Iacopi, Minoru Yamada, and Jaim Nulman, "Compact multilayer bandpass filter using low-temperature additively manufacturing solution," IEEE Transactions on Electron Devices, Vol. 68, No. 7, 3163-3169, Jul. 2021.
doi:10.1109/TED.2021.3072926

10. Rehman, Hafiz Zia Ur, Hyunho Hwang, and Sungon Lee, "Conventional and deep learning methods for skull stripping in brain MRI," Applied Sciences, Vol. 10, No. 5, 1773, Mar. 2020.

11. Suryanarayana, Gunnam, Karthik Chandran, Osamah Ibrahim Khalaf, Youseef Alotaibi, Abdulmajeed Alsufyani, and Saleh Ahmed Alghamdi, "Accurate magnetic resonance image super-resolution using deep networks and gaussian filtering in the stationary wavelet domain," IEEE Access, Vol. 9, 71406-71417, 2021.
doi:10.1109/ACCESS.2021.3077611

12. Palanisamy, Satheeshkumar, Balakumaran Thangaraju, Osamah Ibrahim Khalaf, Youseef Alotaibi, and Saleh Alghamdi, "Design and synthesis of multi-mode bandpass filter for wireless applications," Electronics, Vol. 10, No. 22, 2853, Nov. 2021.
doi:10.3390/electronics10222853

13. Song, Jiajin, Biao Deng, Yingying Pang, and Liguo Sun, "A compact and high selective combline bandpass filter using GaAs IPD technology," 2019 IEEE 5th International Conference on Computer and Communications (ICCC), 309-312, IEEE, Chengdu, China, 2019.

14. Sekhri, Even, Rajiv Kapoor, and Mart Tamre, "Double deep Q-learning approach for tuning microwave cavity filters using locally linear embedding technique," 2020 International Conference Mechatronic Systems and Materials (MSM), 1‑6, IEEE, Bialystok, Poland, Jul. 2020.
doi:10.1109/msm49833.2020.9202393

15. Aghanim, Amina, Rafik Lasri, and Otman Oulhaj, "Implementation of a fuzzy controller to tune the response of a waveguide cavity filter," E-Prime --- Adv. Electr. Eng. Electron. Energy, Vol. 2, 100078, 2022.

16. Yigit, Yarkin and Onder Suvak, "Control architecture for autonomous RF cavity filter and multiplexer tuning," 2022 IEEE Autotestcon, 1‑5, IEEE, National Harbor, MD, USA, Aug. 2022.
doi:10.1109/AUTOTESTCON47462.2022.9984768

17. Lindstahl, Simon and Xiaoyu Lan, "Reinforcement learning with imitation for cavity filter tuning," 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 1335-1340, IEEE, Boston, MA, USA, Jul. 2020.

18. Amari, S. and G. Macchiarella, "Synthesis of inline filters with arbitrarily placed attenuation poles by using nonresonating nodes," IEEE Transactions on Microwave Theory and Techniques, Vol. 53, No. 10, 3075-3081, Oct. 2005.
doi:10.1109/TMTT.2005.855128

19. Thal, H. L., "Computer-aided filter alignment and diagnosis," IEEE Transactions on Microwave Theory and Techniques, Vol. 26, No. 12, 958-963, 1978.
doi:10.1109/TMTT.1978.1129528

20. Alvarez, Jesus, Luis Diaz Angulo, Amelia Rubio Bretones, and Salvador G. Garcia, "A spurious-free discontinuous galerkin time-domain method for the accurate modeling of microwave filters," IEEE Transactions on Microwave Theory and Techniques, Vol. 60, No. 8, 2359-2369, Aug. 2012.
doi:10.1109/TMTT.2012.2202683

21. Sadrossadat, Sayed Alireza, Yazi Cao, and Qi-Jun Zhang, "Parametric modeling of microwave passive components using sensitivity-analysis-based adjoint neural-network technique," IEEE Transactions on Microwave Theory and Techniques, Vol. 61, No. 5, 1733-1747, May 2013.
doi:10.1109/TMTT.2013.2253793

22. Bi, Leyu, Weihua Cao, Wenkai Hu, and Min Wu, "A dynamic-attention-based heuristic fuzzy expert system for the tuning of microwave cavity filters," IEEE Transactions on Fuzzy Systems, Vol. 30, No. 9, 3695-3707, Sep. 2022.
doi:10.1109/TFUZZ.2021.3124643

23. Peng, Sai, Weihua Cao, Leyu Bi, Yan Yuan, and Min Wu, "A tuning strategy for microwave filter using variable universe adaptive fuzzy logic system," 2021 China Automation Congress (CAC), 6061--6066, Oct. 2021.

24. Yao, Shi, Xing-Chang Wei, and Li Ding, "A deembedding method for the S-parameter extraction of surface-mounted devices with asymmetric fixtures," IEEE Microwave and Wireless Components Letters, Vol. 31, No. 2, 211-214, Feb. 2021.
doi:10.1109/LMWC.2020.3034124

25. Yigit, Yarkin and Engin Afacan, "Autonomous RF cavity filter tuning," IEEE Instrumentation & Measurement Magazine, Vol. 26, No. 5, 39-44, Aug. 2023.
doi:10.1109/MIM.2023.10208248

26. Nian, Rui, Jinfeng Liu, and Biao Huang, "A review on reinforcement learning: introduction and applications in industrial process control," Computers & Chemical Engineering, Vol. 139, 106886, Aug. 2020.
doi:10.1016/j.compchemeng.2020.106886

27. Mlika, Zoubeir and Soumaya Cherkaoui, "Network slicing with MEC and deep reinforcement learning for the internet of vehicles," IEEE Network, Vol. 35, No. 3, 132-138, May 2021.
doi:10.1109/MNET.011.2000502

28. Akalin, Neziha and Amy Loutfi, "Reinforcement learning approaches in social robotics," Sensors, Vol. 21, No. 4, Feb. 2021.
doi:10.3390/s21041292

29. Clifton, J. and E. Laber, "Q-Learning: Theory and applications," Annu. Rev. Stat. Its Appl., Vol. 7, No. 1, 279‑301, 2020.
doi:doi: 10.1146/annurev-statistics-031219-041220

30. Ding, Zihan, Yanhua Huang, Hang Yuan, and Hao Dong, "Introduction to reinforcement learning," Deep Reinforcement Learning: Fundamentals, Research and Applications, 47-123, 2020.
doi:10.1007/978-981-15-2081-5_4

31. Wilson, Robert C., Elizabeth Bonawitz, Vincent D. Costa, and R. Becket Ebitz, "Balancing exploration and exploitation with information and randomization," Current Opinion in Behavioral Sciences, Vol. 38, No. SI, 49-56, Apr. 2021.
doi:10.1016/j.cobeha.2020.10.001

Ms. Amina Aghanim

Dr. Hamid Chekenbah

Dr. Otman Oulhaj

Dr. Rafik Lasri