Detachment Risk Constrained Soft Actor Critic for Adhesion Force Control of Magnetic Wall Climbing Robots
Authors
Živojka Janković, Branka Čolaković, Čedomirka RadoševićFiles
Abstract
Magnetic adhesion control must balance contact maintenance against locomotion resistance. This paper presents a soft actor-critic method with a detachment risk budget and independent action projection for magnetic wall-climbing robots. A reduced contact model links magnetic force, tangential demand, overturning moment and uncertain friction to a normalized attachment margin. The controller uses this margin to construct a conservative risk signal, learns a constrained stochastic policy, and projects candidate commands into a feasible actuator interval prior to execution. The numerical evaluation examines force tracking, route energy consumption, attachment reserve, and execution workload under varying operating conditions. The constrained controller reduces the mean route energy under the five operating conditions from 14.62 to 11.28 Wh compared with a conservative fixed-force baseline and reduces the number of unsafe episodes from 38 to 4 over 500 episodes compared to an unconstrained learning baseline. Its root mean square force tracking error is 11.2 N, compared with 17.8 N for the unconstrained baseline. Based on sensitivity analysis, a reduced risk budget leads to higher energy consumption and intervention frequency; increased latency reduces the attachment margin. Numerical comparisons show the interplay among reserve retention, actuator adjustment and route energy. The framework provides a physically interpretable control specification and an auditable evaluation structure linking mechanical requirements, policy preferences, and command feasibility.
References
Kong, J., Ji, A., Han, Q., Shen, H., Liu, S., Xiang, W., & Zhang, Q. (2025). Advances in research of wall-climbing robots: From biology to bionics—A review. Journal of Bionic Engineering, 22(3), 945–981. https://doi.org/10.1007/s42235-025-00689-6
Yang, P., Sun, L., Zhang, M., & Chen, H. (2024). A lightweight optimal design method for magnetic adhesion module of wall-climbing robot based on surrogate model and DBO algorithm. Journal of Mechanical Science and Technology, 38(4), 2041–2053. https://doi.org/10.1007/s12206-024-0334-3
Yun, S., Um, Y., Park, H.-W., & Kim, S. (2024). Static friction of magneto-rheological elastomer pads in wall-climbing robots. CIRP Annals, 73(1), 441–444. https://doi.org/10.1016/j.cirp.2024.04.100
Wang, G., Li, W., & Che, H. (2024). Design and analysis of anti-overturning mechanism for magnetic wall-climbing robot. Journal of Mechanical Science and Technology, 38(1), 379–387. https://doi.org/10.1007/s12206-023-1231-x
Duan, Z., Shi, G., Deng, C., & Yu, C. (2025). Design and implementation of a split-flexible wall-climbing robot with transition ability. Journal of Mechanical Science and Technology, 39(4), 2163–2180. https://doi.org/10.1007/s12206-025-0339-6
Tian, Y., Jitsukawa, H., Ma, S., & Zhang, G. (2024). Magnetic wall-climbing wheels with controllable adhesion reduction via soft magnetic material. IEEE Robotics and Automation Letters, 9(12), 11553–11560. https://doi.org/10.1109/LRA.2024.3496315
Tang, C., Abbatematteo, B., Hu, J., Chandra, R., Martín-Martín, R., & Stone, P. (2025). Deep reinforcement learning for robotics: A survey of real-world successes. Annual Review of Control, Robotics, and Autonomous Systems, 8(1), 153–188. https://doi.org/10.1146/annurev-control-030323-022510
Hwang, H.-S., Kim, Y., & Seok, J. (2025). Generative adversarial soft actor–critic. IEEE Transactions on Neural Networks and Learning Systems, 36(7), 11917–11927. https://doi.org/10.1109/TNNLS.2024.3493113
Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, J., & Knoll, A. (2024). A review of safe reinforcement learning: Methods, theories, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 11216–11235. https://doi.org/10.1109/TPAMI.2024.3457538
Hsu, K.-C., Hu, H., & Fisac, J. F. (2024). The safety filter: A unified view of safety-critical control in autonomous systems. Annual Review of Control, Robotics, and Autonomous Systems, 7(1), 47–72. https://doi.org/10.1146/annurev-control-071723-102940
Wang, B., Li, P., Li, P., Zhang, L., Guan, E., Liu, X., Hu, X., & Zhao, Y. (2025). Development of a wheeled wall-climbing robot with an internal corner wall adaptive magnetic adhesion mechanism. Journal of Field Robotics, 42(1), 97–114. https://doi.org/10.1002/rob.22402
Jia, P., Jia, J., Li, M., Zhang, M., & Zhao, J. (2025). Development of an omnidirectional mobile passive-compliant magnetic-wheeled wall-climbing robot for variable curvature facades. Journal of Field Robotics, 42(7), 3877–3901. https://doi.org/10.1002/rob.22601
Xiao, L., Lyu, Y., Zhang, F., Chen, L., Yu, G., Li, S. E., Ma, F., & Duan, J. (2025). Multi-style distributional soft actor-critic: Learning a unified policy for diverse control behaviors. IEEE Transactions on Intelligent Vehicles, 10(3), 1759–1770. https://doi.org/10.1109/TIV.2024.3432891
Duan, J., Wang, W., Xiao, L., Gao, J., Li, S. E., Liu, C., Zhang, Y.-Q., Cheng, B., & Li, K. (2025). Distributional soft actor-critic with three refinements. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5), 3935–3946. https://doi.org/10.1109/TPAMI.2025.3537087
Baumann, D., & Schön, T. B. (2024). Safe reinforcement learning in uncertain contexts. IEEE Transactions on Robotics, 40, 1828–1841. https://doi.org/10.1109/TRO.2024.3354176
Chen, W., Subramanian, D., & Paternain, S. (2024). Probabilistic constraint for safety-critical reinforcement learning. IEEE Transactions on Automatic Control, 69(10), 6789–6804. https://doi.org/10.1109/TAC.2024.3379246
Modares, A., Sadati, N., Esmaeili, B., Adib Yaghmaie, F., & Modares, H. (2024). Safe reinforcement learning via a model-free safety certifier. IEEE Transactions on Neural Networks and Learning Systems, 35(3), 3302–3311. https://doi.org/10.1109/TNNLS.2023.3264815
Zhang, Q., Leng, S., Ma, X., Liu, Q., Wang, X., Liang, B., Liu, Y., & Yang, J. (2025). CVaR-constrained policy optimization for safe reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 36(1), 830–841. https://doi.org/10.1109/TNNLS.2023.3331304
Zhang, H., Ren, J., Xiao, J., Pan, H., Lu, H., & Xu, X. (2025). FTR-Bench: Benchmarking deep reinforcement learning for flipper-track robot control. Journal of Field Robotics, 42(5), 2375–2389. https://doi.org/10.1002/rob.22528
Li, J., Shi, X., Liang, P., Li, Y., Lv, Y., Zhong, M., & Han, Z. (2024). Research on gait trajectory planning of wall-climbing robot based on improved PSO algorithm. Journal of Bionic Engineering, 21(4), 1747–1760. https://doi.org/10.1007/s42235-024-00538-y
Li, W., Lu, T., Ren, H., Luo, J., Wu, L., Tan, C., & Hu, M. (2025). Design and analysis of a quadruped wall climbing robot with low power consumption. Industrial Robot: The International Journal of Robotics Research and Application, 52(6), 821–832. https://doi.org/10.1108/IR-12-2024-0554
Bai, H., & Cheng, R. (2024). Generalized population-based training for hyperparameter optimization in reinforcement learning. IEEE Transactions on Emerging Topics in Computational Intelligence, 8(5), 3450–3462. https://doi.org/10.1109/TETCI.2024.3389777
Zhong, H., Wang, D., Gong, J., Ban, C., Wei, W., & Long, Y. (2024). Design and force performance analysis of a climbing robot based on Halbach magnetic array. Jordan Journal of Mechanical and Industrial Engineering, 18(4), 811–822. https://doi.org/10.59038/jjmie/180413
Yin, Y., Li, Z., Zhang, W., Zhang, Y., & Xu, Q. (2025). Design of a flexible passive adaptive suction module for wall-climbing robots. IEEE Robotics and Automation Letters, 10(12), 12429–12436. https://doi.org/10.1109/LRA.2025.3621974
Pizarro Bejarano, F., Brunke, L., & Schoellig, A. P. (2025). Safety filtering while training: Improving the performance and sample efficiency of reinforcement learning agents. IEEE Robotics and Automation Letters, 10(1), 788–795. https://doi.org/10.1109/LRA.2024.3512374
Coache, A., & Jaimungal, S. (2024). Reinforcement learning with dynamic convex risk measures. Mathematical Finance, 34(2), 557–587. https://doi.org/10.1111/mafi.12388
Yoo, G., Park, J., & Woo, H. (2024). Risk-conditioned reinforcement learning: A generalized approach for adapting to varying risk measures. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15), 16513–16521. https://doi.org/10.1609/aaai.v38i15.29589
Fatoretto, M. B., Demétrio, C. G. B., Brophy, C., & de Andrade Moral, R. (2026). Bootstrap confidence intervals for generalized linear mixed models in dose-response analysis. Journal of Statistical Computation and Simulation, 96(14), 3630–3653. https://doi.org/10.1080/00949655.2026.2677023