Top

Detachment Risk Constrained Soft Actor Critic for Adhesion Force Control of Magnetic Wall Climbing Robots

Authors

Files

PDF

Abstract

Magnetic adhesion control must balance contact maintenance against locomotion resistance. This paper presents a soft actor-critic method with a detachment risk budget and independent action projection for magnetic wall-climbing robots. A reduced contact model links magnetic force, tangential demand, overturning moment and uncertain friction to a normalized attachment margin. The controller uses this margin to construct a conservative risk signal, learns a constrained stochastic policy, and projects candidate commands into a feasible actuator interval prior to execution. The numerical evaluation examines force tracking, route energy consumption, attachment reserve, and execution workload under varying operating conditions. The constrained controller reduces the mean route energy under the five operating conditions from 14.62 to 11.28 Wh compared with a conservative fixed-force baseline and reduces the number of unsafe episodes from 38 to 4 over 500 episodes compared to an unconstrained learning baseline. Its root mean square force tracking error is 11.2 N, compared with 17.8 N for the unconstrained baseline. Based on sensitivity analysis, a reduced risk budget leads to higher energy consumption and intervention frequency; increased latency reduces the attachment margin. Numerical comparisons show the interplay among reserve retention, actuator adjustment and route energy. The framework provides a physically interpretable control specification and an auditable evaluation structure linking mechanical requirements, policy preferences, and command feasibility.

PDF

References

Kong, J., Ji, A., Han, Q., Shen, H., Liu, S., Xiang, W., & Zhang, Q. (2025). Advances in research of wall-climbing robots: From biology to bionics—A review. Journal of Bionic Engineering, 22(3), 945–981. https://doi.org/10.1007/s42235-025-00689-6

Yang, P., Sun, L., Zhang, M., & Chen, H. (2024). A lightweight optimal design method for magnetic adhesion module of wall-climbing robot based on surrogate model and DBO algorithm. Journal of Mechanical Science and Technology, 38(4), 2041–2053. https://doi.org/10.1007/s12206-024-0334-3

Yun, S., Um, Y., Park, H.-W., & Kim, S. (2024). Static friction of magneto-rheological elastomer pads in wall-climbing robots. CIRP Annals, 73(1), 441–444. https://doi.org/10.1016/j.cirp.2024.04.100

Wang, G., Li, W., & Che, H. (2024). Design and analysis of anti-overturning mechanism for magnetic wall-climbing robot. Journal of Mechanical Science and Technology, 38(1), 379–387. https://doi.org/10.1007/s12206-023-1231-x

Duan, Z., Shi, G., Deng, C., & Yu, C. (2025). Design and implementation of a split-flexible wall-climbing robot with transition ability. Journal of Mechanical Science and Technology, 39(4), 2163–2180. https://doi.org/10.1007/s12206-025-0339-6

Tian, Y., Jitsukawa, H., Ma, S., & Zhang, G. (2024). Magnetic wall-climbing wheels with controllable adhesion reduction via soft magnetic material. IEEE Robotics and Automation Letters, 9(12), 11553–11560. https://doi.org/10.1109/LRA.2024.3496315

Tang, C., Abbatematteo, B., Hu, J., Chandra, R., Martín-Martín, R., & Stone, P. (2025). Deep reinforcement learning for robotics: A survey of real-world successes. Annual Review of Control, Robotics, and Autonomous Systems, 8(1), 153–188. https://doi.org/10.1146/annurev-control-030323-022510

Hwang, H.-S., Kim, Y., & Seok, J. (2025). Generative adversarial soft actor–critic. IEEE Transactions on Neural Networks and Learning Systems, 36(7), 11917–11927. https://doi.org/10.1109/TNNLS.2024.3493113

Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, J., & Knoll, A. (2024). A review of safe reinforcement learning: Methods, theories, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 11216–11235. https://doi.org/10.1109/TPAMI.2024.3457538

Hsu, K.-C., Hu, H., & Fisac, J. F. (2024). The safety filter: A unified view of safety-critical control in autonomous systems. Annual Review of Control, Robotics, and Autonomous Systems, 7(1), 47–72. https://doi.org/10.1146/annurev-control-071723-102940

Wang, B., Li, P., Li, P., Zhang, L., Guan, E., Liu, X., Hu, X., & Zhao, Y. (2025). Development of a wheeled wall-climbing robot with an internal corner wall adaptive magnetic adhesion mechanism. Journal of Field Robotics, 42(1), 97–114. https://doi.org/10.1002/rob.22402

Jia, P., Jia, J., Li, M., Zhang, M., & Zhao, J. (2025). Development of an omnidirectional mobile passive-compliant magnetic-wheeled wall-climbing robot for variable curvature facades. Journal of Field Robotics, 42(7), 3877–3901. https://doi.org/10.1002/rob.22601

Xiao, L., Lyu, Y., Zhang, F., Chen, L., Yu, G., Li, S. E., Ma, F., & Duan, J. (2025). Multi-style distributional soft actor-critic: Learning a unified policy for diverse control behaviors. IEEE Transactions on Intelligent Vehicles, 10(3), 1759–1770. https://doi.org/10.1109/TIV.2024.3432891

Duan, J., Wang, W., Xiao, L., Gao, J., Li, S. E., Liu, C., Zhang, Y.-Q., Cheng, B., & Li, K. (2025). Distributional soft actor-critic with three refinements. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5), 3935–3946. https://doi.org/10.1109/TPAMI.2025.3537087

Baumann, D., & Schön, T. B. (2024). Safe reinforcement learning in uncertain contexts. IEEE Transactions on Robotics, 40, 1828–1841. https://doi.org/10.1109/TRO.2024.3354176

Chen, W., Subramanian, D., & Paternain, S. (2024). Probabilistic constraint for safety-critical reinforcement learning. IEEE Transactions on Automatic Control, 69(10), 6789–6804. https://doi.org/10.1109/TAC.2024.3379246

Modares, A., Sadati, N., Esmaeili, B., Adib Yaghmaie, F., & Modares, H. (2024). Safe reinforcement learning via a model-free safety certifier. IEEE Transactions on Neural Networks and Learning Systems, 35(3), 3302–3311. https://doi.org/10.1109/TNNLS.2023.3264815

Zhang, Q., Leng, S., Ma, X., Liu, Q., Wang, X., Liang, B., Liu, Y., & Yang, J. (2025). CVaR-constrained policy optimization for safe reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 36(1), 830–841. https://doi.org/10.1109/TNNLS.2023.3331304

Zhang, H., Ren, J., Xiao, J., Pan, H., Lu, H., & Xu, X. (2025). FTR-Bench: Benchmarking deep reinforcement learning for flipper-track robot control. Journal of Field Robotics, 42(5), 2375–2389. https://doi.org/10.1002/rob.22528

Li, J., Shi, X., Liang, P., Li, Y., Lv, Y., Zhong, M., & Han, Z. (2024). Research on gait trajectory planning of wall-climbing robot based on improved PSO algorithm. Journal of Bionic Engineering, 21(4), 1747–1760. https://doi.org/10.1007/s42235-024-00538-y

Li, W., Lu, T., Ren, H., Luo, J., Wu, L., Tan, C., & Hu, M. (2025). Design and analysis of a quadruped wall climbing robot with low power consumption. Industrial Robot: The International Journal of Robotics Research and Application, 52(6), 821–832. https://doi.org/10.1108/IR-12-2024-0554

Bai, H., & Cheng, R. (2024). Generalized population-based training for hyperparameter optimization in reinforcement learning. IEEE Transactions on Emerging Topics in Computational Intelligence, 8(5), 3450–3462. https://doi.org/10.1109/TETCI.2024.3389777

Zhong, H., Wang, D., Gong, J., Ban, C., Wei, W., & Long, Y. (2024). Design and force performance analysis of a climbing robot based on Halbach magnetic array. Jordan Journal of Mechanical and Industrial Engineering, 18(4), 811–822. https://doi.org/10.59038/jjmie/180413

Yin, Y., Li, Z., Zhang, W., Zhang, Y., & Xu, Q. (2025). Design of a flexible passive adaptive suction module for wall-climbing robots. IEEE Robotics and Automation Letters, 10(12), 12429–12436. https://doi.org/10.1109/LRA.2025.3621974

Pizarro Bejarano, F., Brunke, L., & Schoellig, A. P. (2025). Safety filtering while training: Improving the performance and sample efficiency of reinforcement learning agents. IEEE Robotics and Automation Letters, 10(1), 788–795. https://doi.org/10.1109/LRA.2024.3512374

Coache, A., & Jaimungal, S. (2024). Reinforcement learning with dynamic convex risk measures. Mathematical Finance, 34(2), 557–587. https://doi.org/10.1111/mafi.12388

Yoo, G., Park, J., & Woo, H. (2024). Risk-conditioned reinforcement learning: A generalized approach for adapting to varying risk measures. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15), 16513–16521. https://doi.org/10.1609/aaai.v38i15.29589

Fatoretto, M. B., Demétrio, C. G. B., Brophy, C., & de Andrade Moral, R. (2026). Bootstrap confidence intervals for generalized linear mixed models in dose-response analysis. Journal of Statistical Computation and Simulation, 96(14), 3630–3653. https://doi.org/10.1080/00949655.2026.2677023

Details