Transformer Model for Rail Squat Classification with Lift off Invariant Eddy Current Embeddings
Authors
Oktay Savaş, Mesut RaufFiles
Abstract
Probe lift-off changes the amplitude, phase and spatial extent of eddy-current responses, complicating classification of rail- squat indication morphology. This study evaluates a two-layer Transformer with lift-off-invariant projected embeddings on a declared numerical response family. Three complex observation channels at nominal excitation settings 25, 50 and 100 kHz are sampled at 96 positions across an 80 mm window. Training-derived channel standardization, window centering and common energy normalization precede 24 ordered spatial tokens. Paired observations of one geometry receive a consistency objective, while supervised contrastive learning organizes projected vectors by four reference families: reference, compact, extended and compound. Independent geometry partitions contain 1,200 training, 320 validation and 640 held-out sources. Each held-out geometry is observed in a seen lift-off interval of 0.5-2.5 mm and an unseen interval of 2.8-4.0 mm. Under the unseen condition, the full model reaches 97.19% accuracy and macro-F1 0.9720, versus 95.00% and 0.9500 for a raw-input Transformer. Its paired-embedding cosine similarity is 0.9655, compared with 0.9226 for the raw model, and seen-range ridge-probe lift-off R squared is 0.0684 versus 0.3491. A class-only objective reaches higher unseen accuracy of 98.28% in the primary run. Validation-selected temperature 0.5 increases unseen expected calibration error from 0.0155 to 0.0241, indicating weaker probability transfer than class decision transfer. At 4.1-5.0 mm lift-off, full-model accuracy is 73.28%; added noise and central evidence gaps yield 76.56% and 74.53%. These outcomes distinguish classification, representation consistency and confidence within the synthetic condition-shift setting.
References
Quercio, M., Santoro, L., Sesana, R., & Riganti Fulginei, F. (2025). Advanced feature analysis of eddy current testing signals for rail surface defect characterization. IEEE Access, 13, 141156–141169. https://doi.org/10.1109/ACCESS.2025.3597079
Shao, Y., Xia, Z., Ding, Y., Crocker, B., Saunders, S., Bai, X., Peyton, A., Conniffe, D., & Yin, W. (2024). Eddy current sensor array for electromagnetic sensing and crack reconstruction with high lift-off in railway tracks. Sensors, 24(13), 4216. https://doi.org/10.3390/s24134216
Ma, Q., Tian, G., Gao, B., Robinson, M., Yang, C., & Ibrahim, E. T. (2024). Novel common-differential inductance coils with dual signal conditionings for separation of lift-off and defects. IEEE Sensors Journal, 24(2), 2055–2065. https://doi.org/10.1109/JSEN.2023.3336066
Rath, M., & Condurache, A. P. (2024). Boosting deep neural networks with geometrical prior knowledge: A survey. Artificial Intelligence Review, 57, 95. https://doi.org/10.1007/s10462-024-10722-5
Khoee, A. G., Yu, Y., & Feldt, R. (2024). Domain generalization through meta-learning: A survey. Artificial Intelligence Review, 57, 285. https://doi.org/10.1007/s10462-024-10922-z
Sasse, L., Nicolaisen-Sobesky, E., Dukart, J., Eickhoff, S. B., Götz, M., Hamdan, S., Komeyer, V., Kulkarni, A., Lahnakoski, J. M., Love, B. C., Raimondo, F., & Patil, K. R. (2025). Overview of leakage scenarios in supervised machine learning. Journal of Big Data, 12, 135. https://doi.org/10.1186/s40537-025-01193-8
Hu, H., Wang, X., Zhang, Y., Chen, Q., & Guan, Q. (2024). A comprehensive survey on contrastive learning. Neurocomputing, 610, 128645. https://doi.org/10.1016/j.neucom.2024.128645
Zhang, K., Wen, Q., Zhang, C., Cai, R., Jin, M., Liu, Y., Zhang, J. Y., Liang, Y., Pang, G., Song, D., & Pan, S. (2024). Self-supervised learning for time series analysis: Taxonomy, progress, and prospects. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(10), 6775–6794. https://doi.org/10.1109/TPAMI.2024.3387317
Balanya, S. A., Maroñas, J., & Ramos, D. (2024). Adaptive temperature scaling for robust calibration of deep neural networks. Neural Computing and Applications, 36(14), 8073–8095. https://doi.org/10.1007/s00521-024-09505-4
Li, Y., Xia, Z., She, S., Shao, Y., Yang, Y., & Yin, W. (2025). A Mobius transformation-based liftoff effect reduction method for crack classification and prediction in eddy current testing. NDT & E International, 149, 103261. https://doi.org/10.1016/j.ndteint.2024.103261
Cho, Y. S., & Kim, S. B. (2024). Supervised contrastive learning for multisensor signals classification in automobile engine manufacturing. IEEE Transactions on Industrial Informatics, 20(5), 7764–7776. https://doi.org/10.1109/TII.2024.3363189
Abdullahi, M., Alhussian, H., Aziz, N., Abdulkadir, S. J., Khairy, S. O. F., Bala, A., Adamu, S., & Yunus, R. B. (2026). A systematic literature review and taxonomy of normalization techniques in training DNNs for time-series data. Neurocomputing, 696, 134095. https://doi.org/10.1016/j.neucom.2026.134095
Meng, B., Zhuang, Z., Ma, J., & Zhao, S. (2025). A pulsed eddy current data-driven interpretable neural network for metallic foreign objects’ characteristics identification. IEEE Sensors Journal, 25(19), 35835–35843. https://doi.org/10.1109/JSEN.2025.3598371
Uelwer, T., Robine, J., Wagner, S. S., Höftmann, M., Upschulte, E., Konietzny, S., Behrendt, M., & Harmeling, S. (2025). A survey on self-supervised methods for visual representation learning. Machine Learning, 114, 111. https://doi.org/10.1007/s10994-024-06708-7
Cui, J., Zhong, Z., Tian, Z., Liu, S., Yu, B., & Jia, J. (2024). Generalized parametric contrastive learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 7463–7474. https://doi.org/10.1109/TPAMI.2023.3278694
Zhong, Y., Zhou, W., & Wang, Z. (2025). A survey of data augmentation in domain generalization. Neural Processing Letters, 57, 34. https://doi.org/10.1007/s11063-025-11747-9
Abbasi, M., & Saeedi, P. (2024). Enhancing multivariate time series classifiers through self-attention and relative positioning infusion. IEEE Access, 12, 67273–67290. https://doi.org/10.1109/ACCESS.2024.3397783
Lones, M. A. (2024). Avoiding common machine learning pitfalls. Patterns, 5(10), 101046. https://doi.org/10.1016/j.patter.2024.101046
Wu, Y., Shi, B., Dong, B., & Zheng, Q. (2026). TransTS: An adaptive post-hoc method for probability calibration under label noise. Science China Information Sciences, 69, 122105. https://doi.org/10.1007/s11432-025-4576-9
Apicella, A., Isgrò, F., & Prevete, R. (2025). Don’t push the button! Exploring data leakage risks in machine learning and transfer learning. Artificial Intelligence Review, 58, 339. https://doi.org/10.1007/s10462-025-11326-3
Migus, P. (2026). Newton–Puiseux analysis for interpretability and calibration of complex-valued neural networks. Neural Networks, 195, 108172. https://doi.org/10.1016/j.neunet.2025.108172
Ilbert, R., Hoang, T. V., & Zhang, Z. (2024). Data augmentation for multivariate time series classification: An experimental study. In 2024 IEEE 40th International Conference on Data Engineering Workshops (ICDEW) (pp. 128–139). IEEE. https://doi.org/10.1109/ICDEW61823.2024.00023
Babu, M. A. A., Pandey, S. K., Durisic, D., Koppisetty, A. C., & Staron, M. (2025). D-LeDe: A data leakage detection method for automotive perception systems. In Proceedings of the 11th International Conference on Vehicle Technology and Intelligent Transport Systems (pp. 210–221). SciTePress. https://doi.org/10.5220/0013476700003941
Mohammadi Foumani, N., Miller, L., Tan, C. W., Webb, G. I., Forestier, G., & Salehi, M. (2024). Deep learning for time series classification and extrinsic regression: A current survey. ACM Computing Surveys, 56(9), Article 217, 1–45. https://doi.org/10.1145/3649448
Opitz, J. (2024). A closer look at classification evaluation metrics and a critical reflection of common evaluation practice. Transactions of the Association for Computational Linguistics, 12, 820–836. https://doi.org/10.1162/tacl_a_00675
Middlehurst, M., Schäfer, P., & Bagnall, A. (2024). Bake off redux: A review and experimental evaluation of recent time series classification algorithms. Data Mining and Knowledge Discovery, 38(4), 1958–2031. https://doi.org/10.1007/s10618-024-01022-
Klabunde, M., Schumacher, T., Strohmaier, M., & Lemmerich, F. (2025). Similarity of neural network models: A survey of functional and representational measures. ACM Computing Surveys, 57(9), Article 242, 1–52. https://doi.org/10.1145/3728458
Waldis, A., Perlitz, Y., Choshen, L., Hou, Y., & Gurevych, I. (2024). Holmes ⌕ A benchmark to assess the linguistic competence of language models. Transactions of the Association for Computational Linguistics, 12, 1616–1647. https://doi.org/10.1162/tacl_a_00718
Li, T., Guo, X., & Ji, C. (2025). A literature survey of shapelet quality measures for time series classification. PeerJ Computer Science, 11, e3115. https://doi.org/10.7717/peerj-cs.3115
Liu, M., Sheng, H., Zhang, N., Zhao, P., Yi, Y., Jiang, Y., & Dai, J. (2024). DSDCLNet: Dual-stream encoder and dual-level contrastive learning network for supervised multivariate time series classification. Knowledge-Based Systems, 292, 111638. https://doi.org/10.1016/j.knosys.2024.111638
Dong, C. G., Li, Z. D., Zheng, L. N., Chen, W., & Zhang, W. E. (2024). Boosting certificate robustness for time series classification with efficient self-ensemble. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (pp. 477–486). ACM. https://doi.org/10.1145/3627673.3679748
Dong, J., Jiang, Z., Pan, D., Chen, Z., Guan, Q., Zhang, H., Gui, G., & Gui, W. (2025). A survey on confidence calibration of deep learning-based classification models under class imbalance data. IEEE Transactions on Neural Networks and Learning Systems, 36(9), 15664–15684. https://doi.org/10.1109/TNNLS.2025.3565159
Vos, G., van Eijk, L., Sarnyai, Z., & Rahimi Azghadi, M. (2025). Stabilizing machine learning for reproducible and explainable results: A novel validation approach to subject-specific insights. Computer Methods and Programs in Biomedicine, 269, 108899. https://doi.org/10.1016/j.cmpb.2025.108899
He, Z., Li, Z., Dong, J., Liu, X., Jiang, M., & Hu, L. (2025). Conjunction subspaces test for conformal and selective classification. Information Sciences, 707, 122037. https://doi.org/10.1016/j.ins.2025.122037
Semmelrock, H., Ross-Hellauer, T., Kopeinik, S., Theiler, D., Haberl, A., Thalmann, S., & Kowald, D. (2025). Reproducibility in machine-learning-based research: Overview, barriers, and drivers. AI Magazine, 46(2), e70002. https://doi.org/10.1002/aaai.70002