Spherical Epipolar Cost Aggregation for ACVNet-Based Fisheye Stereo Depth Estimation
Authors
Emil Pawlak, Marcin Chmielewski, Sebastian Zalewski, Leon KamińskiFiles
Abstract
Fisheye stereo cameras provide a wide field of view, while their nonlinear projection complicates correspondence search and introduces spatial variation into conventional aggregation neighborhoods. This paper presents a spherical epipolar extension of an attention concatenation volume network for depth estimation directly from calibrated fisheye images. The proposed design samples candidate correspondences through calibrated rays, constructs correlation-guided attention over inverse depth, and aggregates evidence within depth-conditioned spherical epipolar neighborhoods. A validity mask excludes unsupported samples, and a residual gate controls the propagation of ambiguous costs. Evaluation uses a common candidate grid to examine overall accuracy, angular dependence, component contributions, calibration sensitivity, and computational cost. Absolute relative depth error decreases from 0.142 for the projection-adapted baseline to 0.109 for the full design, and the proportion of pixels exceeding ten percent relative error decreases from 18.6% to 13.4%. Peripheral error decreases from 0.201 to 0.139, accompanied by a latency increase from 46 to 54 ms per stereo pair. The ordered component sequence shows its largest individual step after validity gating; the contributions of spherical support remain conditional on preceding modules. Perturbation results indicate calibration sensitivity. The formulation connects attention- based matching with fisheye geometry and provides a framework for evaluating accuracy and computational tradeoffs. The analysis identifies peripheral support alignment, candidate resolution, and calibration stability as interconnected factors in wide-field stereo depth estimation. Interpretation of the reported comparisons requires documented data provenance and execution details for independent validation.
References
Roghani, S. E. S., & Koyuncu, E. (2024). Canonical trinocular camera setups and fisheye view for enhanced feature-based visual aerial odometry. IEEE Access, 12, 134888–134901. https://doi.org/10.1109/access.2024.3456909
Zhu, H., Zhang, G., Ye, Z., & Zhou, H. (2024). LVIF: A lightweight tightly coupled stereo-inertial SLAM with fisheye camera. Complex & Intelligent Systems, 10(1), 763–780. https://doi.org/10.1007/s40747-023-01190-5
Simioni, E., Pernechele, C., Erb, W., Marchini, A., Martini, P., Lessio, L., Penasa, L., & Beghini, M. (2024). A-central model for the geometric calibration of hyper-hemispherical lenses. Optics Express, 32(20), 34777–34795. https://doi.org/10.1364/oe.527318
Zhou, G., Tang, Y., Xie, Y., Song, R., Zhu, Q., & Zhou, X. (2024). Calibration of fisheye camera with colinear constraint of the main optical axis and epipolar line orthogonality-transverse axis. IEEE Sensors Journal, 24(12), 19373–19384. https://doi.org/10.1109/jsen.2024.3392810
Zhang, Y., Ma, Y., Wu, Y., & Liu, L. (2024). Achieving widely distributed feature matches using flattened-affine-SIFT algorithm for fisheye images. Optics Express, 32(5), 7969–7986. https://doi.org/10.1364/oe.513531
Jiang, H., Xu, R., Tan, M., & Jiang, W. (2024). RomniStereo: Recurrent omnidirectional stereo matching. IEEE Robotics and Automation Letters, 9(3), 2511–2518. https://doi.org/10.1109/lra.2024.3357315
Chen, Q., Ge, B., & Quan, J. (2024). Unambiguous pyramid cost volumes fusion for stereo matching. IEEE Transactions on Circuits and Systems for Video Technology, 34(10), 9223–9236. https://doi.org/10.1109/tcsvt.2023.3291726
Xu, G., Wang, X., Zhang, Z., Cheng, J., Liao, C., & Yang, X. (2025). IGEV++: Iterative multi-range geometry encoding volumes for stereo matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(8), 7108–7122. https://doi.org/10.1109/tpami.2025.3569218
Xu, G., Wang, Y., Cheng, J., Tang, J., & Yang, X. (2024). Accurate and efficient stereo matching via attention concatenation volume. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(4), 2461–2474. https://doi.org/10.1109/tpami.2023.3335480
Liao, Z., Zhou, W., & Li, H. (2024). DaFIR: Distortion-aware representation learning for fisheye image rectification. IEEE Transactions on Circuits and Systems for Video Technology, 34(5), 3606–3618. https://doi.org/10.1109/tcsvt.2023.3315967
Liu, Y., Cai, Q., Wang, C., Yang, J., Fan, H., Dong, J., & Chen, S. (2024). Geometry-enhanced attentive multi-view stereo for challenging matching scenarios. IEEE Transactions on Circuits and Systems for Video Technology, 34(8), 7401–7416. https://doi.org/10.1109/tcsvt.2024.3376692
Chou, C.-H., & Tsao, L.-H. (2025). Automatic calibration of multiple fisheye cameras using recovered human body mesh. Electronic Imaging, 37(10), 347-1–347-8. https://doi.org/10.2352/ei.2025.37.10.ipas-347
Liang, Z., & Li, C. (2024). Any-Stereo: Arbitrary scale disparity estimation for iterative stereo matching. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4), 3333–3341. https://doi.org/10.1609/aaai.v38i4.28119
Wang, S., Li, B., & Dai, Y. (2024). Efficient multi-view stereo by dynamic cost volume and cross-scale propagation. IEEE Transactions on Circuits and Systems for Video Technology, 34(10), 9414–9427. https://doi.org/10.1109/tcsvt.2024.3398060
Zhou, J., Lin, C., Yang, S., Liao, K., & Zhao, Y. (2025). Calibrating the principal point of vehicle-mounted fisheye cameras using point-oriented representation. IEEE Sensors Journal, 25(11), 19602–19612. https://doi.org/10.1109/jsen.2025.3555836
Wang, Y., Wang, L., Li, K., Zhang, Y., Wu, D. O., & Guo, Y. (2024). Cost volume aggregation in stereo matching revisited: A disparity classification perspective. IEEE Transactions on Image Processing, 33, 6425–6438. https://doi.org/10.1109/tip.2024.3484251
Okae, J., & Qin, H. (2024). DMCNet: Toward lightweight volumetric stereo matching. IEEE Sensors Journal, 24(7), 11936–11945. https://doi.org/10.1109/jsen.2024.3368028
Jin, L., & Xu, K. (2025). MAF-Stereo: Fast stereo matching through multi-branch attention fusion. ISA Transactions, 164, 211–221. https://doi.org/10.1016/j.isatra.2025.05.038
Li, P., Yao, C., Jia, Y., & Wu, Y. (2025). Inter-scale similarity guided cost aggregation for stereo matching. IEEE Transactions on Circuits and Systems for Video Technology, 35(1), 134–147. https://doi.org/10.1109/tcsvt.2024.3453965
Pathak, S., Hamada, T., & Umeda, K. (2024). Trinocular 360-degree stereo for accurate all-round 3D reconstruction considering uncertainty. Advanced Robotics, 38(15), 1038–1051. https://doi.org/10.1080/01691864.2024.2376022
Tosi, F., Bartolomei, L., & Poggi, M. (2025). A survey on deep stereo matching in the twenties. International Journal of Computer Vision, 133(7), 4245–4276. https://doi.org/10.1007/s11263-024-02331-0
Lee, J. Y., Ka, W., Choi, J., & Kim, J. (2024). Modeling stereo-confidence out of the end-to-end stereo-matching network via disparity plane sweep. Proceedings of the AAAI Conference on Artificial Intelligence, 38(4), 2901–2910. https://doi.org/10.1609/aaai.v38i4.28071
Xu, Y., & Dong, Q. (2026). Distortion-aware depth self-updating for self-supervised fisheye monocular depth estimation. IEEE Transactions on Image Processing, 35, 1883–1898. https://doi.org/10.1109/tip.2026.3661813
Zhang, C., Meng, G., Tian, K., Ni, B., & Xiang, S. (2024). Active disparity sampling for stereo matching with adjoint network. IEEE Transactions on Image Processing, 33, 354–365. https://doi.org/10.1109/tip.2023.3343105
Zhu, L., Rigall, E., Gao, Y., Zhang, Z., Bai, Y., & Dong, J. (2025). Region-aware driven distribution optimization for stereo matching. IEEE Transactions on Circuits and Systems for Video Technology, 35(12), 11965–11977. https://doi.org/10.1109/tcsvt.2025.3588869
Su, W., & Tao, W. (2025). Context-aware multi-view stereo network for efficient edge-preserving depth estimation. International Journal of Computer Vision, 133(6), 3367–3391. https://doi.org/10.1007/s11263-024-02337-8
Satouri, B., El Abderrahmani, A., & Satori, K. (2024). End-to-end artificial intelligence-based system for automatic stereo camera self-calibration. IEEE Access, 12, 160927–160945. https://doi.org/10.1109/access.2024.3486017
Zhang, C., Meng, G., Fan, B., Tian, K., Zhang, Z., Xiang, S., & Pan, C. (2024). Reusable architecture growth for continual stereo matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9), 6167–6184. https://doi.org/10.1109/tpami.2024.3378884
Wang, Y., Li, K., Wang, L., Hu, J., Wu, D. O., & Guo, Y. (2025). ADStereo: Efficient stereo matching with adaptive downsampling and disparity alignment. IEEE Transactions on Image Processing, 34, 1204–1218. https://doi.org/10.1109/tip.2025.3540282
Park, C., Kim, J., Kweon, M., & Park, J. (2026). CAR-Stereo: Confidence-aware adaptive disparity refinement for real-time stereo matching. IEEE Robotics and Automation Letters, 11(1), 690–697. https://doi.org/10.1109/lra.2025.3634908