An intelligent threshold selection method to improve orbital angular momentum-encoded quantum key distribution under turbulence

High-dimensional quantum key distribution (HD-QKD) encoded by orbital angular momentum (OAM) presents significant advantages in terms of information capacity. However, perturbations caused by free-space atmospheric turbulence decrease the performance of the system by introducing random fluctuations in the transmittance of OAM photons. Currently, the theoretical performance analysis of OAM-encoded QKD systems exists a gap when concerning the statistical distribution under the free-space link. In this article, we analyzed the security of QKD systems by combining probability distribution of transmission coefficient (PDTC) of OAM with decoy-state BB84 method. To address the problem that the invalid key rate is calculated in the part transmittance interval of the post-processing process, an intelligent threshold method based on neural network is proposed to improve OAM-encoded QKD, which aims to conserve computing resources and enhance system efficiency. Our findings reveal that the ratio of root mean square (RMS) OAM-beam radius to Fried constant plays a crucial role in ensuring secure key generation. Meanwhile, the training error of neural network is at the magnitude around 10${}^{-3}$, indicating the ability to predict optimization parameters quickly and accurately. Our work contributes to the advancement of parameter optimization and prediction for free-space OAM-encoded HD-QKD systems. Furthermore, it provides valuable theoretical insights to support the development of free-space experimental setups.