Cut, overlap and locate: a deep learning approach for the 3D localization of particles in astigmatic optical setups

Astigmatic optical systems encode the depth location of spherical objects in the defocus blur of their images. This allows the simultaneous imaging of 3D positions of a large number of such objects, which can act as tracer particles in the study of fluid flows. The challenge lies in decoding the depth information, as defocused particle images might be overlapping or have low maximum intensity values. Current methods are not able to simultaneously detect and locate overlapping and low-intensity particle images. In addition, their cost of computation increases with particle image density. We show how semi-synthetic images of defocused particle images with proximate center point positions can be employed to train an end-to-end trainable particle image detector. This allows for the detection of low-intensity and overlapping particle images in a single pass of an image through a neural network. We present a thorough evaluation of the uncertainty of the method for the application of particles in fluid flow measurements. We achieve a similar error in the depth predictions to previous algorithms for non-overlapping particle images. In the case of neighboring particle images, the location error increases with decreasing particle image center distances and peaks when particle image centers share the same location. When dealing with actual measurement images, the location error increases by approximately a factor of two when particle images share the same center point locations. The trained model detects low-intensity particle images close to the visibility limit and covers 91.4% of the depth range of a human annotator. For the employed experimental arrangement, this increased the depth range along which particle images can be detected by 67% over a previously employed thresholding detection method (Franchini et al. in Adv Water Resour 124:1–8, 2019).