Variational inference with the spacings estimator
Background Variational inference (VI) is a key framework in Bayesian deep learning, enabling scalable approximations of complex posterior distributions. Accurate entropy estimation is critical in VI but remains challenging, particularly for high-dimensional or multimodal distributions. Traditional methods, such as closed-form approximations or Monte Carlo sampling, can be computationally intensive or inaccurate. The spacings estimator, a non-parametric technique leveraging the ordering and spacing of samples, offers a promising alternative for efficient and robust entropy estimation. ...