Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs (Journal of Machine Learning Research, 2026)