Frequentist Guarantees of Distributed (Non)-Bayesian Inference (Journal of Machine Learning Research, 2026)