Similar Items: Pessimism-Free Offline Learning in General-Sum Games via KL Regularization