Text this: Misaligned by Reward: Socially Undesirable Preferences in LLMs