Text this: Memory-Efficient Distributed Unlearning