In the past, unmap sometimes occured on RM clients that have no thread, PD, or translation table assigned. However, this shouldn't be the case anymore. Fixes #504