This reverts commit 88325aeef26b11f9d6dfe91aaeb550df10b53ad4. Turns out a conference with a password triggers a failed conference join. It's going to be tricky to decipher when to do actual cleanup, and where to shove that code, so reverting is easier for now.