PPP auth fails with special characters in password

My provisioned password had a '#' in it and the PPP/L2TP login just kept failing on Windows. Regenerating with an alphanumeric-only password fixed it, so something is mangling the credential.