A hacker can simply click "forgot password" when logging in and have a password reset link sent straight to your inbox, which they now check. Criminals gaining access to your email can easily reset the passwords of other accounts, gaining more information about your personal data and associated accounts. This means that even if an unauthorised party is able to crack your password, they will still need physical access to your phone in order to access your email account.