What is the correct path to store a user specific credentials file?

have a look at the freedesktop.org's Secret Storage Spec (Version 0.2 Working Draft)

This way you can have the credentials saved in a keyring that is unlocked when the user logs in.

Why not put it next to your SSH key in ~/.ssh? Sounds like it serves kind of the same purpose.

Alternatively, you could use something like Hashicorp Vault to store secrets.