If excess sodium is bad for humans, why don't our bodies just remove it?

This is one of those questions where I realized two everyday facts don’t connect for me. Excessive sodium in the diet is supposed to be bad for us. But surely our bodies have mechanisms for getting rid of excess sodium. (I don’t know the details, but I know enough about neurons to know that our cells can move sodium when they need to.) Why don’t our bodies just keep our sodium levels at a healthy level?