How to delete the existing polygon feature value inside the new polygon so that it borders the new polygon instead of overlapping?

You need to select the last drawn polygon, which is the default behaviour when you draw a new polygon, and then go to Editing toolbar’s dropdown menu (little downward arrow). Find Clip tool there and pick the Discard area overlapping option. Make sure only polygon that you want to clip is selected and only polygon layer to be clipped is visible. If you have multiple selection, they all will clip anything overlapping. If you have more than one visible “editable” layers visible, the clip will be applied to all.

As far as I know there is no way to automate this on-the-fly. You can set topology rules though, which will highlight overlapping parts.