What to do if one out of 2 one-hot encoding variables have a very high p-value?

By all means keep Embarked_C. Consider the following example: You're predicting whether or not someone's favorite color is blue. You know the color of their favorite shirt- it's either blue, yellow, or red. Color_Blue is going to be significant, the other two one hot encoded variables would not be. You'd still want to keep Color_Blue as a feature.

You should keep all of levels as they collectively describe the feature. Removing the insignificant ones will bias your coefficients and distort your interpretation (e.i. change the reference level).

Here are some stats.exchange references:

• Is it advisable to drop certain levels of a categorical variable?
• Can I ignore coefficients for non-significant levels of factors in a linear model?