In most of the engineering literature I'm familiar with, white noise is introduced as an idealized random process $n(t)$ with a flat power spectrum

$$S_N(f)=frac{N_0}{2}tag{1}$$

and the corresponding autocorrelation function

$$R_N(tau)=frac{N_0}{2}delta(tau)tag{2}$$

The reason for defining white noise in this way is because it closely approximates the properties of thermal noise for frequencies below about $10^{12}$ Hz.

According to above definition, white noise is a WSS random process. Note that $(1)$ and $(2)$ imply that $n(t)$ has a constant mean equal to zero. I would claim that this is the standard definition of white noise in textbooks in the field of signal processing and digital communications.

White noise can also be defined in a less restrictive sense, namely as a process $n(t)$ for which the values $n(t_1)$ and $n(t_2)$ are uncorrelated for all $t_1$ and $t_2neq t_1$. I.e., the autocovariance function of $n(t)$ has the form

$$C_N(t_1,t_2)=q(t_1)delta(t_1-t_2),qquad q(t)ge 0tag{3}$$

This definition can be found in Probablity, Random Variables, and Stochastic Processes by Papoulis (p. 295 of the 3rd edition). Eq. $(3)$ implies an autocorrelation function of the form

$$R_N(t_1,t_2)=q(t_1)delta(t_1-t_2)+mu_N(t_1)mu_N(t_2)tag{4}$$

with $mu_N(t)=E{n(t)}$. Defined in that way, white noise is generally non-stationary and doesn't have a power spectrum in the conventional sense.

The "engineering definition" of white noise given above is obtained from the less restrictive definition by assuming that $q(t)$ is constant and that $mu_N(t)=0$. Note that if we assume a constant but non-zero $mu_N(t)$, the process would be WSS but the power spectrum would have a Dirac delta impulse at DC, which wouldn't be a good model for thermal noise.

White noise is not "WSS by nature" whatever you mean by that phrase but it can be treated as a (zero-mean) WSS process insofar as its effects in linear systems are concerned.

For example, standard linear system theory ways when the input to an LTI system is an ordinary WSS process ${X(t)}$ with autocorrelation function $R_X(tau)$, then the output of the LTI system is a WSS process ${Y(t)}$ with autocorrelation function $R_Y(tau)$ given by $$R_Y = hstar tilde{h} star R_X tag{1}$$ where $h(t)$ is the impulse response of the LTI system and $tilde{h}(t) = h(-t)$ is the time-reversed impulse response of the LTI system. The power spectral densities are related as $$S_Y(f) = |H(f)|^2S_X(f)tag{2}$$ where $H(f)$ is the transfer function of the LTI system. If ${X(t)}$ is a white noise process with autocorrelation function $Kdelta(tau)$ and we pretend that $(1)$ and $(2)$ are still applicable, we get that ${Y(t)}$ is a zero-mean WSS process with autocorrelation function $R_Y = Kcdot hstar tilde{h}$ and power spectral density $S_Y(f) = Kcdot |H(f)|^2$. Of course, mathematicians would laugh at this calculation but physical experiments using the naturally occurring thermal noise in electrical circuits as a stand-in for a white noise process show that these results are pretty close to reality. As engineers, we seek equations that match the universe as we observe it (physicists seek universes that match their equations while mathematicians don't care) and so we go blithely on our way treating white noise as a WSS process in linear systems and everything works out OK. The troubles start when we start treating white noise as a WSS process in nonlinear systems and the world comes crashing down about our ears and we need to start paying attention to what the math people are saying.

White noise is referred as white Gaussian noise if we pretend or claim or insist that ${Y(t)}$ is a Gaussian process which means that not only are all the random variables $Y(t)$ Gaussian random variables, but every finite set ${Y(t_1), Y(t_2), cdots, Y(t_n)}, n geq 2,$ of random variables is a set of jointly Gaussian random variables. Standard random process theory says that when a Gaussian process is passed through an LTI system, the output is a Gaussian process but this fact does not allow us to reverse-engineer the result and claim that all the $X(t)$'s are also Gaussian random variables.

Disclaimer: this might very well be wrong. Still pondering it, but Dilip Sarwate has convincing points.

When you say "white" you assume it's WSS to begin with. For non-WSS processes, "white" isn't defined, since no only lag-dependent autocorrelation can be found. (And a process is white, exactly if its autocorrelation takes the form of a delta dirac impulse.)

So, yes, any process that is called "white" is inherently WSS.

"Gaussian white noise" is white noise whose amplitude is Gaussian-distributed. Amplitude distribution has nothing to do with whiteness or stationarity: a non-stationary process can still be Gaussian distributed at any point in time.