Is MVUE (Minimum Variance Unbiased Estimator) unique in every case?

I believe the answer is yes, MVUE's are unique. By Lemma 1.1 in Chapter 2 of Lehmann's "Theory of Point Estimation" the set of all unbiased estimators is an affine subspace. More specifically, if $delta_0$ is an unbiased estimator, then any unbiased estimator can be written in the form $delta = delta_0 - U$ for some unbiased estimator $U$ of 0.

So, if you restrict to estimators with finite variance, e.g. functions in $L^2$, then minimizing the variance is equivalent to finding $U$ which minimizes $E[(delta_0 - U)^2]$. In other words, you are looking for the element in the subspace that is closest to $delta_0$, which is unique.

If by complete/sufficient you mean that the data matrix $A$ is full rank, then I think there is no MVUE for the incomplete case:

If $x$ is the unknown vector, then $Ax$ cancels out at least one of the entries of $x$ (let's call it $x_k$) and the estimator $mathbb{hat{x}}$ will be biased. To see this, write $Ax$ as a linear combination of the entries of $x$ except for $x_k$. Then $mathbb{hat{x}}$ is independent from $x_k$. Hence $mathrm{E},mathbb{hat{x}} neq x$, except for one value of $x_k$.