Is it always possible a closed form solution for a norm minimization problem? Which one is the best approach closed form solution or gradient based?

As, we know that under-determined linear systems are having infinitely many solutions and we look for least norm solution via convex norm minimization constraint on the linear system. The underline norm minimization problem is having a unique solution because of convexity of the problem. So, based on convexity of the problem with respect to any convex norm other than $ell_2-$ norm, is there always a closed form solution of the norm minimization problem. Whether this convexity is enough to explain the existence of closed form solution? Regarding, second part of my question, from computational point of view, a gradient based method is always helpful. Is there any situation, where closed form solution performs better than gradient based approach?