Aproximating unitaries with elements from a t-design

It’s straightforward to see that a random element $U$ drawn from a design is, with very high probability, far away from any fixed unitary $V$, i.e. you can show that the distance between $U$ and $V$ (in operator norm or in the diamond norm of their channels) is on average very close to maximal.

But what you’re asking about is if there exists at least one element in the design which is close to some fixed unitary $V$. Without knowing anything else about the structure of the design, you’re asking when do unitary $t$-designs cover the unitary group (i.e. every $U$ in $U(d)$ is at most $epsilon$ away, in some norm, from some element of the design. Equivalently, putting $epsilon$-balls around each unitary, $U(d)$ is contained in the union of the balls.)

This doesn't address your question rigorously, but here's an argument for why the design order should be at least $t=O(d^2)$, in order for $t$-designs to cover $U(d)$. For an exact unitary $t$-design $mathcal{E}$, a lower bound on the cardinality is $|mathcal{E}| geq d^{2t}/t!$. If the number of elements in $mathcal{E}$ is some constant multiple of this lower bound, then we find that $|mathcal{E}|$ becomes comparable to the volume of the unitary group (more precisely, the vol($U(d)$)/vol($epsilon$-ball)) when $tsim d^2$. But to rigorously show that for some $t$, unitary $t$-designs cover the unitary group will require more work.