Parallel transport: Lie derivative vs covariant derivative

Consider the expression $nabla_{mathbf X_p} mathbf T$, for some arbitrary tangent vector $mathbf X_p$ and tensor field $mathbf T$. Since a tangent vector is always defined at a point $p$, we are to understand this expression to mean the rate of change of $mathbf T$ along $mathbf X_p$ at the point $p$ (where $mathbf X_p$ lives). If $mathbf T$ is a $(r,s)$-tensor field, then the quantity $nabla_{mathbf{X}_p} mathbf T$ is a $(r,s)$-tensor at the point $p$.

Of course, if you have a vector field $mathbf Y$, then we can define the object $nabla_{mathbf Y}mathbf T$ to be a $(r,s)$-tensor field. The tensor at the point $p$ is then simply $left(nabla_mathbf Y mathbf Tright)_p := nabla_{mathbf Y_p}mathbf T$.

The point is that the covariant derivative is taken with respect to a vector, not a vector field, and returns a $(r,s)$-tensor, not an $(r,s)$-tensor field. If we happen to have a vector field lying around then we can of course plug in the corresponding vector at every point to obtain a $(r,s)$-tensor field, but this is not necessary. Contrast this with the Lie derivative, for which $mathcal L_mathbf X mathbf T$ requires $mathbf X$ to be a vector field. This is the price we pay for not introducing the additional structure of a connection; the Lie derivative requires information about $mathbf X$ in a neighborhood, not just at a point.

My question is the following, why do we need covariant derivative which requires additional structure to find curves along which a tensor is parallel-transported.

If I give you a curve $gamma$ and ask you to parallel-transport a tensor $mathbf T$ along it with the Lie derivative, then what would you do? Presumably you'd compute the tangent vector $mathbf V_p= gamma'(p)$ at every point $p$ along the curve, and then plug it into $mathcal L_{mathbf V_p} mathbf T=0$. However, in computing this, you'd find yourself taking the derivatives of the components of $mathbf V_p$ in arbitrary directions, not just along $gamma$.

How would you do this? If all you have is the curve, then it doesn't even make sense to ask how the tangent vector changes in arbitrary directions. You'd need to define a vector field in the neighborhood of $gamma$ for which $gamma$ is an integral curve, and herein lies the problem: there are an infinity of vector fields which fit this description, and parallel-transporting via two different such vector fields would generically give two different results.

In order for this to be a well-defined procedure, you need additional structure - namely, a choice of vector field (for which $gamma$ is an integral curve) along which to parallel-transport your tensor. If you want this procedure to be consistent for all curves, then you'd need to provide some kind of criterion for choosing the "correct" vector field given only one of its integral curves, and this is precisely the additional structure which is required to define a connection.