How to find the acceleration of a spool pulled by a force and its work?

You need to be careful in applying the right hand rule to determine the direction of the torques and the angular acceleration. Notice that since the spool is rolling without slipping, it will have angular acceleration in the clockwise direction. By the right hand rule, this means that the angular acceleration is into the page. If we take both the angular acceleration and horizontal acceleration to be positive, which you've done with $a = Ralpha$, then the torque from $F$ is directed out of the page (i.e. negative) while the torque from friction is into the page (i.e. positive). Thus you should have the following equations: $$Fcosalpha-F_f = m a_{||} a_{||} = Ralpha F_f R - Fr = I alpha.$$ Solving those for $a_{||}$ yields the answer from the book.

The second part is definitely trickier. My suggestion is the following. We know that $$Delta K = W_{ext}.$$ We also know that the force of friction does no work since the spool doesn't slip. Therefore if we compute $Delta K$ then we know how much work was done by the force. Remember that the kinetic energy has two contributions: one from the horizontal motion and one from the rotational motion. I found a final answer of $$W_{ext} = frac{1}{2} frac{F^2(cosalpha-frac{r}{R})^2}{m(1+gamma)}t^2.$$

Good luck!

One solution is the above. The other one is from the rotation at the point when the spool touching the ground. For simplicity, let $alpha = 0$:

kinetic relation: $a_{||} = R frac{d omega}{dt}$

Newton law in rotation: $F(R-r) = (gamma + 1) R^2frac{d omega}{dt}$

So that: $a_{||} = frac{1- frac{r}{R}}{gamma + 1} frac{F}{m}$