I need to help with a formula to find the length of a line.

Since the problem remains the same when the rectangle is rotated/flipped, we can assume for simplicity that the starting point is on the bottom border of the rectangle. (When rotating the problem, make sure to get the angle and width/height of the rotated problem correct.)

Let $w$ denote the width and $h$ the height of the rectangle. Let the starting point have distance $x$ from the left border of the rectangle. Let $alpha$ be the angle between your line and the bottom line that you get by "rotating your line to the left."

Image:

We want to compute the point $overline{EF}$. If $operatorname{arcsec}$ denotes the inverse function of $xmapstofrac1{cos(x)}$ and $$alpha=operatorname{arcsec}left(frac{overline{CE}}{overline{AE}}right)=operatorname{arcsec}left(frac{sqrt{x^2+h^2}}{x}right),$$

then your line goes straight to the top left corner and the length is $sqrt{x^2+h^2}$. If $alpha$ is less than that, then you will "bump" into the left border and we have $overline{AF}=tan(alpha) x$. Hence the length of the line is $$sqrt{(tan(alpha)^2+1) x^2}=lvertsec(alpha)xrvert.$$

Let $beta=pi-alpha$. You get the same thing: If $$beta=operatorname{arcsec}left(frac{overline{ED}}{overline{BE}}right)=operatorname{arcsec}left(frac{sqrt{(w-x)^2+h^2}}{(w-x)}right),$$

then the line goes straight up to the top right corner. So the length is $sqrt{(w-x)^2+h^2}$. If $beta$ is less than that, then analogously to before, the length of the line is $$lvertsec(beta)(w-x)rvert.$$

Now, if $alpha$ is big enough such that you don't bump into the left border and $beta$ is big enough such that you don't bump into the right border, you will bump into the top border (as in my image). In this case, let $F^top$ be the orthogonal projection of $F$ onto the bottom border. Then $overline{F^top E}=hcot(alpha)$ so that the length of the line is $$sqrt{h^2cot(alpha)^2+h^2}=lvert hcsc(alpha)rvert.$$

---

Example (30°): After flipping we see $h=10, w=6, beta=fracpi6, x=2$. We compute $$operatorname{arcsec}left(frac{sqrt{(w-x)^2+h^2}}{(w-x)}right)=operatorname{arcsec}(sqrt{29}/2)approx1.19>beta.$$

So we will bump into the right border with a line length of $$lvertsec(beta)(w-x)rvert=4sec(pi/6)=frac{8}{sqrt 3}.$$

Assuming the lines begin and end on the rectangle, use trigonometric functions sin, cos. For example, in the third rectangle, denoting the line length $ell$ we have:

$$ell = 6/cos(35^circ)$$