Why do raids with one unit fail?

No one really knows. There doesn't seem to be any innate defense to villages. I think it's just hard-coded in almost. What's interesting is that stronger units can die alone if the person you're attacking has a lot lower population. There's no real answer as to why though.

Another thing to consider is that villages can get defense from a residence or palace even if they don't have any troops there. Residences and palaces give small amounts of both types of defense, as listed in this post, which can also be increased by city walls.

Also note that if you upgrade your units at the blacksmith, you might be able to survive, depending on the type of unit that you're upgrading.