What is bad with using the wrong cable housing?

The opposite case, using lengthwise-braced shifter cable for a brake:

1. The brake inner cable probably won't fit through the shifter outer housing, because its slightly thicker than a gear cable.
2. Brakes might feel harsher because there's no shock absorbing from the housing's spiral.
3. The gear outer cable would be a loose fit into levers and calipers, permitting the ingress of more water/dirt.
4. Also, a smaller cable outer will move around inside a socket more than a good fitting outer, letting it mush the ends a bit more.

spiraled brake housing would compress when shifting which would diminish the accuracy and using lengthwise cables for brakes can't handle the stress and split wherever the tension is too great.

this article explains as much: https://www.pinkbike.com/news/To-The-Point-Shift-Cables-2013.html