How can I remove a broken plastic piece from a coolant box?

You need to treat it like any other broken bolt. To get a metal bolt out, you'd need a drill and easy out. Since this already has a hole in it, 1/2 the work is already done. You just need something which you can use to wedge against the inside of the piece so you can turn it. I'd suggest you use something like a flat file ... the end which fits into the handle. Wedge it inside of it and turn it. The right size chisel would most likely do it as well. You could also think about splitting it, but you'd run the risk of messing up the threads, so you'd have to be careful.