Should I change transmission fluid if it's black?

Rough or slow gear engagement (as mentioned in the comments) could indicate that the shift solenoids are having trouble actuating due to the dirty fluid. As racefever suggested, you should change the filter, but other than having to buy new fluid, I personally don't see any reason not to change the fluid at the same time.

I would go ahead and change the transmission fluid & filter, even though it doesn't seem to be posing a problem at the moment. I've heard/read the same things that changing old fluid can dislodge debris and leave you worse off than you previously were. While I have never personally run into that problem when changing fluids for friends/family, it wouldn't have become a common concern if it never happened in the first place. As such, you assume an inherent risk when choosing to replace the fluid on a high mileage vehicle, but I am of the belief that if you do your research, have the right tools, take your time, and get the job done right (or find a shop that cares), it will be fine.

I attribute 'dislodged gunk' failures to quickly-lube style oil change garages screwing something up in the change/flush process, or using it as an excuse to cover their butts for more obvious oversights (ie replacing with used/improper fluid, filling to wrong level, not removing/cleaning the pan, using old/improper gaskets, forgetting to refill fluid entirely, under/over tightening the drain plug).