Drywall or cement board in gap between bathtub and corner

The spot next to the tub is a trouble area. I have found that the more robust the wall material is (i.e. less porous, harder) the longer it will last without water damage, that is why I tiled that area next to my tub.

"...should I use cement board or dry wall?"

I recommend cement board. Furthermore, I encourage you to use a harder plaster material (instead of joint compound) such as a "setting-type" plaster (e.g. water putty, "fixall", plaster of paris, etc.) as the base coat on top of the cement board and at the joint.

Setting-type plasters are a bit hard to work with because they have a short working time, they set hard, and are difficult to sand or smooth after they set. However, hard plaster as a substrate to normal joint compound should help prevent structural damage in case of water contact over time. NOTE- all plaster is porous, but hard plater is less porous. Use joint compound as the final coat so you can sand it flat and smooth, use a high quality paint with a gloss, and caulk well.

Not only cement board (hardi is fine) but you must go 3-4 feet up. Do not just cut a strip near your tub. For a tub the absolute minimum is 3' up around the tub is either cement board or drywall with membrane or drywall plus cement board.

The 3' isn't "code" either. Your local inspector might require higher. I am just giving you the minimum so you do have issues later - this is knowing that the tile will extend past the 3' mark.

The last thing in the world you want to do with a tub or shower is put a strip of backing along the bottom creating TWO seams in the at most risk area!

My answer here - Should the backer board be in front of or behind the tub flange? - goes over options that you have. But let's just say you want to do this quick and dirty. I would cut out the entire tub at 4' all the way around. Put in 1/4" drywall and 1/4" hardi - with plastic sheathing in between. Both of these are very very easy to work with and can be put up and sealed in a couple hours even for a newbie. Consider this vs trying to fit that strip and fill in the gaps securely which could take the same amount of time plus leave a lip.