Is it correct to say exorbitantly expensive?

Yes, it is correct.

To say that something is at a higher than expected or higher than a reasonable price is to say that the price is exorbitant. Such prices are sky-high or crazy. But one must remember that price is always defined as what someone will pay for it. Often this is only what the last guy payed, poor fellow, but the next purchase is your decision alone.