Can I shift the AT gear while on handbrake without using the brake pedal?

Probably I will seem a bit silly, but still.

I have Toyota Corolla 2006 with automatic transmission.
Today, after the ride, I was curious to see how the motor behaves on different gears. Car was stopped and on handbrake/emergency brake.

I was shifting gears P->R->N->D and back. Only then I understood that I’m not pressing break pedal.
It all seemed fine. No strange noise or revs. Shifting absolutely fine.

Should I be worried about the occasional damage to AT?