"At the beginning" or "during the beginning"?

Valid, yes, equivalent, no.

at

used to show an exact position or particular place:

during

from the beginning to the end of a particular period:

Which can be a period known as the "beginning."

Hence, one could say that at the beginning of a class, half the students had arrived, but during the beginning, as the teacher took attendance, all but two arrived.