Is this a case of a parenthetical remark?

It could be parenthetical, but it probably isn't.

If you mean "He finished before his birthday (by the way it was his 20th birthday, but you don't need to know that)" The you could put it in parenthesis. Use round brackets, not commas, in this case.

But usually, you want to tell people which birthday you are talking about. It is relevant information, so normally you wouldn't put 20th in round brackets. Or if it is irrelevant, then just omit it.