Is "at least" a conjuctive adverb? Can it be used in the middle of a sentence separated with two commas?

For example, this sentence:

It takes ten years to grow a tree, at least, that is what my father said.

Is this grammatically correct? Or is it a conjuctive adverb and then should be used as:

It takes ten years to grow a tree; at least, that is what my father said.

Not wondering which one sounds better. Only wondering if the first one is technically grammatically correct.