Neutron Damage in HPGe Detectors (Or any semiconductor)

From what I understand, neutrons hitting a germanium detector will cause damage to it by displacing atoms from the lattice, thus creating vacancies which become a trap for charge carriers.

These vacancies in turn, broaden the peak in an unsymmetrical way.
The question is, why does this cause a broadening of the peak? (Both to the left and right of the original peak)