Question Regarding the conservation of momentum in an inelastic collision of two rods

In an isolated system, angular momentum is always conserved.

The nail provides a connection between your system of two rods and the outside world. So your system of two rods is not isolated. You can't guarantee conservation of energy (you said already that the collision is not elastic). The nail will exert an external force on the system of two rods, so you should not expect conservation of linear momentum. And you have already intuited that the nail will exert a torque on the system, so you can't rely on conservation of angular momentum, either.

However, the total angular momentum in a system depends on your choice of origin. (An exercise that you should do if you haven't: prove to yourself that an isolated object moving in a straight line at constant speed has constant angular momentum, by choosing some centers of rotation both on and off of its line of motion.) The same holds for an external torque: the magnitude of the torque $vec tau = vec r times vec F$ depends on the displacement $vec r$ between the point of application of the external force $vec F$ and your arbitrary choice of a center of rotation.

If you can find a point where the torque $vectau$ from the nail is guaranteed to be zero, the angular momentum in that frame will be the same before and after the collision. You have correctly identified this privileged center of rotation as the nail, which sets $vec r = 0$.