Conductor with hollow cavity under external electric field

Say I have a conductor with hollow cavity inside with no charge inside the cavity. Now the conductor is subjected to external electric field. Now we know that electric field inside conductor is zero, hence the inside cavity is equipotential. Hence for the given charge distribution above, there will be E-field inside cavity, hence the cavity will not be equipotential.

I understand that there can not be any charge (even if net charge is zero) on the inside cavity and the reasoning.

But lets consider an alternate viewpoint. When the conductor is subjected to the external E-field for the first time and for that initial nano-nano seconds when charges are running around to make the field inside conductor zero. During this charge distribution time, under the field some +ive charge can be deposited on right side of cavity, and hence some -ive charge on the left face of the cavity. And also the net charge around the cavity is zero. Now the E-field lines do not know that they have to satisfy the equipotential constraint. They work such that field inside conductor is zero. Also please note that the charge distribution around the cavity will reinforce a field opposing the applied external field.

What I can not understand why the above charge distribution is not feasible. I understand from equipotential constraint, it can’t be, but field lines do not know that, they have no memory! Also when I searched in google, one argument was flux is zero inside conductor, but zero flux does not imply absence of field lines! Also note that the above charge distribution on cavity satisfies Gauss law, as I assumed that idential amount of +ive and -ive charge is on the cavity wall.

Counter Argument: if there is such charge distribution, there will be field inside cavity, and hence equipotential won’t hold. I got it, but field lines do not know that.

Can someone please throw some light on this.