How can reflection and refraction be explained classically and microscopically?

Consider a plane polarized light wave impinging on glass. The charges inside the glass oscillate in some way so that the original wave is canceled, and both a refracted and reflected wave are produced.

We start with a bunch of charges all oscillating in the same direction (presumably), and somehow the charges produce radiation in exactly three directions.

Is the assumption that all the charges in the plane oscillate in the same direction incorrect? Does something different happen right at the interface? Otherwise, how can charges all oscillating the same way produce radiation in three directions, rather than one, two, or infinitely many?

All oscillators oscillate in the same direction, we know this from macroscopic theory where polarization has the same direction everywhere.

The net effect seems to be to cancel the primary wave in the medium and produce another wave of different wavelength and direction. (But this is only apparent effect in macroscopic theory. It does not mean that the oscillators in the medium do not experience force due to primary wave.)

This also happens only under special circumstances: a boundary that is long and smooth; the medium is dense enough, so the oscillators are close to each other so scattering is limited.

Should the boundary be rough or comparable in length to wavelength, the resulting radiation would be probably much more complex than two waves. Also if the medium was low density gas or dust, there would not be a single refracted plane wave, but the radiation would be more diffuse in all directions.

I know you are not interested in macroscopic reasons, but they are the most reliable explanation as they do not use any particular model of the medium. They are the guiding information to use when setting up and analyzing a microscopic model.

The above conditions translate into details of the microscopic model - oscillators are located in half space, they are distributed with uniform density high enough so mutual distances are much smaller than wavelength; their positions are limited by a planar boundary.

I do not know how to analyze mutual interactions of many particles under those conditions and answer the question: why in the macroscopic description the primary wave is not present and why the refracted awave has different wavelength and direction. Part of the answer is finding a reasonable connection between the microscopic field and macroscopic field for this kind of model, which is not easy (they have different wavelengths and the macroscopic field has to mimic the field of force experienced by the oscillators).

But it is plausible that if the oscillators are close enough and have uniform density (uniform index of refraction) and are excited by a single plane wave (primary wave), then the elementary secondary waves add up with random phases and tend to cancel each other except one direction, where they will reinforce each other.

It is similar to how diffraction on special gratings results in radiation being transmitted only in certain directions, or to phase-locked antenna arrays, which are designed to radiate in a few (single) desired directions.

Although you stated you are not interested in Huygens' principle, I want to add a note on this explanation. I will need in my answer the expression for the electric field of a radiating dipole

$$boldsymbol{rm E}left(boldsymbol{r},tright)=-frac{omega^{2}mu_{0}p_{0}}{4pi}sinthetafrac{e^{iomegaleft(frac{r}{c}-tright)}}{r}hat{theta}$$

This expression assumes the dipole oscillates in the $hat{z}$ direction. Now look for example at this picture

taken from here. This illustration seems to discard the polarization of the incoming wave (as you said), but if you think about it more - it turns out it doesn't. The radiation in the plane of incident is circular only for $s$-polarized light, since then each dipole is oscillating in the $z$ direction (in and out the page) and radiating a field given by

$$boldsymbol{rm E}left(r,theta=frac{pi}{2},varphi,tright)=-frac{omega^{2}mu_{0}p_{0}}{4pi}frac{e^{iomegaleft(frac{r}{c}-tright)}}{r}hat{z}$$

independent of $varphi$ and $s$-polarized too. If, on the other hand, you want to treat $p$-polarized light, then each lattice point should radiate like in this image

taken from here, and it will definitely have other consequences from the $s$-polarized dipoles. A popular example is the existence of the Brewster's angle, which is the result of the dipole not radiating on its oscillations axis. Also, as before, you can see that the polarization of the far field radiation is parallel to the direction of the oscillations of the dipole. This means that the $p$-polarization is maintained.

As is typical for these problems, let's assume a solution and then show that it satisfies Maxwell's equations. We'll have the interface of vacuum and a medium at the $x=0$ plane. The electric field shall be taken as

$$ vec{E}(vec{r_{x<0}}, t) = exp(i(k y sintheta - omega tau)) left{0, 0, exp(i k xcostheta) + r exp(-i k xcostheta) right} $$

i.e. the sum of the incident and reflected wave in vacuum. On the other side of the boundary it shall be

$$ vec{E}(vec{r_{x>0}}, t) = exp(-i omega tau) left{0, 0, exp(i (x k_x + y k_y)right} $$

i.e. the refracted wave. The corresponding $vec{B}$ fields are easily found from $nabla times vec{E} = -frac{partial vec{B}}{partial t}$.

$$ vec{B}(vec{r_{x<0}}, t) = frac{k}{omega}exp(i(k y sintheta - omega tau)) times left{(exp(i k x costheta) + r exp(-ikxcostheta))sintheta , (-exp(i k x costheta) + r exp(-ikxcostheta))costheta, 0 right} $$

$$ vec{B}(vec{r_{x>0}}, t) = frac{1}{omega}exp(i(k_x x + k_y y - omega tau)) left{k_y t, k_x t, 0right} $$

By matching the $vec{E}$ fields at $x=0$ we find $t=1+r$ and $k_y = k sintheta$. By matching the magnetic fields we also find

$$ t = frac{2kcostheta}{k_x + kcostheta} $$

With some inspection we can realize that we have reached a possible valid solution which consists of three waves and satisfies Maxwell's equations. But why must it be so? Why can't we have $r=0, t=1, k_x=k costheta$? The incident wavevector is given (i.e. $omega/k=c$ and $theta$ is defined).

We need to relate the wavevector inside a medium to the frequency and material properties. A classical approach says that our medium is a somewhat polarizable mix of negative and positive charge carriers which can be displaced by an electric field, optionally with friction and a restoring force. Putting an equation for a harmonic oscillator together with two of Maxwell's equations we get

$$ k vec{P} + gamma dot{vec{P}} + m ddot{vec{P}} = q^2 vec{E} $$

$$ -frac{partialvec{B}}{partial x} = mu_0 dot{vec{P}} + dot{vec{E}}/c^2$$

$$ frac{partial vec{E}}{partial x} = -dot{vec{B}} $$

Where $q$ is the effective charge density (depends on frequency, could be shell electrons, could be the ions, etc), $k, gamma, m$ are the effective spring, damping, and mass constants normalized to unit volume. Finding the steady state solution of this set of equations will give us for a given $omega$ a wavevector which is generally different from $omega/c$, so $k_x$ will generally be different from $k costheta$ and thus a solution for a wave incident on a surface will necessarily need both a refracted and a reflected wave.

It should not be too hard to reproduce this logic for a $p$ polarized wave.