My ceiling fan light turns off then back on after I replaced the switch from a dimmer to a standard

If the lights flicker (or just occasionally blink off then on) when the fan is running, it could be that one of the wire connections is loose and vibrating as the fan runs, causing an intermittent disconnect.

Turn off the power and double check that all of your wire nuts are securely tightened. Hold the nut and pull on each wire to ensure it doesn't wiggle at all or worse, just come out.

As a side note, a standard, fully variable dimmer will allow the fan to run at speeds it's not designed to, thus creating the annoying humming/buzzing sound you don't like. If you purchase a 3-position dimmer - usually labeled as a 3-speed fan switch - it will allow the fan to run at the 3 speeds it's designed to, so you can have different settings from the wall.

We've used these on all of our ceiling fans for nearly 30 years and nary a buzz. Once installed, put the wall switch on "high", then pull the fan-speed pull-chain on the fan until the fan is actually on it's highest speed. Now the wall switch will correctly control the fan speed without issue.

You will, though, need some extra wire in the wall to control the lights & fan independently. If you don't have that, then ignore the fan speed controller suggestion, but do double check all your wiring.