How did Millikan measure Voltage?

All equation's I've seen for Millikan's oil drop experiment involve quantitatively knowing the Voltage beforehand.

Voltmeters were available at the time of Millikan's experiment (1909). The voltage was applied between two parallel plates, like a parallel plate capacitor (invented in the 1746 and called the Leyden jar), so that the electric field was considered uniform at the mid point of the plates where the oil drops entered the field and equal to the voltage divided by the plate separation distance.

I don't see a way to calculate the Voltage using Coulomb's law without knowing the correct unit for coulombs

They didn't calculate the voltage. Instead, with the known electric field, they could determine the charge on the oil droplet. By repeating the experiment, they confirmed that the charges were all small integer values of a base value of 1.5924 x 10$^{-19}$C. Then they proposed that this was magnitude of the negative charge of a single electron. You can find details of the experiment on Wikipedia.

I remember doing this experiment (a very long time ago!) in my college freshman physics lab.

Hope this helps.

Back in those days the Volt was defined in terms of the potential produced by a standard reference cell. It was not ncessary to involve Coulomb's law or units of charge.