Can hydrogen plasma react with oxygen?

The solar wind is primarily made of protons and electrons (i.e., the constituent parts of $^{1}H$) with roughly ~1-5% alpha-particles and then much much smaller fractions of heavier ions (e.g., see https://iopscience.iop.org/article/10.3847/1538-4365/aab71c/ and references therein). That is, the solar wind is a quasi-neutral plasma.

What if you put hydrogen in a vacuum and turned it into a plasma?

No, neutral hydrogen atoms exposed to vacuum (in the absence of other effects) will not spontaneously ionize due to the vacuum alone.

There is no oxygen in the vacuum, but once you eject the plasma would it react drastically with the oxygen? Would it explode?

I think you are asking whether shooting a plasma into Earth's atmosphere would cause an explosive reaction, right? The answer is most likely not. The recombination rate would be so high that the plasma would not live long, though it would heat up the local atmosphere and cause some secondary electron generation, at least temporarily.

The question may be better posed if you asked whether there is a critical limit beyond which a runaway ignition would occur. This was a serious concern for the scientists initially working on the first atomic weapons (e.g., see https://blogs.scientificamerican.com/cross-check/bethe-teller-trinity-and-the-end-of-earth/). It turned out to not be a concern for any of the bombs tested, obviously (and thankfully).

Would it be a bigger explosion than if it wasn't plasma?

I am not sure what you are asking. Are you asking whether an explosion would be bigger if neutral hydrogen were released into our atmosphere? Neutral hydrogen would not cause an explosion but would combine with other hydrogen to form diatomic molecules, if not already in that form, and maybe some $OH^{+}$ ions temporarily. However, most of this would occur very rapidly as there are ~$10^{23}$ particles per mole sitting around waiting to react.

Could this be a propulsion mechanism?

I assume you mean something similar/analogous to rocket propulsion? I suppose in principle, one could create thrust if the plasma were ejected from one side of an object. The problem is that it would be extremely inefficient, energy-wise. It takes a lot of energy to ionize particles and then you need to generate enough energy to accelerate said particles so momentum balance produces a net thrust on the object. This is a lot of energy that is not free, i.e., it would need to be supplied by something within the propelled object. Thus, I would guess this would not make a feasible propulsion system.