How an ETF reinvests dividends

Let me answer by parts:

When a company gives dividends, the share price drops by the dividend amount.

Not always by that exact amount for many different reasons (e.g. there are transaction costs if you reinvest, dividend taxes, etc). I have tested that empirically.

Now, if all the shareholders choose to reinvest their dividends, will the share price go back up to what it was prior to the dividend?

That is an interesting question. The final theoretical price of the company does not need to be that. When a company distributes dividends its liquidity diminish, there is an impact on the balance sheet of the company. If all investors go to the secondary market and reinvest the dividends in the shares, that does not restore the cash in the balance sheet of the company, hence the theoretical real value of the company is different before the dividends.

Of course, in practice there is not such a thing as one theoretical value. In reality, if everybody reinvest the dividend, that will put upward pressure over the price of the company and, depending on the depth of the offers, meaning how many orders will counterbalance the upward pressure at the moment, the final price will be determined, which can be higher or lower than before, not necessarily equal.

I ask because some ETFs like SPY automatically reinvest dividends. So what is the effect of this reinvestment on the stock price?

Let us see the mechanics of these purchases. When a non distributing ETF receives cash from the dividends of the companies, it takes that cash and reinvest it in the whole basket of stocks that compose the index, not just in the companies that provided the dividends. The net effect of that is a small leverage effect. Let us say you bought one unit of SPY, and during the whole year the shares pay 2% of dividends that are reinvested. At the end of that year, it will be equivalent to having 1.02 units of SPY.