What do we observe with gravity in a superposition?

It is very likely that we will not observe gravitational effects of superpositions in the near future, given the discrepancy between the largest objects we can hold in a coherent superposition (a few atoms) and the smallest gravitational forces we can measure (between macroscopic objects weighing multiple grams). We haven't even conclusively measured the gravitational interaction of antimatter!

As you mentioned, we also have no quantum theory of gravity, so we cannot really predict what would happen. The simplest approaches to quantum gravity (which are not mathematically consistent and produce unwanted infinities) describe the gravitational field in terms of gravitons, so the outcome would be the same as with any other force (like electromagnetic interaction): After exchanging a force carrier, two particles would become entangled.