What determines the shape of this orbit?

Let me address your second point first; all Newtonian/Kepler orbits are elliptical; that means that the earth is currently in an elliptical orbit and will remain in one if a comet hits it (if the comet is not so large or so fast to cause the earth to cease orbiting). But I assume in your class; you are working with the circular orbit approximation; so I will use that in answering your question.

You state correctly that $v=sqrt{frac{GM}{R}}$ and so obviously $G$ and $M$ don't change, therefore $R$ must change to maintain the equality. Specifically. $frac{v_1^2}{v_2^2}R_1=R_2$. Again this is using spherical orbit approximations; in reality there are more involved differential equations that require you to specify "how" the velocity changes.

Now if a comet of mass $m_c$ and velocity $v_c$ where to crash into earth, you would have to run through the conservation of momentum calculations to find the new velocity; and plug it into the equations above. But there is a point of importance here which is "at what angle does the commet hit earth"? You see if the comet hits the earth in the same direction as its velocity; then the conservation of momentum can be done in only one direction and we are good, however if it has an angle; or is even perpendicular to the earth's velocity; then it will "squish" the orbit into an ellipse (note: since in reality we are already in an ellipse; this would just change the 'ovalness' of that ellipse).

These types of questions have many layers of depth and intricacy and you just have to decide where you want to end, if the comet is moving really fast and really heavy it can force the earth to reach its escape velocity and leave orbit; or it can force the earth to crash into the sun. Moreover because of the conservation of angular momentum of the earth's rotation about its own axis; this comet would change the tilt of the earth and in turn affect our seasons (I believe this is why Uranus has a tilt of $82^o$).

Something that might be of interest to you is looking up why orbits are elliptical; and we can explain it using Newton's Laws which I'm sure you are familiar with. This is an absolutely beautiful video by Grant Sanderson which explains it better than I ever could: https://www.youtube.com/watch?v=xdIjYBtnvZU&t=453s