What is meant by unitary time evolution?

Quantum mechanics is a probabilistic theory and all probabilities must always add up to 1. This puts a constraint on the theory; as the state of the system evolves in time the total probability must remain fixed.

If we denote the state of the system at time $t$ as $|psi(t)rangle$ then we can define the time-evolution operator $U(t^prime)$ as the operator by $$ U(t^prime)|psi(t)rangle = |psi(t+t^prime)rangle $$ so that it takes a state at time $t$ and gives us the state a time $t^prime$ later. Now the requirement that probability is conserved tells us that

begin{align} langle psi(t) | psi(t)rangle &= langle psi(t+t^prime) | psi(t+t^prime)rangle &= langle psi(t) |U(t^prime)^dagger U(t^prime)| psi(t)rangle end{align} for all states $ | psi(t)rangle$. This implies that $$U(t^prime)^dagger U(t^prime) = 1$$ which implies that $U(t)$ is a unitary operator.

Consequently this type of time evolution is known as unitary time evolution. This has many important results for what is possible when a quantum state is time evolved, such as the no-cloning theorem

Yes, there is a difference. Unitary time evolution is the specific type of time evolution where probability is conserved. In quantum mechanics, one typically deals with unitary time evolution.

Suppose you have a state (at time $t=0$) given by $|alpha rangle$. To find the state at a later time $t=T$ given by $|alpha (T) rangle$, we apply the (unitary) time evolution operator $U$:

$$U|alpha rangle = |alpha (T) rangle$$

where

$$U = e^{-iHT}$$

and $H= $ Hamiltonian of the system, which is Hermitian.

Conservation of probability mathematically means:

$$langle alpha|alpha rangle = langle alpha (T)|alpha (T) rangle$$

Physically, it means that the probability of existence of the quantum system, described initially by the state $|alpha rangle$ and later by $|alpha (T) rangle$, does not change with time. The quantum system exists at $t=0$ with probability $=1$, and also at $t=T$ with probability $=1$. The state evolves in time from $t=0$ to $t=T$, but no information about the quantum system leaks out during this time interval. The system that existed at $t=0$ continues to exist in its totality later at $t=T$. This is physically meaningful to demand from a theory, because information about a state should not get lost during evolution. Sure, the information might get tangled up as time goes on, but all of it should still be there, in principle. For example, if you burn a book on coal, information inside the book is lost for all practical purposes. But, all information still exists, in principle, encoded in the correlations between coal and ash particles.

Sometimes, it is easier/useful to explain certain phenomena by abandoning unitary time evolution, for instance, in unstable particles or radioactive decay. There, as time goes on, the mother state decays into daughter states. If you observe only the mother state subsystem, it does not undergo unitary time evolution because it loses information about its state with the passage of time. The information is lost to the daughter state subsystems. Probability (of the mother state existing) is not conserved; it decreases (exponentially) with time. If you look at the full system, as whole, evolution is unitary, as expected. But in radioactivity, we often just need to know how the mother state subsystem disintegrates.

Yes, there is a difference between time evolution and unitary time evolution. Non-unitary evolution stems from having a subsystem.

Consider a statistical mixture of pure systems, i.e. assume that there is an ensemble of systems so that a fraction $p_1$ is in state $|1>$, $p_2$ are in state $|2>$, and so on. Then the entire statistical ensemble can be described by a so-called density matrix, $rho = sum_i |i><i|$. We can consider the evolution of this density operator, which is often written as the so-called Liouville equation $frac{drho}{dt}=mathcal{L}rho$. Here $mathcal{L}$ is the "Liouvillian", an operator that transitions $rho$ in time much like $U$ transitions the pure state in time for a pure state.

If the dynamics is governed by a Hamiltonian, then the Liouvillian (the "propogator" of the state) is simply its commutation relation with the state, $mathcal{L}=-i/hbar (Hrho-rho H)$. This corresponds to standard Hamiltonian evolution, and is unitary.

Consider, however, looking at only a subsystem of the extended system. The state of such a subsystem can be described by a "reduced" density operator, which is really just a density operator (matrix) describing just the subsystem. However, even though the evolution of the extended system is unitary and follows a Hamiltonian, the evolution of the subsystem's density generally does not follow a Hamiltonian and is not unitary. Instead, it is often "dissipative". For example, the subsystem might dissipate energy into the environment and cool down into a thermal state.

There is no general simple form for the Liouvillian (the propagator) for subsystems. A relatively simple case is when the subsystem evolves in a Markovian manner, which is often the case for systems equilibrating thermally. Then the Liouvillian has a unitary term, driven by a Hamiltonian just as above, but also a correction: a non-Hamiltonian term, a term which follows a special form called "Lindblad form" rather than the commutation-with-the-state written above.

So - non unitary evolution is actually everywhere. Whenever you can't neglect the interactions with the environment, so that you e.g. equilibrate thermally with it, you have non-unitary evolution (there are a few caveats, but it's almost always true). What's difficult is getting a unitary evolution going. You have to isolate your system from the environment enough to be able to neglect thermalization etc.

The biggest difference, perhaps, between these two kinds of evolutions is that non-unitary evolution is (generically) not reversible. That's how you get thermalization. Non-unitary dynamics can also change the populations, reflecting for example a decrease in highly-energetic states as the system cools down towards its ground-state.