Aren't Qubits just Ternary?

Simple answer: no.

Qubits are the same as regular bits in almost every way; except two fundamental differences, superposition and entanglement (I will only address superposition since it is the focus of your question). Qubits can only be observed in $| 0 rangle$ or $| 1 rangle$, something you have probably heard as 'wave function collapse', but the basic idea is that when we observe qubits, they carry the same information as regular bits. Which begs the question; what is superposition?

The way quantum mechanics works generally, is that there are certain states we can observe for a given system; whether that be energy, spin, angular momentum, etc. And observations do not commute, this means that our state in fact cannot be described as existing in a single state, rather it must be a in a 'distribution' of states; or better said a 'superposition' of states.

There are many nuances and subtleties as to how these probabilities come up, and why they are represented by complex numbers, but the basic idea is as follows; a Qubit $| x rangle$ is written as :

$$| x rangle = alpha| 0 rangle + beta| 1 rangle$$

We have a probability $|alpha|^2$ of finding it in $|0 rangle$ and probability $|beta|^2$ of finding it in $|1 rangle$, we can never actually measure it "in between 0 and 1". Notice also that superposition is a fundamental aspect of quantum physics, which is nature. It is how nature works as hard as that may be to understand, and we use our engineering and computer-science techniques to take advantage of this superposition for the implementation of various algorithms.

Since your question was about why qubits are not ternary this would be a good place to stop, but you might at this point be wondering, that just seems like a more complicated way to do normal computing... in part you would be correct. However, the idea of superposition and entanglement opens a new Pandora's box of computational methods, and allows us to approach some problems in different ways, which is what makes quantum computers so cool!

Qubits have more than three distinct states. Here are example six such states

$$ |0rangle |1rangle |+rangle = frac{|0rangle + |1rangle}{sqrt{2}} |-rangle = frac{|0rangle - |1rangle}{sqrt{2}} |{+i}rangle = frac{|0rangle +i |1rangle}{sqrt{2}} |{-i}rangle = frac{|0rangle -i |1rangle}{sqrt{2}}. $$

In fact, a qubit has an infinite number of distinct states. For every $theta in [0, pi]$ and $phi in [0, 2pi)$

$$ cosfrac{theta}{2} |0rangle + e^{iphi}sinfrac{theta}{2} |1rangle $$

is a distinct pure state of a qubit.