Do "time points" and "durations" have an algebraic structure?

As already been said in the comments, the relevant algebraic structure is a one-dimensional affine space. This concept originates in geometry, where it is sometimes convenient to distinguish points from vectors. You can add/subtract vectors, you can add a vector to a point, you can subtract points to get a vector, but you cannot add points meaningfully. With this correspondence, durations are vectors and time points are points. Note that these are still two separate sets, connected via common operations.

The analogy with $(0,1)$ that you've discovered is commonly used to put an affine space $A$ and the associated vector space $V$ to a common ground. Take the vector space $V oplus mathbb R$. The vectors from $V$ form a subspace of $V oplus mathbb R$ of the form ${(v,0) | v in V}$. Now, pick any point $p_0 in A$ and define an embedding of $A$ into $V oplus mathbb R$ by mapping $p in A$ to $(p - p_0, 1)$ (note that $p-p_0 in V$). This embedding has several useful features:

• Vectors behave exactly as they did in $V$, just with a zero attached as an extra coordinate
• Adding a vector to a point gives an element of $V oplus mathbb R$ with the last coordinate equal to $1$, i.e. a point
• Subtracting two points gives an element of $V oplus mathbb R$ with the last coordinate equal to $0$, i.e. a vector
• Adding two points gives an element of $V oplus mathbb R$ with the last coordinate equal to $2$, which is neither a point nor a vector (although one can speculate on the meaning of such elements)
• In an affine space, any linear combination of points with the sum of coeffieients equal to $1$ is meaningful and produces a point, while a combination with the sum of coeffieients equal to $0$ is meaningful as well and produces a vector. Likewise, if one takes a linear combination of points embedded in $V oplus mathbb R$, the last coordinate will be just the sum of coefficients of the combination, giving $1$ (a point) or $0$ (a vector) for the aforementioned cases.
• This embedding allows affine transformations to be represented as linear operators and thus as matrices, which is extensively used in 3D graphics.

However, note that none of the standard data types of C++ actually form a field (and only a few of them form a ring), due to various reasons (signed integer overflow is UB, floating-points are not even associative, etc), and durations & time points are subject to all those problems as well. What it means is that while this correspondence between durations & time points and affine spaces is a mentally fruitful one, care must be taken since one cannot in general use this as a strict mathematical model and/or prove anything about these C++ types.