Why dependency direction is determined by levels and not by change frequency?

There are two perspectives to consider here. The first is that we want to isolate (to the extent that is feasible) the higher-level policies (in this case, the "core" of the application, or the domain model, the thing that solves the core problem) from considerations that come from lower-level libraries/frameworks/tools that we use, but that aren't essential to the core problem itself. Of course, some aspects of the low-level details cannot be ignored for practical reasons, but generally speaking, some amount of isolation can be achieved. You might consider that a bird's-eye view of things.

The second perspective is one that presents a closer look at layer boundaries. You control dependencies between layers (and often within layers) by having the lower layer depend on an abstraction in a higher-level layer. It is these abstractions¹ that need to be comparatively stable (change less frequently); the rest of the layer is hidden behind them.

[lower-level layer]-------->[higher-level layer]

is actually:

[low-level detail]---------||----->[abstraction]<-------[high-level policy]
                           ^          ^
               (layer boundary)     (abstraction is owned by higher-level layer)

So, as you're developing, you'll recognize the more stable aspects of the problem and codify these into things like input and output ports² (ports and adapters style), or other kinds of inter-layer interfaces, and then you'll be able to refactor and restructure behind those. Since at the start you have limited understanding of the domain and of the forces of change involved, chances are that you'll have to revise some of these abstractions as you go along, at certain points; but if things are going right, they should stabilize further over time. That said, it's not advisable to invest the same amount of design effort in all parts of the system (there's a cost/benefit tradeoff to be considered); but for those parts that experience the most change and the most activity, you want to get to something that's reliant on stable abstractions and is fairly open to the kinds of changes that are most likely³ (open/closed principle).

---

¹ By abstraction, I don't mean just an abstract class or an interface type (a la C#, Java); I'm using the word in a broader sense. E.g. a Facade to a subsystem or a component is also a kind of an abstraction.

² Again, these aren't necessarily (C#, Java) interfaces. Initially, it may not even be a separate element; it could just be the public interface (public methods and properties) of a class. But that might change as the system becomes more complex; e.g., at some point you might do something like refactor the code to extract a new class (or two) out of the original one. The new classes then become internal to the layer, with the original one serving as a kind of a boundary object.

³ You're not supposed to guess what's most likely, although you might be able to do so, to some extent. Instead, this comes over time, with the growing understanding of the domain - as long as you consciously take note of what is actually going on as the project evolves.