Why do we call it a "positive definite matrix" rather than a "positively definite matrix"?

Yes, it seems that there are linguistic reasons¹ why positive definite works better than positively definite.

^{1BTW, for that reason, I think that it was a mistake to migrate this question from the English Language and Usage (EL&U) StackExchange to the History of Science and Mathematics (HSM) StackExchange.}

It seems that when we are picking adjectives as labels for types, in English we prefer the adjective+adjective construction to the adverb+adjective one.

It is unlikely we can really answer why we have that preference any more than we can answer why we prefer e.g. all friends of Kim's to all Kim's friends; at the present state of our knowledge, probably the best we can reliably do is state trends and tendencies. Despite that, I will speculate about one possible reason, below.

Discussion

Consider this example (I'm of course not saying that the actual claims made in the example are correct):

There are two types of Indic languages, namely

(a) anciently Indic languages and contemporarily Indic languages.
(b) ancient-Indic languages and contemporary-Indic languages.

In (a), anciently Indic is a syntactic construction: it is an adjective phrase (AdjP) whose head is the adjective Indic and which has the adverb anciently as a modifier. The meaning is 'Indic in an ancient manner' (and analogously for contemporarily Indic).

According to CGEL (pp. 1657–1658), in (b), ancient-Indic is a morphological compound: ancient and Indic combine to produce a new word (and similarly for contemporary-Indic).

The point is that, to my ear at least, (b) is clearly preferable to (a).

I admit that I don't understand why CGEL is so sure that the construction in (b) is morphological and not syntactical (I will probably post a separate question about it). But if CGEL is correct about this, then perhaps this might be the reason why we prefer (b): we would like the type labels to be syntactically simple, to be actual terms, lexical items, as opposed to syntactic phrases.

There are numerous other examples that could be constructed:

We manufacture sandals in two types of blue, so we have light-blue sandals and dark-blue sandals.

(preferred to lightly blue sandals and darkly blue sandals)

Thus you end up with two types of cream: the steaming-hot cream and icy-cold cream.

(preferred to steamingly hot cream and icyly cold cream)

Positive definite

As far as positive definite, let's start with some definitions. First of all, when it comes to the notion of definiteness, the basic object is a quadratic form. The next step is to realize that to every matrix, one can associate a quadratic form. A matrix is then called definite if the associated quadratic form is definite; it's called positive definite if the associated quadratic form is positive definite, etc.

^{For a diagonalizable matrix, these properties of definiteness are easily relatable to the properties of the eigenvalues of the matrix, and some sources proceed to simply define positive definiteness of a matrix in terms of the properties of its eigenvalues. I personally dislike that practice, but that's definitely a matter of taste.}

So let's talk about quadratic forms now. We have this (source):

Definite Quadratic Forms. Since it is homogeneous, every quadratic form is zero at the origin. We call the quadratic form Q definite if it is nonzero everywhere else: Q(x)≠0 for x≠0.
…
If Q(x) is a definite quadratic form, then one of the following inequalities holds:

Q(x)>0 for all x≠0 (Q is positive definite), or
Q(x)<0 for all x≠0 (Q is negative definite).

Thus we have two types of definite quadratic forms: positive definite and negative definite.

The analogy with what I said above about ancient-Indic, dark-blue, etc. would be complete if positive definite were hyphenated. It is definitely sometimes hyphenated, including in two of the examples of usage in the OED (the ones from 1904 and 1957):

positive definite adj. Mathematics (of a function) having positive (formerly, positive or zero) values for all non-zero values of its argument; (of a square matrix) having all its eigenvalues positive; (more widely, of an operator on a Hilbert space) such that the inner product of any element of the space with its image under the operator is greater than zero.

^{1904   Trans. Amer. Math. Soc. 5 464   It is well known that there is always one such invariant, a positive-definite Hermitian form.
1948   W. V. Houston Princ. Math. Physics (ed. 2) vii. 120   The potential energy will be a quadratic expression in the coordinates that, if the equilibrium is stable, will be a positive definite expression.
1957   L. Fox Numerical Solution Two-point Boundary Probl. vii. 179   If all the λτ are positive, which is the case in many physical problems, and corresponds to some structure of the differential system corresponding to a positive-definite matrix A.., we can also assert [etc.].
1990   IMA Jrnl. Numerical Anal. 10 546   Hk is a positive definite matrix that approximates the inverse reduced Hessian matrix.}

It is an interesting question why hyphenation came to be disfavored. But the fact that it came to be disfavored does not, I think, make this case substantially different from ancient-Indic, dark-blue, etc.