What are New Latin's comma rules?

While I don't know about the history of proponents and critics of this punctuation style who have explicitly discussed it, like you and others here I've also noticed it has been very common to use this style of punctuation, with a heavy use of commas, in Latin for some centuries now.

You're only asking about what the rules are though, and that's easier to answer. The main differences, I believe, are:

1. A comma is used between a relative clause and its head before it.
2. A comma is used before a relative clause that has been attached at the end of a sentence (a grammatical pattern absent in contemporary, but not older, English.)
3. While not precisely about punctuation, lowercase letters are often used after sentence-final punctuation like periods (full stops) or question marks within a line of verse.

Examples

meus amicus, qui cum Panaetio vixit, ...*
‘my friend who lived with Panaetio’
(Cicero, De Oratore III.78).

[P]erbeati fuisse, Quinte frater, illi videri solent, qui in optima re publica, cum et honoribus et rerum gestarum gloria florerent, eum vitae cursum tenere potuerunt.
‘They seem to have been very fortunate, my brother Quintus, [and I mean] those who in the best time of the Republic were able to do the cursus as they flourished in the glory of their deeds and honours.’
(Cicero, De Oratore I.1; qui refers to illi.)

Cras vives? hodie iam vivere, Postume, serum est:
Ille sapit, quisquis, Postume, vixit heri.
‘You will live tomorrow? To live today, Postumus, is already late: the wise man, Postumus, has [already] lived yesterday.’
(Martial, Epigrammata V.58; and note the comma before quisquis due to being a relative clause attached at the end.)

Conclusion

I think these three things will cover most of the strangeness you will see. The rest is just Latin poets being fond of hyperbaton and parenthetical remarks, and editors trying to keep the punctuation sensical by distinguishing the clause levels of the syntax:

[Laxavit melior.] vale, libelle:
Navem, scis, puto, non moratur unus.
Goodbye, my book: a ship, you know, I think, is not delayed by one passenger.
(Martial, X.104; the main verb is moratur, and both scis and puto are parenthetical remarks.)

(By the way, I turned the sentence passive in English to render the word order in a natural way —a very common use of the English passive, in contrast to Latin which often uses plain object-subject-verb or object-verb-subject word order instead. Moror is both deponent and a "labile ambitransitive" (to use linguist Martin Haspelmath's term): moramur ‘we're lingering, waiting, delaying ourselves’, moramur hostes ‘we're delaying the enemy, making them wait’.)

I took the above passages from the PHI Latin corpus, but really just about any edition of the texts I mentioned here will have the punctuation that appears in my quotations.

Outside Latin

I should really mention, too, that this style of punctuation was very common in early modern Western Europe. You can find English and Spanish texts from that time using it. E.g., from the Preface of the King James Bible:

Lastly, we have on the one side avoided the scrupulosity of the Puritans, who leave the old Ecclesiastical words, and betake them to other, as when they put washing for Baptism, and Congregation instead of Church: as also on the other side we have shunned the obscurity of the Papists, in their Azimes, Tunike, Rational, Holocausts, Præpuce, Pasche, and a number of such like, whereof their late translation is full, and that of purpose to darken the sense

(Notice the strange use of the colon for a longer pause, which you still find in Lewis & Short's and Smith & Hall's dictionaries, from the late 19th century, after each ancient citation inside an entry.)