General treatment of connections and covariant derivatives

The standard reference for connections is Foundations of Differential Geometry by Kobayashi & Nomizu (especially volume 1). This book contains only a treatment of connections on principal fibre bundles, which also include vector bundle connections as a special case.

The case of general Ehresmann connections are not treated here, however I'd argue that those are not super relevant to physics (some theories based on Finsler geometries use them, also Lagrangian mechanics can be reformulated in a way that the equations of motion are geodesics of a non-equivariant/non-linear connection on the tangent bundle, but these are kinda fringe cases).

A good discussion on connections (including the general case) can also be found in Natural operations in differential geometry by Kolár/Michor/Slovák ( https://www.mat.univie.ac.at/~michor/kmsbookh.pdf ).

A nice overview of the various different formalisms for connections and their relations to each other can be found in A Comprehensive Introduction to Differential Geometry by Spivak, especially volume 2 or 3 iirc (I don't remember exactly, and I am not in position right now to check).

Gauge Theory and Variational Principles by David Bleecker is a treatment of (classical) Yang-Mills theory and General Relativity via principal fibre bundles, so a good link between mathematics and physics can be found in this book.

Natural and Gauge-Natural Formalism for Classical Field Theories by Fatibene and Francaviglia is like the big brother of Bleecker, it is basically a book on the mathematical structure of classical field theory in general, thus it contains plenty of material on connections.