Is my DSLR viewfinder dark because there's no battery in the camera?

Your camera, like many modern DSLRs, has a “transmissive” optical viewfinder which requires battery power in order to be fully transparent. Here is what Canon says about Transmissive Viewfinders in Digital camera features: Transmissive LCD viewfinders:

This new viewfinder uses a transmissive LCD which, unlike traditional viewfinders, does not feature interchangeable focusing screens but, instead, uses an LCD screen through which light passes, giving an optical view but with the ability to overlay extra information on the screen as required.

In An overview of the Intelligent Viewfinder Display system in the EOS-1D X and EOS 5D Mark III, Rudy Winston writes:

The LCD overlay does require a tiny amount of electrical power to operate. This is obviously no concern when the camera is turned on, but if the battery is removed the transmissive LCD suddenly loses a lot of brightness and contrast. This is perfectly normal and will return to full brightness once a battery is reinstalled in the camera (the camera doesn't have to be turned on; it only requires a functioning battery pack to draw power for proper viewfinder operation).