A finite or infinite amount plural or singular?

Should it be:

Are there a finite or infinite number of them?

or

Is there a finite or infinite number of them?

I found that ‘a number of’ is should have a plural verb, e.g., there are a number of problems, but I’m unsure about what happens after adding finite or infinite.