Worshiping one God at a time?

Is monolatry the word you're looking for?

Monolatry, in contrast to monotheism, refers to the worship of a single god, without denying the validity of others.

Another word for this is henotheism, which is very similar to the already-mentioned monolatry. Both are religions where one only worships one God without denying the existence of other gods.

There are some good answers already offered, and I was going to jump in with monolatry myself. However I noticed that none of them actually quite meet the specifics of your question:

given that different gods exist, the practise of worshiping only one of these at a given time, then moving on to worship another god.

• Kathenotheism is somewhat similar to what you want, but doesn't actually mean you only worship one god; the kathenotheist may worship many gods, but believes only one is supreme. The ruler of the pantheon is what changes from time to time, not the object of worship.
• Monolatry doesn't move on to other gods. The monolatrist believes in the existence of many gods, but worships only one.
• Summodeism is another related concept; the summodeist accepts the existence of multiple gods, but believes that they are all aspects or emanations of one supreme being. He adopts different forms of worship according to which aspect he wish to appeal to, but considers them all the same being.
• Henotheism is difficult because it is used in different ways by different authors. As originally defined by von Schelling, it is a synonym of monolatry. Müller (the guy who coined "kathenotheism") used it to mean something like summodeism. Edward Wente seems to make the subtle distinction that the monolatrist considers only one god suitable for worship (either because other gods are unworthy, or because of a special relationship to, say, his tribe) whereas the henotheist chooses a preferred god to worship. Either way, thought, there's no chopping and changing: once a henotheist has picked a team, he sticks with it.

So, what then is the word for believing in many gods, but worshiping one at a time? It's just polytheism. That is exactly what polytheists actually did. The Romans, for example, had 12 major gods, 12 minor gods, and hundreds of indigitamenta; the Greeks had even more. No-one can possibly worship more than, at most, one or two at a time! So if you're preparing for the harvest, you offer a sacrifice to Ceres; if it's Bacchanale time, you have a drunken orgy; if you're sick, you offer a cockerel to Asclepius.

A less complex term might be serial monotheism; coined on the model of "serial monogamy". The reason I would prefer this term to many of the linguistically accurate offerings above is that it's immediately clear what is meant, and doesn't leave the reader floundering for a definition.