How many epochs to run during hyperparameter search?

Question

If I'm doing a hyperparameter search and comparing two different hyperparameters (but not number of epochs), is there some established rule of thumb for how many epochs to run?  If I just compare after a few epochs, will that give me a good idea about how it will perform fully converged (say for example after 1000 epochs).  Is there any research papers where people have studied this question?

Brian Spiering · Answer

The general rule of thumb is to run the number of epochs until validation error starts to increase.
Sometimes fast initial learning will not lead to the best performance later.

How many epochs to run during hyperparameter search?

One Answer

Add your own answers!

Ask a Question