Is this sentence grammatically correct? (tenses)

"I know. You dreaded our meetups because my condition would always
have worsened by the time we met again."

Is the use of "would have" here correct? If so (or if not), would someone mind explaining it to me? I’m really struggling to grasp the past tense.

Thanks in advance for your help!