meaning of fine-tuning in nlp task

There are two types of transfer learning model. One is
feature extraction, where the weights of the pre-trained model are not changed while training on the actual task and other is the weights of the pre-trained model can be changed.

According to those categorizations, static word vector like word2vec is a feature extraction model, where each vector encodes the meaning of the word.

The meaning of the word changes context. For example, “Bank of the river” vs “Bank as a financial institute”. These word2vec vectors do not differentiate between these meaning.

Current models like Bert consider the context. Bert is a language representation model. That means, it internally can represent word by contextual word vectors.

By default, Bert is a fine-tuning model. This is where my imagination about fine-tuning started to fall apart.
Let’s say, on top of Bert model we created some task-specific layer. Now, if we fine tune, by definition, the weights of the lower level (language representation layer) will change at least a bit that means, the vector of the word will also change (if we compare before and after fine tune). That means the meaning of the word change a bit because of the new task.
If my above interpretation is correct, I can not comprehend this phenomenon for example, the word vectors of task sentiment analysis are different that the word vectors (of the same word) of task question answering. Can anybody help me?

Please correct me if anything above is wrong. Thanks