In French, can you say « passer X à côté de Y » to express the idea of "I wanna stay with you forever"?

In terms of grammar, both sentences are correct, and so is the sentence suggested by Laure.

However, as someone mentioned, there is an expression, written "aux côtés de", that would fit perfectly in this context. This expression is the figurative version of "à côté de". It means "to be at someone's side, but not litterally". In your example, you don't need to be next to your beloved one all the time to be considered as "à ses côtés". This very subtle difference is discussed here.

In my opinion, "aux côtés de" implies a form of permanent support that you give to the other person. It means that whatever happens, you are standing next to him/her, no matter if you are with him/her physically or not. For this reason, this is the expression I would choose in your very context.

Tu es la personne aux côtés de laquelle je veux passer le restant de mes jours.