How do you call it when someone makes faces while trying to remember something

Not sure a single word can be found. I would go for squeezing her eyes. Children do that:

On PsychologyToday I found this:

Have you ever noticed that when little kids are trying their hardest to remember something, they squeeze their eyes closed? They may be on to something. Several studies over the past few years show that closing your eyes boosts brain power, improves memory, and even ignites creativity.

You could include closed or not. So your sentence would look something like

She was squeezing her eyes (closed), trying to remember where she’d seen my aunt, and that made her look so cute.

Children do look cute when they squeeze their eyes.

You can use the expression "to strain one's facial muscles".

(ref. 1) The effort strained his facial muscles. She returned the smile. “Just fine. I have the latest data on compression strength and fracture resistance for the honeycomb structure.

(ref. 2) Kuster~Laap strained his facial muscles searching his memories. »I'm not very familiar with these convoys.

You can also use "screw one's face up". (OALD¹)

(ref.) Monk screwed his face in concentration . “ How about Dixie ? That's better yet . ” “ Dixie what ? ” Barney asked patiently . “ Dixie nuthin ' , ” Monk said . “ Just plain Dixie . ”

¹_{After correction; see user Mitch's comment}