Do I need the key to extract a Borg backup from my USB flash drive?

It depends on how you have initialized your backup repository. If you used "repokey" or "repokey-blake2" as encryption mode, the keyfile will be stored inside the repository and you will only need the passphrase to extract the backup. However, if you used "keyfile" or "keyfile-blake2" you have to provide the keyfile to borg, too.

If you do not remember which encryption mode you have used, you can check using borg info:

borg info /path/to/repo