Ways to speed up Polygonizing of rasters/orthophotos (binary)?

From a orthophoto of the Wadden Sea, I want to create a polygon from a class of this TIFF/raster (binary) representing an oyster reef, calculate the area of the reef and form a polygon around the reef for further processing.

But since polygonizing this raster takes already so many hours I don’t know what I can do.

Is there any way to speed up the process or decrease the size/information of the raster? Or other ideas? Should I decrease the original resolution of the orthophoto?

Size of the TIFF is 3.9 GB and pixel size is 1.86e-7,-1.87e-7

QGIS version: 3.16.4-Hannover
Ubuntu 18.04
64 GB Ram
Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
M.2-PCIe-NVMe-SSD-Disk, 512 GB