Approach fpr extracting/cropping features images using deeplearning and no annotations

So all you have is a set of images containing only cropped hats? One idea is to leverage synthetic data to learn an object detector. It would be better if you had crops of natural images with hats, not just hats alone (as it will lessen the domain shift).

Basically, the following: (1) take your hat images and generate "ground truth" images by randomly pasting a random number of hats (in random poses) into a random background image (from anywhere, ideally similar to the backgrounds of images that you plan to run on at test time). (2) Train an object detector model (e.g. Mask-RCNN, Faster R-CNN, or YOLO9000) using your synthetic dataset (since you know the "real" hat positions in the images, since you made them). (3) given a video, decompose it into a sequence of image frames, and run your trained detector frame-by-frame. (4) use the output of the detector to obtain the hat positions and crop them out.

A simple extension would be to generate synthetic videos for training, where your hats move around and you keep track of them; another one might be to try to make the synthetic images more realistic, by say attaching random people or at least heads underneath the hats.

The main challenge is the domain adaptation between synthetic and real images. Any of the many modern techniques for handling domain shift would likely be helpful. For instance, using GANs (also here).

It may interest you to know that this simple synthetic methodology has proven useful in practice. For instance, Tremblay et al at NVIDIA recently used it to also train an object detector via domain randomization. Other applications include robotics.

---

A much more trivial, but easier to implement solution is to train a hat classifier $C$, which takes in an image or image patch, and output whether the patch contains a hat or not. Using your cropped hat dataset as the true positives, and getting true negatives by randomly cropping patches from any random dataset of images, you can train $C$. Given a video, you then simply use a sliding window approach, where for every image patch at every frame, you run $C$ on it. You can then threshold the output of $C$ and/or crop out areas with high values from the classifier.