Non Max Suppression and Object Detection

Question

My understanding on how non max suppression work is suppress all overlapping boxes that are over  jaccard overlap threshold (may be 0.5). The boxes to be considered are on confident score (may be 0.2 or something). My knowing is if there is boxes that got score over 0.2 (may be score is 0.3 and overlap is 0.4) the boxes won't suppress. By this way one objects will be predicted by many boxes,one high score box and many low confident score boxes but I found that the model predict only one box for one object. Can someone enlighten me? 
I currently viewing the ssd from  https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Object-Detection

Here is the coding..

#Finding Jaccap Overlap and sorting scotes
class_scores, sort_ind = class_scores.sort(dim=0, descending=True)
class_decoded_locs = class_decoded_locs[sort_ind]  # (n_min_score, 4)
overlap = find_jaccard_overlap(class_decoded_locs, class_decoded_locs)  
suppress = torch.zeros((n_above_min_score), dtype=torch.uint8).to(device)

for box in range(class_decoded_locs.size(0)):
# If this box is already marked for suppression
    if suppress[box] == 1:
    continue
    suppress = torch.max(suppress, overlap[box] > max_overlap)
    suppress[box] = 0

JVGD · Answer

I might be able to help with the theory, but the coding... it is a non standard API such as Tensorflow or Pytorch (it might be custom code for what I can tell).
The key element here is that the bouding boxes are removed only if they hold a prediciton for the same class that the box that is overlapping with (but with less confidence, that is why it gets removed).
Here is an example, where we have:

Two classes $c in [c_1, c_2] = [$ "star"$, $ "moon" $]$
Three bounding boxes

The blue bounding boxes holds prediction for the class $c_1$ so their predictions are $p(c_1)_{box1} = 0.8$ and $ p(c_1)_{box2} = 0.9$. On the other hand, the green box holds a prediction for the class $c_2$.
The three boxes are highly overlaping so the overlap between any box $x$ with any box $y$ will be above the IoU threshold: $IoU(box_x, box_y) > 0.5$. So in principle all boxes are suceptible to be removed.
However the NMS only applies for boxes predicting the same class (in the case the blue one). So the NMS algorihtm is: if the boxes are overlapping, $IoU(box_1, box_2) > 0.5$, which is true, remove all non maximal class probability boxes. Said differently, take just the box with highest $p(c_1)$ and remove the rest. So the $box_1$ with class probability $p(c_1) = 0.8$ would be removed.
So what happens with green box? Isn't it overlapping as well? Yes, but consider that the green box is not trying to predict the same object, is trying to predict another object, $c_2$, which happens to be very close to the first object, $c_1$. This way object detectors support detection of different overlapping objects.

Non Max Suppression and Object Detection

One Answer

Add your own answers!

Ask a Question