TransWikia.com

How to segment rectangular photographs on a table?

Mathematica Asked on March 27, 2021

Given an image of a table with photographs (take from any angle), I’m trying to extract the photographs and correctly align them into perfect rectangles. There are four steps to the algorithm:

  1. Detect quadrangles in the input image

  2. Filter out non-photograph quadrangles

  3. Re-project the quadrangles to a bird’s-eye view

  4. Correct for curved edges in the sub-images

Here’s an example of doing the first two steps:

enter image description here

  • For each of detected photographic quadrangles an orange outline is drawn.
  • I’ve added a green one to emphasize the fidelity of solution I’m looking for, we can’t lose any pixels in this process.
  • Notice how the orange rectangles do not contain any white margins – we just want them to shrink-wrap the photographic content.

Ok, then here’s what the remaining steps look like:

enter image description here

Note that for each of reprojected photograph (shown on the left) we fix the warping/curvature of the photograph by transforming it into a perfectly rectangular image (on the right).

Here’s an example image to try:

Import @ CloudObject[
 "https://www.wolframcloud.com/objects/d3999fcf-7a01-4d60-bb9c-38604d252475"]

And as requested, some additional examples to test with:

CloudGet @ CloudObject["https://www.wolframcloud.com/objects/068cf32f-b753-4c38-bc01-f378bc64d54a"]

enter image description here

2 Answers

Here's an idea, but it only works (partially) with the easiest of the test cases.

img = CloudGet[CloudObject["https://www.wolframcloud.com/objects/068cf32f-b753-4c38-bc01-
f378bc64d54a"]][[5]]

enter image description here

(* Binarize - this step depends a lot on the background in the image *)
img2 = ChanVeseBinarize[img, Automatic, {Automatic, Automatic, 2, 1}]

enter image description here

(* Find hulls, and discard ones that are tiny or touching a border *)
comps = MorphologicalComponents[img2, Method -> "ConvexHull"];
comps2 = SelectComponents[comps, #AdjacentBorderCount == 0 && #Count > 50 &];

(* show perimeters, to see how it's working *)
perims = ComponentMeasurements[comps2, "PerimeterPositions"];
HighlightImage[img, {PointSize[Medium], Red, Point @@@ perims[[All, 2]]}]

enter image description here

(* Extract the single images *)
singleimgs = ComponentMeasurements[ {comps2, img} , "MaskedImage"][[All, 2]]

enter image description here

transform[img_] := 
 Module[{img2, frame, comp, p, c, c2, cornerdemo, t, img3},
  
  (*add some padding,to make the corner detection work better*)
  img2 = ImagePad[img, 2];
  
  (*get the perimeter of the image*)
  frame = AlphaChannel[img2];
  comp = MorphologicalComponents[frame, Method -> "ConvexHull"];
  p = Flatten[
    ComponentMeasurements[comp, "PerimeterPositions"][[1, 2]], 1];
  
  (*find the corners of the image*)
  c = Sort[ImageCorners[frame, 5, 0.01]];
  
  (*the desired new corners*)
  c2 = Sort[Tuples[MinMax /@ Transpose[p]]];
  
  (*just to demonstrate the corner detection*)
  colors = {Red, Orange, Yellow, Blue};
  cornerdemo = 
   Show[frame, 
    Graphics[{Table[{colors[[n]], Disk[c2[[n]], 5]}, {n, 4}]}], 
    Graphics[Table[{colors[[n]], Circle[c[[n]], 5]}, {n, 4}]]];
  
  (*transform such that the image corners are mapped to the desired corners*)
  t = Last[FindGeometricTransform[c2, c]];
  img3 = ImagePerspectiveTransformation[img2, t, DataRange -> Full];
  
  {cornerdemo, img3}]

transform /@ singleimgs

enter image description here

Answered by MelaGo on March 27, 2021

With some of the examples given, we can bring all the images in the correct position at once, before extracting them. Here I chose 4 points using the coordinates tool.

enter image description here

imgs = CloudGet@
   CloudObject[
    "https://www.wolframcloud.com/objects/068cf32f-b753-4c38-bc01-
f378bc64d54a"];

fourPoints = {{43.8398, 70.2148}, {336.398, 68.4375}, {342.199, 219.031}, {56.6094,209.16}}

This function processes the image. It takes the image and the four points.

processImage[img_, pts_List] := (
  rect = {{pts[[1]][[1]], pts[[1]][[2]]}, {pts[[3]][[1]], 
     pts[[1]][[2]]}, {pts[[3]][[1]], pts[[3]][[2]]}, {pts[[1]][[1]], 
     pts[[3]][[2]]}};
  transform = FindGeometricTransform[rect, pts][[2]];
  Show[fullImage = 
    Sharpen[ImagePerspectiveTransformation[img, transform, 
      PlotRange -> Full], 1], 
   Graphics[{PointSize[Large], Green, Point[rect], Transparent, 
     EdgeForm[Green], Polygon[rect]}], ImageSize -> Medium]
  )

processImage[imgs[[1]],fourPoints];

enter image description here

Now we isolate the pictures.

perim = MorphologicalPerimeter[fullImage, 60];
i2 = DeleteSmallComponents[perim];
morphBox = MorphologicalTransform[i2, {"BoundingBoxes", "Clean"}, Infinity]; 
ima=ImageMultiply[fullImage,morphBox];

enter image description here

Extract as a list of images and discard those not meeting a certain dimensions.

singles = ComponentMeasurements[ima, "MaskedImage"][[All, 2]];
validSingles = 
 Select[singles, 
  0.4 < Min[ImageDimensions[#]]/Max[ImageDimensions[#]] &]

Go through the list of images and attempt to trim the borders. It is difficult to get something that will work for all images, particularly when the border is the same color as the background of the photo. This works for some images, but not all of them.

imageList = {};
n = 1;
While[n <= Length[validSingles],
 v = validSingles[[n]];
 dim = ImageDimensions[v];
 bd = BorderDimensions[ImageAdjust[v, {-0.1, 0.1}], 0.04];
 crop1 = ImageCrop[v, 
   dim - {bd[[1]][[1]], bd[[2]][[1]]}, {Left, Bottom}];
 dim2 = ImageDimensions[crop1];
 crop2 = ImageCrop[crop1, 
   dim2 - {bd[[1]][[2]], bd[[2]][[2]]}, {Right, Top}];
 AppendTo[imageList, crop2];
 n++;
 ]
Row[imageList, "  "]

enter image description here

Better with a black background.

enter image description here

Answered by Jean-Pierre on March 27, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP