How to make segmentation mask using given coordinates in xml file in python?

Question

I have xml files that contain coordinates for creating masks. I am using this code to pass the coordinates and to extract the mask from it:

def extract_masks(self, filename):
   # load and parse the file
    tree = ET.parse(filename)
    # get the root of the document
    root = tree.getroot()
    # extract each bounding box
    # get details of image
    info = self.image_info[image_id]
    # define box file location
    path = info['annotation']
    masks = np.zeros((224, 224), dtype=np.uint8)
    classes = []
    points = []
    for member in root.findall('object'):
      points = member[4].text
      print(points)
      point = np.array(points)
      cv2.fillPoly(masks, point, 255)
      #cv2.drawContours(masks, [pts],-1, (0,255,0), -1)
      classes.append(self.class_names.index(member[0].text))
    return masks, classes  

But I didn't get the required output instead I got the following error:

TypeError                                 Traceback (most recent call last)
<ipython-input-9-2d5205c6367c> in <module>()
      3 for image_id in image_ids:
      4     image = dataset_train.load_image(image_id)
----> 5     mask, class_ids = dataset_train.load_mask(image_id)
      6     visualize.display_top_masks(image, mask, class_ids, dataset_train.class_names)

1 frames
<ipython-input-7-110d0d3d701f> in extract_masks(self, filename)
     51           print(points)
     52           point = np.array(points)
---> 53           cv2.fillPoly(masks, [point], 255)
     54           #cv2.drawContours(masks, [pts],-1, (0,255,0), -1)
     55           classes.append(self.class_names.index(member[0].text))

TypeError: pts data type = 19 is not supported

Here is my XML file:

<annotation>
<folder>G:\Mask RCNN\train_xml</folder>
<filename>R_Image [1].jpeg</filename>
<path>G:\YOLO\Train\imagesR_Image [1].jpeg</path>
<source>
<database>Unknown</database>
</source>
<size>
<width>224</width>
<height>224</height>
<depth>3</depth>
</size>
<segmented>0</segmented>
<object>
<name>Meningioma</name>
<pose>Unspecified</pose>
<truncated>0</truncated>
<difficult>0</difficult>
<points>[[97,41], [96,42], [92,42], [91,43], [89,43], [88,44], [87,44], [86,45], [85,45], [81,49], [80,49], [74,55], [74,57], [73,58], [73,66], [74,67], [74,69], [75,70], [75,71], [76,72], [76,73], [81,78], [82,78], [86,82], [87,82], [91,86], [92,86], [96,90], [97,90], [99,92], [100,92], [101,93], [102,93], [103,94], [104,93], [105,94], [107,94], [108,93], [111,93], [112,92], [113,92], [114,91], [115,91], [119,87], [120,87], [120,86], [121,85], [121,84], [122,83], [122,82], [123,81], [123,80], [122,79], [122,72], [121,71], [121,70], [120,69], [120,68], [119,67], [119,66], [118,65], [118,64], [116,62], [116,61], [115,60], [115,58], [113,56], [113,55], [112,54], [112,45], [109,42], [107,42], [106,41]]</points>
<bndbox>
<xmin>117</xmin>
<ymin>77</ymin>
<xmax>166</xmax>
<ymax>111</ymax>
</bndbox>
</object>
</annotation>

Any assistance on this matter will be greatly appreciated, Thanks.

Answer 1

As stated in my comment, I think you can parse the points coming from the xml file in a string form with the below function. The below should be more permissive to having spaces or brackets in a slighlty different arrangement.

polygon = "[[3603,100], [3676,110], [3676,137], [3603,147]  ]"
def points_string_to_array(points_string):
    # remove space and square brackets
    points_string = points_string.replace('[', '').replace(']', '').replace(' ', '')
    # create a 1D array with only floats
    polygon_array = np.array([float(x) for x in points_string.split(",")])
    # reshape the array to have a 2D array 
    polygon_array = polygon_array.reshape((len(polygon_array)//2, 2))
    return polygon_array

points_string_to_array(polygon)

For the below result:

array([[3603.,  100.],
       [3676.,  110.],
       [3676.,  137.],
       [3603.,  147.]])