What is Image Segmentation? What's the difference between instance segmentation and semantic segmentation?

Easy Last updated on July 27, 2022, 12:40 a.m.

Markdown Monster icon

When we look at the above picture, Our brain will likely comprehend the image as “There is a boat on a beautiful beach with palm trees and wooden houses lined up in the backdrop!” It means the image is interpreted as a combination of different objects - ‘a boat’, ‘trees,’ ‘houses,’ and ‘beach.’ Voila! This is precisely what image segmentation does. In the real world, image segmentation is used for multiple applications such as self-driving cars, video surveillance, medical imaging, visual image search, traffic analysis, virtual try-on, virtual make-up, and so on.

Image segmentation is the task of clustering similar objects and classifying them with some labels. An image consists of various pixels, and these pixels grouped together define different elements in the image. Image segmentation performs per-pixel classification to create clusters by assigning labels to individual pixels. Image segmentation can be of different types - Instance Segmentation and Semantic Segmentation.

Markdown Monster icon
Semantic Segmentation vs. Instance Segmentation. [Source: https://www.v7labs.com/blog/image-segmentation-guide]

Instance Segmentation vs. Semantic Segmentation

Instance segmentation refers to classifying different instances or occurrences of the same object into different categories. For example, in the above picture, there are two cats. Instance segmentation will label these cats distinctively. On the contrary, when all the cats are labeled as just ‘cats,’ it is called semantic segmentation, i.e., labeling all instances of the same object as the same category.