Centaur Labs launches auto-segmentation powered by SAM 

Tom Gellatly, VP of Engineering
January 11, 2024

Learn how Meta’s Segment Anything Model (SAM) powers Centaur Labs’ auto-annotation tools to deliver more accurate labels faster.

The advancement of medical AI hinges on accurate ML models trained on trustworthy ground-truth data.  Centaur Labs’ unique crowd-labeling model follows a three-step process; we develop a set of Gold Standard ground truth labels in collaboration with customers, our expert crowd generates multiple opinions on each image in as little as 24 hours, and our algorithm scores and aggregates the top labelers based on their performance against Gold Standards. Our new auto-annotation tool accelerates the creation of these Gold Standards and minimizes human error by leveraging Meta’s open-source Segment Anything Model (SAM), enabling customers to launch projects faster than ever. Read more to learn how our SAM integration works and how auto-annotation can improve your ML model. 

Introducing Centaur Labs’ auto-segmentation feature powered by SAM 

Polygon segmentation is traditionally a tedious task of drawing dozens of single points around objects of interest within an image. SAM minimizes that work to a two-click action to instantly draw an accurate boundary around your objects. 

SAM is an open-source foundation model for auto-segmenting images with best-in-class generalization capabilities and state-of-the-art technology in image segmentation. Its ability to adapt to a wide variety of visual contexts makes it the tool for medical imaging, where diversity in data types is common. 

We developed our auto-segmentation capabilities by integrating with SAM based on its proven effectiveness across a variety of medical images from fundus images to chest X-rays. Centaur Labs’ auto-segmentation feature is now available to all customers using web labeling. 

How to use auto-segmentation for data labeling

Our auto-annotation tools are now live in our desktop labeling platform and available to Centaur Labs customers. New customers can request a demo with our team to see this tool in action. Existing customers can follow this step-by-step guide to begin auto-segmenting polygons today. 

  1. Navigate to a Polygon Segmentation task in an Image Project
  2. Under Add Labels, click “Begin Web Labeling”
  3. Select the “Auto Segmentation” lightning bolt icon to activate auto-segmentation
  4. Identify an object in the image that you would like to segment
  5. Draw two points on the image, “bounding” the object
  6. Instantly, our AI service will generate a polygon segmentation for you
  7. Repeat as many times as you like on the same image!
  8. Refine points on your image as needed
  9. Right-click and select Delete to get rid of a segmentation

Benefits of auto-segmentation for data labeling 

Traditional polygon segmentation is a tiring, painstaking process, which can increase the risk of human error creating a negative ripple effect in your immediate results and long-term ML algorithm. Our new tool combines the skills of the expert human in the loop and AI-assisted segmentation to create Gold Standard ground truth labels faster with a focus on segmentation refinement rather than generation. 

Medical data labeling is a unique challenge and the stakes are very high. We understand the importance of keeping expert humans in the loop to produce accurate labels that will shape the future of medical AI and improve patient outcomes. Our tool improves the workflow and throughput of labelers while minimizing the risk of human error in a critical phase of the task setup process. 

Customer benefits of auto-annotation 

Labeler benefits of auto-annotation 

Sample data types improved by auto-segmentation 

We know firsthand from working with a wide range of medical AI customers that there are vast amounts of data types and formats that can be leveraged to advance the field of medical AI. See how pathology and video frame segmentation can benefit from auto-segmentation.


Pathology images can be massive, and locating and segmenting cell structures can take even the best pathologists hours per slide. Auto-segmentation enables labelers to complete tasks in a fraction of the time, especially on pathology slides containing dozens or hundreds of objects of interest.

Video frame segmentation for object tracking 

In video frame segmentation tasks, labelers view a series of still images taken from a video clip to identify anatomical structures or foreign bodies, E.g. surgical tools. Identifying and segmenting the same object across many frames can be tedious and time-consuming. Auto-segmentation enables labelers to repeatedly segment structures across time series data much more quickly and accurately.

The use cases listed here are just a preview of how our auto-annotation tools can accelerate your labeling projects. If you don’t see your use case listed, connect with our sales team and engineers to learn how your unique use case can benefit from auto-segmentation. Book a call to learn more. 

Related posts

September 3, 2021

Announcing our $15M Series A led by Matrix Partners to annotate the world's medical data

We are so humbled and excited to share our recent $15M Series A funding round led by Matrix Partners!

Continue reading →
July 8, 2021

Centaur Labs teams up with Brigham and Women's Hospital on Massachusetts Life Sciences funded project

Learn more about how Centaur Labs is working with the Brigham and Women's Hospital team to develop multiple AI applications for point of care ultrasound.

Continue reading →
February 18, 2021

AIMed interviews Centaur Labs CEO Erik Duhaime

Founder and CEO of Centaur Labs talks to AI Med magazine about the power of collective intelligence.

Continue reading →