Auto-label images with SAM 2
Auto-labeling turns a point or box prompt into a segmentation mask. It is most useful as a fast first pass: prompt the object, inspect the preview, refine its edges, then assign a class and save. The three short recordings below use the current AnyLearning interface and a real construction-safety photograph.
Step 1: Turn on AI and choose a model
Open an Object Detection or Image Segmentation project, start labelling, then select the brain-shaped AI tool in the left toolbar. Choose a model in the new toolbar at the top of the canvas.
- SAM 2 Hiera-Small is the recommended default for the cleanest masks.
- SAM 2 Hiera-Tiny is a faster SAM 2 option on slower CPUs.
- MobileSAM is the fastest and smallest choice when responsiveness matters more than boundary detail.
- SAM 2 Hiera-Base+ and Hiera-Large are the slowest, largest options.
MobileSAM, Hiera-Tiny and Hiera-Small ship with AnyLearning and work offline. Base+ and Large are downloaded the first time you select them.
Step 2: Prompt and refine the object
Start with Include box when the object has a clear extent. Drag a box around the object, then use Include point on a missing region or Exclude point on unwanted background. Every new prompt updates the preview; in this example, a single exclusion point removes the worker's temple while keeping the helmet rim.
The prompt controls also have keyboard shortcuts:
- Include point:
A - Exclude point:
D - Include box:
R - Clear prompts:
C - Finish object:
F
Choose Polygon to keep the generated outline, or Rectangle when your project only needs a bounding box. Clear removes the current prompts and preview without deleting labels you already finished.
Step 3: Assign a class and save
When the preview follows the object, select Finish object, choose its class, and save the annotations. The generated mask then uses the class color and name like a manually drawn shape. Inspect the boundary before moving to the next image; auto-labeling accelerates annotation, but it does not replace review.
First-run behavior
The first prediction on an image is slower because the model computes an image embedding. Later prompts on that image reuse it and normally respond much faster. Wait for the status line to say the model is ready before drawing your first prompt.
The demonstration image comes from the Safety Helmet and Reflective Jacket dataset and is used under the Apache-2.0 license. The recording shows SAM 2 Hiera-Small's real output in AnyLearning; the outline is not hand-drawn or composited for the guide.