How to Annotate Images for Object Detection Models (YOLO, Faster R-CNN)

Step 1. Design the class taxonomy before anyone draws a box

Classes must be separable from pixels inside the box. That is the test.

If an annotator needs surrounding context to decide between “delivery van” and “service van,” a detector cropping to the region proposal cannot decide either. Merge the classes and recover the distinction downstream with an attribute model.

Practical rules we apply on scoping calls:

• Any class expected to hold under roughly 1 to 2 percent of total instances will be starved. Merge it, or plan targeted collection.
• Ultralytics recommends at least 1,500 images and 10,000 instances per class for a YOLO dataset. Treat that as an order of magnitude, not a law.
• Avoid encoding attributes as classes. Six vehicle types across four colors is 24 classes, each with a quarter of the data it needs.

Step 2. Write object detection labeling guidelines, then calibrate on them

An annotation guideline that does not answer these questions is not finished:

Modal versus amodal is the one that silently ruins datasets. COCO’s convention labels the visible extent. If your annotators guess at hidden extents on some images and not others, the model receives two different definitions of the same class and the localization loss never converges cleanly.

Calibration matters more than the document. Have every annotator label the same 50 images, measure pairwise IoU, and discuss every disagreement before production starts. HabileData runs this as a formal stage because guideline drift is cheaper to catch in 50 images than in 50,000.

Step 3. Lock the annotation format contract before labeling starts

Decide the target format before annotation, not at delivery. Conversion is lossy in ways that are not obvious. Converting COCO polygons to YOLO detection boxes discards the mask, which is fine for a detector and fatal if anyone downstream wanted segmentation.

Step 4. Pre-label with a model, then correct the output

Model-assisted labeling is standard now. A detector trained on the first few thousand images, or a foundation model like SAM, produces candidate boxes that annotators adjust rather than draw.

This cuts per-image time by 40 to 60 percent on structured datasets in our production work. It also introduces a failure mode that took us a while to name: annotators accept plausible-looking boxes without checking them. A box that is 6 pixels loose looks correct at normal zoom. Nobody edits it.

The counter is a QA sample drawn specifically from accepted-without-edit pre-labels, not from the general population. Those are the boxes nobody looked at twice.

Step 5. Set bounding box tightness in scale-relative terms

Most annotation specs state box tolerance in pixels. The arithmetic says that is the wrong unit.

Take a 2 pixel halo around a box, uniform on all sides. On a 200 x 200 pixel object, the resulting IoU against ground truth is 40000 / 41616, about 0.96. Negligible.

On a 20 x 20 pixel object, the same 2 pixel halo gives 400 / 576, about 0.69. That instance now fails a 0.7 IoU evaluation threshold entirely.

A flat “within 2 pixels” tolerance is therefore meaningless. It is trivially easy on large objects and impossible on small ones. Specify tolerance as a fraction of object size, and tighten review sampling on the small-object bucket, where COCO defines small as area under 32 x 32 pixels.

Step 6. Gate every batch on IoU and inter-annotator agreement

Quality has to be a number attached to a batch, not an assurance in a proposal.

Our standard thresholds are 95 percent or higher IoU for bounding boxes, 93 percent for polygon, and 92 percent for semantic segmentation, measured per class on every batch. Batches below threshold get re-annotated at no charge.

Report errors by type, not as a single score. The TIDE framework (Bolya et al., ECCV 2020) splits detection error into six types: classification, localization, both together, duplicates, background false positives, and missed ground truth.

That taxonomy works just as well for label QA as for model evaluation. A batch failing on localization needs tooling and tolerance changes. A batch failing on classification needs a guideline rewrite. A single aggregate score tells you neither.

Step 7. Split the dataset by source, not by image

Random image splits leak. Video-derived datasets are the worst case: consecutive frames are near-duplicates, so a random split puts almost-identical images in train and validation.

The result is validation mAP that looks excellent and production performance that does not. Split by scene, session, camera, or capture day.

Step 8. Render the labels before you train

Load 200 random training samples with boxes drawn and look at them. YOLO’s train_batch*.jpg output exists for exactly this. Most format bugs are visible in ten seconds and invisible in a metrics dashboard.

YOLO annotation format: structure, rules and common mistakes

Each line is one object:

All four coordinates are normalized by image width and height, so every value falls between 0 and 1. Class IDs start at 0.

Directory structure is not optional. Ultralytics derives the label path by substituting /images/ with /labels/ in the image path. Put label files in the image folder and training runs with zero labels found, often without an obvious error.

    
dataset/
├── images/train/img001.jpg
├── labels/train/img001.txt
└── data.yaml

Two YOLO-specific points worth knowing:

• An image with no .txt file is a valid background image. Ultralytics recommends roughly 0 to 10 percent background images to reduce false positives, noting COCO carries about 1 percent.
• There is no background class. Index 0 is a real object class, which is the opposite of the Faster R-CNN convention below.

Faster R-CNN dataset preparation in torchvision

Torchvision’s Faster R-CNN expects per-image target dictionaries:

Three constraints that cause real training failures:

1. Coordinates must satisfy 0 <= x1 < x2 <= W and 0 <= y1 < y2 <= H. Degenerate boxes where x1 == x2 throw an assertion during training. These usually come from rounding tiny boxes during a format conversion.
2. Your classes must start at 1. Set num_classes to your class count plus one for background.
3. COCO category IDs are not contiguous. COCO's 80 classes occupy IDs from 1 to 90. Feeding raw category IDs as labels silently creates dead output neurons.

The COCO evaluation fields area and iscrowd are also used at eval time to bucket results by object size and to ignore crowd regions. Annotation vendors that leave these fields as placeholder zeros produce datasets that train but evaluate incorrectly.

Converting between annotation formats without losing precision

YOLO to absolute corner coordinates:

 
x1 = (x_center - width/2)  * image_width
y1 = (y_center - height/2) * image_height
x2 = (x_center + width/2)  * image_width
y2 = (y_center + height/2) * image_height
 

COCO to Pascal VOC is x_max = x + w, y_max = y + h. Watch this one: the original VOC development kit treated pixel coordinates as 1-based, so naive converters introduce a one pixel shift throughout the dataset. Harmless on large objects, measurable on small ones.