Medical imaging annotation

The rankings are my assessment specifically for medical diagnostic-AI labeling, rather than generic computer-vision annotation.

1. Best overall for a production medical-AI team: Encord

Encord is particularly strong if you're building a sizable radiology dataset and want annotation, review, data curation, and model-in-the-loop workflows in one system.

Its medical tooling supports DICOM/NIfTI, CT, MRI, X-ray, PET, ultrasound and mammography, with multiplanar views, 3D annotation, study-level classification, customizable hanging protocols, and AI-assisted segmentation. It also supports multi-stage workflows such as annotation → review → consensus and lets you bring your own models for pre-labeling.encord.com

A particularly useful workflow is:

AI pre-label → radiologist correction → second-reader review → consensus → export → train → identify uncertain cases → relabel

That is much more scalable than asking radiologists to manually outline every lesion from scratch.

Encord medical imaging annotation

Best when: you're a startup, hospital AI group, CRO, or enterprise team moving toward production.

Caveat: it's a considerably more sophisticated platform than you need for a small academic segmentation study.

2. Best open-source stack: OHIF + MONAI Label

This is probably my favorite architecture if you have strong engineering resources and don't want your workflow locked into a commercial annotation vendor.

OHIF provides a web-based DICOM viewer with MPR, multimodal fusion, measurement/annotation tools and 3D segmentation. It connects to imaging archives through standards such as DICOMweb and is explicitly designed to be extensible with custom workflows.ohif.org

MONAI Label adds the AI layer: interactive and automated segmentation, including DeepGrow/DeepEdit-style workflows and models that can continuously improve from annotator interaction.docs.monai.io

The resulting architecture can look like:

PACS/DICOMweb → OHIF → MONAI Label → radiologist correction → DICOM SEG/NIfTI → MONAI/PyTorch training

This is especially attractive for academic medical centers because you can customize the viewer and models rather than adapting your research to a vendor's workflow.

MONAI Label documentation
OHIF platform and features

3. Best flexible commercial alternative: Supervisely

Supervisely has become quite interesting for medical imaging.

Its DICOM workflow supports CT, MRI and PET volumes, viewing axial/sagittal/coronal planes, 3D masks, AI-assisted segmentation and interpolation between manually labeled slices. It also supports DICOM, NIfTI and NRRD.docs.supervisely.com

One useful feature is that an annotator can label key slices rather than every slice, then use interpolation to reconstruct the segmentation through the volume and inspect/edit it in 3D.docs.supervisely.com+2

That can dramatically reduce the amount of tedious slice-by-slice work.

Supervisely medical/DICOM annotation

4. Best research segmentation tool: ITK-SNAP

ITK-SNAP remains excellent when your primary task is accurate 3D anatomical segmentation, rather than managing thousands of annotators.

It supports DICOM and NIfTI, simultaneous orthogonal-plane viewing, manual segmentation, registration, interpolation between slices, and semi-automatic segmentation.itksnap.org+1

It's particularly good for:

• brain MRI
• tumor segmentation
• organ segmentation
• vascular structures
• small research datasets
• creating a high-quality gold-standard subset

ITK-SNAP

The downside is that it isn't really a full annotation-operations platform: task assignment, large-team QA, consensus management, dataset analytics and active-learning loops aren't its core strengths.

5. Best for highly customized research: 3D Slicer

3D Slicer is the other heavyweight I'd put on every research team's shortlist.

It's particularly powerful when annotation is only one part of the workflow and you also need registration, image processing, visualization, quantitative analysis or custom algorithms.

I'd choose Slicer over a SaaS annotation platform when a research group says:

“We need to do something unusual with these volumes and we'll probably write our own extension.”

For straightforward production annotation, however, I'd generally favor Encord/Supervisely or OHIF + MONAI Label.

6. CVAT: excellent general annotation infrastructure, but not my first choice for 3D radiology

CVAT is an excellent open-source annotation system with collaboration, review, APIs/SDKs, automation and AI-assisted annotation. Its current documentation includes segmentation masks, automated annotation and model integrations.docs.cvat.ai

It becomes particularly attractive if you're labeling 2D X-rays rather than volumetric CT/MRI.

For example:

Chest X-ray → bounding boxes / polygons / classification → AI pre-label → radiologist verification

is a very natural CVAT workflow.

For complex volumetric radiology, though, I would choose MONAI/OHIF, Supervisely, Encord, Slicer or ITK-SNAP before CVAT.

CVAT documentation

The workflow I'd recommend

Regardless of platform, I'd structure the project around four annotation levels.

Level 1 — Case-level labels

Start with labels such as:

• normal / abnormal
• diagnosis
• laterality
• body region
• acquisition type
• disease severity
• presence/absence of finding

For X-rays, this is often more valuable initially than pixel-perfect segmentation.

Level 2 — Finding localization

For abnormalities:

• bounding box
• point
• polygon
• coarse ROI

This gives you detection/localization training data much faster than full segmentation.

Level 3 — Pixel/voxel segmentation

Use this selectively for findings where shape matters:

• tumors
• organs
• lesions
• infarcts
• hemorrhage
• nodules
• vessels

For CT/MRI, preserve the actual 3D voxel geometry rather than flattening everything into PNG/JPEG slices.

Level 4 — Expert adjudication

Have a second expert review difficult or high-value cases.

A good pipeline is:

Annotator A → automated/model-assisted pre-label → correction → Annotator B → disagreement detection → senior radiologist adjudication → gold-standard dataset

Consensus and expert review are particularly important because medical findings frequently have ambiguous boundaries or legitimately different interpretations. Encord, for example, explicitly supports multi-stage review and consensus workflows.encord.com+1

Don't overlook the data model

This is one of the biggest differences between ordinary image annotation and medical imaging annotation.

For CT/MRI, don't treat the dataset as:

patient123_slice001.png

Treat it as:

Patient → Study → Series → Volume → Slice/Frame → Annotation

Preserve:

• StudyInstanceUID
• SeriesInstanceUID
• SOPInstanceUID
• modality
• acquisition parameters
• spatial orientation
• voxel spacing
• slice thickness
• window/level
• laterality
• relevant sequence information
• annotation provenance

DICOM contains substantial metadata, while NIfTI is commonly used for research-oriented volumetric imaging. Tools need to preserve the spatial relationship between image and mask; even seemingly small coordinate-system mistakes can make a segmentation appear correct in a viewer while being wrong when consumed downstream.docs.supervisely.com

AI-assisted labeling is now essential

For a serious dataset, I would not build a workflow around fully manual annotation.

Instead:

1. Manually annotate a representative seed set.
2. Train or adapt a segmentation/detection model.
3. Use the model to generate preliminary labels.
4. Have experts correct them.
5. Feed corrected labels back into training.
6. Prioritize uncertain/disagreement cases.
7. Repeat.

This is essentially human-in-the-loop active learning.

MONAI Label is explicitly designed around this concept, while Supervisely and Encord provide model-assisted workflows for medical data.docs.monai.io

The key metric isn't:

“How many images can my annotators label per hour?”

It's:

“How many clinically correct labels do we produce per expert hour?”

My practical recommendations

If you're building a radiology AI startup:
→ Encord first; evaluate Supervisely as the main alternative.

If you're an academic medical center with engineering expertise:
→ OHIF + MONAI Label, with 3D Slicer/ITK-SNAP for difficult research segmentation.

If you're primarily labeling chest X-rays:
→ Encord, CVAT, or Supervisely. You probably don't need a sophisticated 3D platform.

If you're doing CT/MRI tumor or organ segmentation:
→ Encord, MONAI Label, Supervisely, ITK-SNAP, or 3D Slicer.

If you need 100% control over PHI and infrastructure:
→ strongly consider self-hosted OHIF + MONAI Label, or an on-prem deployment of an enterprise platform.

If you need a polished enterprise annotation operation:
→ Encord is probably the strongest starting point.

One important current note: I would not start a new medical-labeling program around Amazon SageMaker Ground Truth. AWS currently says Ground Truth is closed to new customers and that it does not plan to add new features, although existing customers can continue using it.docs.aws.amazon.com

Bottom line

If I were designing a new diagnostic-AI labeling stack today, my default choices would be:

Commercial:
DICOM/PACS → Encord → AI pre-label → radiologist → expert QA → consensus → training

Open source:
DICOMweb/PACS → OHIF → MONAI Label → radiologist → MONAI/PyTorch → active learning

And I'd keep ITK-SNAP/3D Slicer available as specialist tools for difficult 3D cases rather than forcing every annotation task through one platform.docs.monai.io

If you're choosing tools for a specific project (e.g., 100k chest X-rays vs. 5k 3D brain MRIs), the optimal stack changes quite a bit.

Best-Suited Annotation Workflows

To ensure diagnostic precision and prepare models for regulatory approval (e.g., FDA 510k), a successful workflow relies on an Expert-in-the-Loop structure:[doctorsintheloop.com](https://imerit.ai/resources/blog/what-to-look-for-in-a-medical-image-annotation-platform-in-2025-who-offers-what/)

1. Data Ingestion & De-identification
   • Securely ingest raw DICOM files, automatically stripping Protected Health Information (PHI) from metadata headers to maintain compliance.
2. AI-Assisted Pre-Labeling (Active Learning)
   • Pass incoming scans through a base model (via tools like MONAI Label) to generate preliminary bounding boxes or volumetric segmentations.
3. Primary Clinician Annotation
   • Junior radiologists, specialized technicians, or trained annotators refine the AI-generated masks using multiplanar 3D views (simultaneous axial, sagittal, and coronal adjustments).[doctorsintheloop.com](https://imerit.ai/resources/blog/what-to-look-for-in-a-medical-image-annotation-platform-in-2025-who-offers-what/)
4. Consensus & Senior Review
   • Board-certified specialists or lead physicians review ambiguous edge cases, perform consensus reads, and override inaccurate labels to establish the gold-standard "ground truth".[doctorsintheloop.com](https://imerit.ai/resources/blog/what-to-look-for-in-a-medical-image-annotation-platform-in-2025-who-offers-what/)
5. Quality Assurance & Export
   • Run automated validation metrics (e.g., inter-observer variability checks) before exporting structured annotations into training pipelines.

To help tailor this recommendation, are you working primarily with radiology data (CT/MRI/X-ray), digital pathology (WSI) , or ultrasound ? Also, let me know if your team needs an open-source developer framework or a fully managed SaaS platform with compliance support.

Best for: MRI segmentation, 3D masks, research prototypes, and custom AI pipelines.

Links: MONAI Label ·3D Slicer

2. MD.ai Annotator

(Best radiology-focused workflow)

A DICOM-native annotation platform designed for radiology projects, with image review, annotation, reporting, AI-assisted tools, and PHI detection and de-identification features.www.md.ai+1

Best for: Chest X-ray findings, radiologist-led labeling, structured reports, and clinical imaging research.

Link:MD.ai Annotator

3. OHIF Viewer + MONAI Label

(Best browser-based option)

A flexible, open-source web imaging viewer that can be connected to MONAI Label for AI-assisted annotation. Useful for distributed teams and DICOMweb-based imaging infrastructure.project-monai.github.io+1

Best for: Multi-site research, browser-based review, and teams that want to integrate annotation into existing imaging systems.

Link:OHIF Viewer

4. Encord

A commercial data-annotation platform worth evaluating for larger teams, complex annotation projects, and workflows that combine medical images with other data. Its DICOM-specific guidance emphasizes format compatibility, 3D review, and AI-assisted annotation.encord.com

Best for: Organizations prioritizing managed workflows, collaboration, and scaling annotation operations.

Link:Encord

5. ITK-SNAP

A useful research tool for manually delineating and inspecting 3D anatomical structures in medical images. It is especially worth considering when precise segmentation matters more than enterprise project management.

Best for: Brain MRI, lesion delineation, manual segmentation, and reference-mask creation.

Link:ITK-SNAP

These are candidates to evaluate, not a claim that one platform has been proven universally superior. Test each against your actual modalities, annotation tasks, security requirements, and export format.

2. Match the platform to the annotation task

The key distinction is that classification, detection, and segmentation require different annotation designs. A bounding box is usually insufficient as ground truth for a model expected to delineate a tumor's exact boundaries.

3. Recommended end-to-end annotation process

The platform is only one part of the solution. I recommend this six-stage workflow, particularly if the goal is diagnostic AI that could eventually be evaluated for clinical use.

1. Define the clinical labeling protocol

   Specify the target condition, inclusion and exclusion criteria, annotation boundaries, uncertainty labels, and rules for ambiguous cases. For example, define whether a chest X-ray finding is “present,” “absent,” or “indeterminate.”

2. Curate and de-identify the images

   Validate DICOM metadata, remove or protect identifying information, check image quality, detect duplicates, and link images to the correct patient and study.

3. Create expert reference labels

   Have qualified radiologists annotate a representative sample. Use image-level labels for classification, boxes or points for detection, and pixel- or voxel-level masks for segmentation.

4. Scale with AI-assisted annotation

   Use pretrained models to propose labels or masks. Annotators correct the proposals rather than accepting them automatically. Prioritize difficult cases for expert review.

5. Perform independent quality assurance

   Double-annotate a subset, adjudicate disagreements, track inter-reader agreement, and audit difficult positives, negatives, and uncertain cases.

6. Export, validate, and freeze the dataset

   Check label consistency, spatial alignment, metadata, and export compatibility. Split data by patient—not individual image or MRI slice—and keep test data isolated from training.

These practices align with published medical imaging AI guidance on representative data, independent test sets, reference-standard quality, and human oversight.www.fda.gov+2

4. What to measure before choosing a platform

A critical caution: a large dataset with inconsistent labels can be less useful than a smaller, carefully curated dataset. Medical imaging research has documented how inconsistent labeling schemas and reader disagreement can undermine the reliability of supposedly “ground-truth” labels.www.fda.gov+3

My recommendation

• Academic or hospital research team: Start with MONAI Label + 3D Slicer. It provides a flexible, open-source route to expert-reviewed 3D segmentation and model-assisted labeling.
• Radiology team focused on X-rays and clinical interpretation: Pilot MD.ai Annotator and evaluate its fit with your DICOM infrastructure and review process.
• Large organization with substantial annotation operations: Evaluate Encord alongside a radiology-specific option, using the same representative test dataset and acceptance criteria.
• MRI tumor or organ segmentation project: Start with MONAI Label + 3D Slicer or ITK-SNAP, depending on how much AI assistance and workflow automation you need.

Before committing, run a pilot on a few hundred representative studies, including difficult cases, and compare annotation time, expert agreement, correction rates, export quality, and total cost.

One question to narrow this down

What is the primary diagnostic AI task you're preparing the dataset for?

Imaging modality

X-rays (2D)

MRI (3D)

Both X-rays and MRI

Primary AI task

Disease detection or classification

Tumor, lesion, or organ segmentation

Measurements or structured reporting

Project setting

Research team or hospital

Startup or commercial AI development

Large enterprise or multi-site clinical study

[Personalize recommendations]