How to calculate Segmentation Accuracy for SynthSeg Algorithm

I am pleased and honored to contact you

Hi, I really appreciate your example for segment brain MRI using a deep neural network

I wonder how to verify the accuracy of the segmenation and extraction of 3D voxels of the globi pallidi of my dataset (brain MR images NOT isotropic – matrix size 144x256x256 from pediatric patients (age range 5 years - 18 years)) carried out with SynthSeg

The segmentation is carried out but I cannot determine the measure of segmentation accuracy because of the different format of the ground truth labels compared to my set of images.

In other words, which metrics can I use to evaluate the accuracy of the SynthSeg algorithm on my image set?

I thank you very much in advance for your time to reply to this questions.

[Donatas]

There are various metrics to evaluate the segmentation accuracy: Dice score, volumetric change and etc. But first, are the 'ground truth' labels in the same image space as your MR images that you used for SynthSeg?

[Marco Guerrieri]

Yes, the input RM volume and the Ground Truth (label) volume have the same size, i.e. 256 x 256 x 256. In particular, - we used FreeSurfer to get the "true segmentation": file aseg.mgz (transformed in .nii) - resolution: [1 1 1], image size: [256 256 256] --> Ground Truth volume - and we used an anatomical file that was also elaborated by FreeSurfer as the input to the synthseg: file T1w.mgz (transformed in .nii) - resolution: [1 1 1], image size: [256 256 256] --> Input volume Thus, We applied SynthSeg and we obtained a value of Average Dice Loss close to 0.85. '''

Answer 1

[Neha]

Hi Marco,

I understand that you want to calculate the segmentation accuracy for SynthSeg algorithm. With the Average Dice Loss value close to 0.85, it indicates a reasonably good segmentation performance by the algorithm. Here are a few other metrics you can also consider:

1. The Dice Similarity Coefficient (DSC) is a commonly used metric for evaluating segmentation accuracy. A DSC value of 1 indicates a perfect overlap between the predicted segmentation and the ground truth, while a value of 0 indicates no overlap at all.
2. Jaccard Index (also known as Intersection over Union, IoU): Like DSC, IoU measures the overlap between the predicted and ground truth regions. It is defined as the intersection divided by the union of the predicted and ground truth regions.
3. BFScore: The BF score measures how close the predicted boundary of an object matches the ground truth boundary. It combines precision and recall to provide a balanced evaluation of the boundary delineation.

You can refer to the following documentation links for more information on these metrics:

• https://www.mathworks.com/help/images/ref/dice.html
• https://www.mathworks.com/help/images/ref/jaccard.html
• https://www.mathworks.com/help/images/ref/bfscore.html

Hope this helps!