what is the meaning of gold standard in DRIVE data base?

hi would you please tell me about gold standard in DRIVE database(retinal images database)? should I compare my results ( I mean segmentation blood vessel results) to gold standard or human observer? Thanks

Answer 1

[Thorsten]

The data base states

The set of 40 images has been divided into a training and a test set, both containing 20
images. For the training images, a single manual segmentation of the vasculature is
available. For the test cases, two manual segmentations are available; one is used as gold
standard, the other one can be used to compare computer generated segmentations with those
of an independent human observer. All human observers that manually segmented the
vasculature were instructed and trained by an experienced ophthalmologist. They were asked
to mark all pixels for which they were for at least 70% certain that they were vessel.

So there is actually no difference between the "gold standard" segmentations and the "independent human observer". Both are segmentations by human observers trained by an experienced ophthalmologist. There is actually no real ground truth to compare your results to. So the "gold standard" is served as a pseudo-ground truth. You should compare your images to the gold standard. Then to evaluate the quality of your segmentation, you can compare it to how a well another human observer does compared to the gold standard.

Answer 2

[Walter Roberson]

Usually, the "gold standard" refers to the results that would typically be obtained by a highly trained human with years of experience.

[roya a]

Thanks for your answer. but my main problem comparing my results ( I mean segmentation blood vessel results) to gold standard or human observer?

[Walter Roberson]

The gold standard images have been prepared by human observers. The other images have been prepared by less expert but trained human observers. If you can match those images you will have done as well as could be expected without years and years of experience; if you can match the gold standard images then you will have done very well.

[Thorsten]

I have found no hint that for the DRIVE DB the gold standard was actually segmented by an expert and the other segmentation by a less trained human observer. Both are equally well trained, and one was taken as "gold standard". http://www.isi.uu.nl/Research/Databases/DRIVE/

[roya a]

Thanks for your answer. I saw this website before but I did not find further information about gold standard and also in some paper do not mention about it. They said we test our result on DRIVE database. I wanted to know they compare their result to which part of Drive database. because only 20 images is segmented in gold standard part and we can see 40 images in this DB. thanks