Abstract: Objective To explore the feasibility of using the bidirectional local distance based medical similarity index (MSI) to evaluate automatic segmentation on medical images. Methods Taking the intermediate risk clinical target volume for nasopharyngeal carcinoma manually segmented by an experience radiation oncologist as region of interest, using Atlas-based and deep-learning-based methods to obtain automatic segmentation respectively, and calculated multiple MSI and Dice similarity coefficient (DSC) between manual segmentation and automatic segmentation. Then the difference between MSI and DSC was comparatively analyzed. Results DSC values for Atlas-based and deep-learning-based automatic segmentation were 0.73 and 0.84 respectively. MSI values for them varied between 0.29~0.78 and 0.44~0.91 under different inside-outside-level. Conclusion It is feasible to use MSI to evaluate the results of automatic segmentation. By setting the penalty coefficient, it can reflect phenomena such as under-delineation and over-delineation, and improve the sensitivity of medical image contour similarity evaluation.