Providing accurate predictions efficiently is vital for the success of recommender systems. There are various factors that might affect the quality of the predictions and online performance. Similarity metric used to determine neighbors is one of such factors. Therefore, given a set of metrics, determining and utilizing the best one is critical for the overall success of collaborative filtering schemes. We scrutinize several binary similarity measures in terms of accuracy and performance. We conduct various real data-based experiments in order to determine the best similarity measure. Our empirical outcomes show that Yule and Kulczynski metrics provide the best results.