From Sahih Muslim (Hadith 4773):
It has been narrated by Abu Huraira that the Prophet (may pceace be upon him) said:Banu Isra'il were ruled over by the Prophets. When one Prophet died, another succeeded him; but after me there is no prophet and there will be caliphs and they will be quite large in number. His Companions said: What do you order us to do (in case we come to have more than one Caliph)? He said: The one to whom allegiance is sworn first has a supremacy over the others. Concede to them their due rights (i. e. obey them). God (Himself) will question them about the subjects whom He had entrusted to them
Reference: Muslim 4773