From Sahih Bukhari (Hadith 4156):
Narrated Mirdas Al-Aslami:Who was among those (who had given the Pledge of allegiance) under the Tree: Pious people will die in succession, and there will remain the dregs of society who will be like the useless residues of dates and barley and Allah will pay no attention to them
Reference: Bukhari 4156