From Sahih Bukhari (Hadith 1887):
Narrated Anas:(The people of) Bani Salama intended to shift near the mosque (of the Prophet) but Allah's Messenger (ﷺ) disliked to see Medina vacated and said, "O the people of Bani Salama! Don't you think that you will be rewarded for your footsteps which you take towards the mosque?" So, they stayed at their old places
Reference: Bukhari 1887