From Sahih Bukhari (Hadith 4484):
Narrated `Aisha:(The wife of the Prophet) Allah's Messenger (ﷺ) said, "Don't you see that when your people built the Ka`ba, they did not build it on all Abraham's foundations?" I said, "O Allah's Messenger (ﷺ)! Why don't you rebuild it on Abraham's foundations?" He said, "Were your people not so close to (the period of Heathenism, i.e. the Period between their being Muslims and being infidels), I would do so." The sub-narrator, `Abdullah bin `Umar said, "Aisha had surely heard Allah's Messenger (ﷺ) saying that, for I do not think that Allah's Messenger (ﷺ) left touching the two corners of the Ka`ba facing Al-Hijr except because the Ka`ba was not built on all Abraham's foundations
Reference: Bukhari 4484