From Sahih Muslim (Hadith 3248):
Abu Qaza'ah reported that while Abd al-Malik b. Marwan was circumambulating the Ka'ba he said:May Allah ruin Ibn Zubair that he lies in attributing to the Mother of the Faithful, as he says: I heard her stating that Allah's Messenger (may'peace be upon him) had said: 'A'isha, if your people had not been new converts to Islam, I would have demolished the House and would have added (in it area) from the Hijr for your people have reduced the area from its foundations. Harith b. 'Abdullah b. Abu Rabi'a (Allah be pleased with him) said: Commander of the Faithful, don't say that, for I heard the Mother of the Faithful saying this, whereupon he said: If I had heard this before demolishing it, I would have left it in the state in which Ibn Zabair had built it
Reference: Muslim 3248