From Sahih Muslim (Hadith 3297):
Al-'Ali' b. al-Hadrami reported Allah's Messenger (may peace he upon as saying:For a Mahijir, it is only three (days') stay at Mecca, after completing (the Hajj or 'Umra) that is allowed, and it seemed as if he was saying that he should not (stay) beyond this (period)
Reference: Muslim 3297