From Sahih Muslim (Hadith 5696):
Abdullah reported Allah's Messenger (ﷺ) as saying:If you are three, two amongst you should not converse secretly between yourselves to the exclusion of the other (third one), unti some other people join him (and dispel his loneliness), for it may hurt his feelings
Reference: Muslim 5696