Trending @HCL; How to avoid calls from an interviewees?

Here is a scenario that I interviewed someone and had to reject him as he did not meet the expectation. The person from somewhere got my personal phone number and called me, I was busy at that time so could not talk but now I’m feeling how to avoid confrontation and it seems he knows many people in the organisation as he is an ex-employee too

Can someone advice