These are hypothetical scenarios. But even if I take your scenario, it's still problematic because now we're not only talking about direct harm to the person in question, but emotional harm to other people. Where do we draw the line? Immediate family? Extended family? Friends? And why do we choose that arbitrary line? What level of emotional harm should be considered acceptable, and to how many people? How about a hermit who doesn't have family and friends? I can go on and on.
It is not possible to construct a system of morality based on a single irreducible first principle. I used to be very active in the atheist community, but one of the reasons I quit being involved is because there existed a dogma among them that "morality is very simple, just use the golden rule!". It is not so.