It is not so simple. Moral abstracts like the dignity of life and body must extend farther than simple "who's getting hurt" calculation for them to have any coherence. Sex is the easiest example one can come up with. what's wrong with necrophilia? Or having non-penetrative sex with an infant who won't remember? Or drugging someone and having sex with them? But other non-sexual scenarios can be constructed as well. What's the problem of a policeman robbing the corpse of a murder victim? But then if consent is only a requirement if someone is getting harmed, then why not when the someone's harm is less than someone else's gain? By attaching a non-sequitur requirement, such as level of harm, to the axiom of "need for consent", you can erode it until it is utterly meaningless as moral principle.