Ultimately I think this hurts A&E. But they were in a no win situation. if they don't take him off the left has a field day. If they do the right has a field day as they currently are.
I have no data on this, but given the nature of the show I wouldn't be surprised if the majority of their viewers fall into the "right" category. Interesting to me that they chose to piss THAT group off.