But threw thing is we've already been shown that not all the saviors are devoted to Negan. He rules by fear. Kill him and I'd bet more than a few of them would let things go if not outright turn on the others.
After that initial fight I absolutely agree. But in the moment when the bullets are flying I think they kill the threat to themselves and not take the chance. I even think a bunch of them would die. but in the end they win that fight. The time to kill him IMO was at the gate.
Rosita puts one in his head while the gate is shut. All you can see through the gate is shadow. Walk towards him with the rifle like you are opening up and bam! No more Negan. All the Saviors are outside the wall still. They retreat and maybe Spencer and Rosita die and 3-4 Saviors. Everyone takes a minute to think and some of the Saviors take off and the group splinters and breaks apart. Once they were inside it was to late.