What would people think of a man who backs down from a fight?

There are two extreme results.

First, he may be a coward because he is afraid of fighting. A real man should be brave enough to face any challange including fight.

The other result is that he has good emotional quality. He can control his temper when he face a nonsense fight.